Understanding Statistics in the Behavioral Sciences
This page intentionally left blank
Understanding Statistics in ...
105 downloads
1584 Views
7MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Understanding Statistics in the Behavioral Sciences
This page intentionally left blank
Understanding Statistics in the Behavioral Sciences
by
Roger Bakeman Byron F. Robinson Georgia State University
2005
LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS Mahwah, New Jersey London
Senior Editor: Editorial Assistant: Cover Design: Textbook Production Manager: Text and Cover Printer:
Debra Riegert Kerry Breen Kathryn Houghtaling Lacey Paul Smolensk! Hamilton Printing Company
Camera ready copy for this book was provided by the authors
Copyright © 2005 by Lawrence Erlbaum Associates, Inc. All rights reserved. No part of this book may be reproduced in any form, by photostat, microform, retrieval system, or any other means, without prior written permission of the publisher. Lawrence Erlbaum Associates, Inc., Publishers 10 Industrial Avenue Mahwah, New Jersey 07430 www.erlbaum.com Library of Congress Cataloging-in-Publication Data Bakeman, Roger. Understanding statistics in the behavioral sciences / by Roger Bakeman and Byron F. Robinson, p. cm. Includes bibliographical references and index. ISBN 0-8058-4944-0 (casebound : alk. paper) 1. Psychology—Statistical methods—Textbooks. 2. Social sciences—Statistical methods—Textbooks. 3. Psychometrics—Textbooks. I. Robinson, Byron F. II. Title.
BF39.B325 2004 150'.1'5195—dc22
2004056417 CIP
Books published by Lawrence Erlbaum Associates are printed on acid-free paper, and their bindings are chosen for strength and durability. Printed in the United States of America 10 9 8 7 6 5 4 3 2 1
Disclaimer: This eBook does not include the ancillary media that was packaged with the original printed version of the book.
Contents
Preface
xi 1
1
Preliminaries: How to Use This Book
1.1 1.2 1.3
Statistics and the Behavioral Sciences Computing Statistics by Hand and Computer An Integrated Approach to Learning Statistics
1 3 12
2
Getting Started: The Logic of Hypothesis Testing
17
2.1 2.2 2.3 2.4
Statistics, Samples, and Populations Hypothesis Testing: An Introduction False Claims, Real Effects, and Power Why Discuss Inferential Before Descriptive Statistics?
17 20 24 30
3
Inferring From a Sample: The Binomial Distribution
31
3.1 3.2
The Binomial Distribution The Sign Test
31 39
4
Measuring Variables: Some Basic Vocabulary
45
4.1 4.2 4.3
Scales of Measurement Designing a Study: Independent and Dependent Variables Matching Study Designs With Statistical Procedures
45 48 49
5
Describing a Sample: Basic Descriptive Statistics
53
5.1 5.2 5.3 5.4
The Mean The Variance The Standard Deviation Standard Scores
54 60 63 66
V
CONTENTS
VI
6
Describing a Sample: Graphical Techniques
71
6.1 6.2
Principles of good design Graphical Techniques Explained
72 73
7
Inferring From a Sample: The Normal and t Distributions
83
7.1 7.2 7.3 7.4 7.5 7.6
The Normal Approximation for the Binomial The Normal Distribution The Central Limit Theorem The t Distribution Single-Sample Tests Ninety-Five Percent Confidence Intervals
84 87 91 92 93 98
8
Accounting for Variance: A Single Predictor
103
8.1 8.2
Simple Regression and Correlation What Accounting for Variance Means
103 113
9
Bivariate Relations: The Regression and Correlation Coefficients 117
9.1 9.2 9.3 9.4
Computing the Slope and the Y Intercept Computing the Correlation Coefficient Detecting Group Differences with a Binary Predictor Graphing the Regression Line
119 124 127 132
1O
Inferring From a Sample: The F Distribution
137
1O.1 1O.2 10.3 10.4 10.5
Estimating Population Variance The F Distribution The F Test The Analysis of Variance: Two Independent Groups Assumptions of the F test
137 140 142 149 152
11
Accounting for Variance: Multiple Predictors
155
11.1 11.2
11.3
11.4 11.5
Multiple Regression and Correlation Significance Testing With Multiple Predictors Accounting For Unique Additional Variance Hierarchic MRC and the Analysis of Covariance More Than Two Predictors
156 166 168 171 178
12
Single-Factor Between-Subjects Studies
181
12.1 12.2 12.3
Coding Categorical Predictor Variables One-Way Analysis of Variance Trend Analysis
182 194 197
13
Planned Comparisons, Post Hoc Tests, and Adjusted Means
201
13.1 13.2 13.3 13.4 13.5
Organizing Stepwise Statistics Planned Comparisons Post Hoc Tests Unequal Numbers of Subjects Per Group Adjusted Means and the Analysis of Covariance
203 205 206 210 212
CONTENTS
viii
14
Studies With Multiple Between-Subjects Factors
223
14.1 14.2 14.3 14.4
Between-Subjects Factorial Studies Significance Testing for Main Effects And Interactions Interpreting Significant Main Effects and Interactions Magnitude of Effects and Partial Eta Squared
224 233 235 238
15
Single-Factor Within-Subjects Studies
245
15-1 15.2 15.3 15.4
Within-Subjects or Repeated-Measures Factors Controlling Between-Subjects Variability Modifying the Source Table for Repeated Measures Assumptions of the Repeated Measure ANOVA
245 250 258 266
16
Two-Factor Studies With Repeated Measures
269
16.1 16.2 16.3 16.4
One Between- and One Within-Subjects Factor Two Within-Subjects Factors Explicating Interactions With Repeated Measures Generalizing to More Complex Designs
269 278 284 286
17
Power, Pitfalls, and Practical Matters
289
17.1 17.2
Pretest, Posttest: Repeated Measure Or Covariate? Power Analysis: How Many Subjects Are Enough?
289 295
References Glossary of Symbols and Key Terms Appendix A: SAS exercises Appendix B: Answers To Selected Exercises Appendix C: Statistical Tables
301 303 309 325
A. B. C. D.1 D.2 E.1 E..2 F.1 F.2
345 347 350 351 352 353 354 355 356
Critical Values for the Binomial Distribution, P = 0.5 Areas Under the Normal Curve Critical Values for the t Distribution Critical Values for the F Distribution, a = .05 Critical Values for the F Distribution, a = .01 Distribution of the Studentized Range Statistic, a = .05 Distribution of the Studentized Range Statistic, a = .01 L Values for a = .05 L Values for a = .01
Author Index Subject Index
357 359
This page intentionally left blank
Preface
There are at least three reasons why you might buy this book: 1. You are a student in the behavioral sciences taking a statistics course and the instructor has assigned this book as the primary text. 2. You are a student taking a statistics course and the instructor has assigned this book for enrichment and for the exercises it provides. 3. You are a behavioral science researcher who feels a need to brush up on your statistics and you find working on exercises at your own pace appealing. If you are an instructor, your reasons for assigning this book might be similar: 1. You assigned this book as the primary text because its unified and economic approach appeals to you. You also like the idea of presenting to students a book that they can truly master in its entirety. 2. You assigned another text because of the encyclopedic coverage of statistical topics it offers, but you also assigned this book because you thought the straightforward prose and the conceptually based exercises would prove useful to your students. When sitting in statistics classes or when trying to read and understand statistical material, too many otherwise intelligent and capable students and researchers feel dumb. This book is intended as an antidote. It is designed to make you feel smart and competent. Its approach is conservative in that it attempts to identify and present the essentials of data analysis as developed by statisticians over the last two or three centuries. But the selection and organization of topics and the manner in which they are presented result from a radical rethinking of how basic statistics should be taught to students whose primary concern lies with behavioral science research or clinical practice, and not with the formal, mathematical properties of statistical theory. This book is designed to develop your conceptual and practical understanding of basic data analysis. It is not a cookbook, although by reading IX
X
PREFACE the text and working through the exercises you will gain considerable practical knowledge and the ability to analyze data using multiple regression and the analysis of variance. We assume that your goal is basic statistical literacy—the ability to understand and critique the statistical analyses others perform, and the ability to proceed knowledgeably with your own data analyses. We also assume that you are far more interested in the results and empirical implications of your analyses than in the formal properties of the statistics used—and therefore will welcome the practical, intuitive approach to statistics taken here. Several sorts of audiences, from advanced undergraduates, to beginning and advanced graduate students, to seasoned researchers, should find this book useful and appealing. The primary audience we had in mind when writing it consisted of graduate students in the behavioral sciences, students who were required to take a statistics course as undergraduates, did reasonably well at the time, but frankly never quite "got it." When reading journal articles and other research reports, they tend to skip over the results section. They see ps and Fs and lots of other numbers and their eyes glaze over. They look forward to the required graduate statistics courses with a mixture of terror and loathing and cannot quite imagine how they will analyze data for their required thesis or dissertation on their own. We hope that such students will find this book something of a relief, perhaps even a delight, and as they read it and work through the exercises their past coursework will suddenly make sense in new ways and they will increasingly feel competent to read journal articles and to pursue data analyses on their own. For such students, the present volume can serve as the basis for a semester or quarter course. We recommend that as students read the text, they faithfully perform the exercises, and not skip over them as some readers might a results section in a research report. But we also recommend that readers seek out data sets relevant to their area of interest, and use them to practice the statistical techniques presented here. Usually it is not too difficult to borrow a small data set; if you cannot find one relevant to your area, a practice data set is provided on the accompanying CD. Any attempt to apply the principles learned here to new data is helpful. There is something particularly motivating when you create exercises for yourself and when they are relevant to your areas of interest. Similarly, learning is enhanced and consolidated immensely when, in the context of reading journal articles relevant to your own interests, you suddenly recognize that the author knows, and has used, what you have just come to understand. A second audience for which this book should prove useful consists of active and productive researchers. Such researchers, like the graduate students we described earlier, should also find this book a relief and a delight—and for most of the same reasons. As they read the text and work through the exercises, much of the data analyses they have been doing fairly mechanically will suddenly make more sense to them and they will be infused with a new sense of understanding and control. The exercises will serve them especially well. These are designed to be done at the reader's own pace, in the privacy of the office or home as desired. Researchers who read this text in order to upgrade their statistical understanding may miss the help and nudging that fellow students in a class often provide, but the book is intended to work equally well as a self-study guide and a classroom text. For all of these audiences, the radical redefining of the topics required for a basic understanding of statistics that this book exemplifies will matter. It is a cliche of modern architecture (usually attributed to Mies van der Rohe) that less is more. Jacob Cohen (1990), whose writings on statistics have had considerable influence on us, argued that the same principle applies equally well when
PREFACE
_xi
considering the application of statistics to the behavioral sciences generally. Certainly it is a precept that has guided us throughout the design and writing of this book. CONTENTS There are a staggering number of undergraduate statistics books on the market today, several dozen of which are designed for behavioral science majors, and a fair number designed specifically for graduate courses, many of which are encyclopedic in scope. Over the past several decades there has come to be a "standard content" for these texts, a content from which few texts deviate. We have come to believe that many of the topics presented in traditional behavioral science statistics texts are nice but not necessary for a basic understanding of data analysis as practiced by researchers— and that several of the more essential topics could be presented in a far more unified manner than is usual. In writing this book, we began by asking what students and researchers need to know in order to understand and perform most of the basic kinds of data analyses used in the work-a-day world of the behavioral sciences. We then set out to provide a conceptually based but practice-oriented description of the requisite knowledge. As a result, some, but by no means all, of the topics found in traditional texts are covered here. And although students are hardly harmed by exposure to the full array of traditional topics, the "less is more" approach exemplified here has several advantages, not the least of which is efficiency. Here students are exposed to fewer topics, but those presented possess considerable power, generality, and practical importance. Two relatively recent developments have made this book possible, one conceptual and one technical. Over the past few decades, there has been a growing realization that statistical topics formerly kept in quite separate compartments can be viewed in a unified way. Thus the analysis of variance is conveniently, and economically, viewed as simply another application of multiple regression. This unified view permits considerable savings. Students learn multiple regression, which is extremely useful in its own right, and at the same time— and with little additional cost— learn how to analyze data associated with the basic analysis-of-variance designs. Moreover, analysis of covariance— a topic that traditional experimental design texts make quite complex— is thrown in almost for free. The unified view has been expressed in journal articles for several decades (e.g., Cohen, 1968) and has greatly influenced general-purpose statistical computer packages. But only now is it beginning to have an influence on textbooks. The result can be a dramatic trimming of traditional topics with little sacrifice in data analysis capability. The technical development concerns the now almost universal availability of personal computers. We assume that readers will have access to a microcomputer and a spreadsheet program such as Microsoft's Excel and a statistical package such as SPSS or SAS. Statistical packages are powerful tools for rapidly conducting complex analyses, and any serious student or researcher should be familiar with their operation. The ease with which analyses can be carried out in a statistical package, however, can also promote the mechanical application of analytical procedures without • the background knowledge necessary to select the correct analysis and interpret the results appropriately. Spreadsheets, on the other hand, make the formulas used in an analysis explicit and allow for the exploration of the conceptual underpinnings of the analyses and their constituent formulas. Spreadsheets relieve the user of laborious computations; this allowed us to develop exercises that use more meaningful
_XII
PREFACE definitional formulas rather than the sometimes opaque computational formulas necessary when problem sets are to be worked by hand. Together, spreadsheets and statistical packages provide a ready means for teaching conceptual knowledge and concrete skills to readers interested in learning how to conduct and interpret analyses appropriately. For readers who have already had statistics, the first seven chapters should constitute a brief review of some important, introductory topics (hypothesis testing, descriptive statistics, the normal and t distributions). For students new to statistics, chapters 1-7 should provide a sufficient basis for the material to follow. The foundation of the unified view (simple and multiple regression, accounting for variance) is presented in chapters 8-11. Chapters 12-16 present the basic analysis-of-variance designs—single and multifactor factorial designs, including designs with repeated measures—and also discuss post hoc tests. Mastery of these topics should make it possible to understand most of the simpler analyses appearing in journal articles and to perform basic analyses of data. In addition, mastery of this material should adequately prepare readers to move on to more advanced texts and topics, such as the analysis of categorical data. The interested reader is referred to our companion to this text, Understanding Loglinear Analysis with ILOG (Bakeman & Robinson, 1994).
LEARNING TOOLS This book contains a number of features that should aid learning. Most chapters contain exercises, and answers to selected exercises are provided in an appendix. Many of the exercises use spreadsheets or statistical software. The computerbased exercises, which allow you to learn by doing, are a central and essential feature of this book. The spreadsheet exercises, in particular, promote conceptual understanding. Spreadsheets allow you to perform meaningful analyses quickly and efficiently and in conceptually informative ways. The spreadsheet-based exercises are almost as central to this book as the unified view of basic statistics. In fact, as you will see, the two dovetail remarkably. Naturally, we wish that all the examples and exercises were relevant to each reader's particular interests and concerns. We could never create enough different examples to cover the breadth of topics readers will bring with them. And so, instead of even attempting this, we have based most of the examples and exercises on just a few paradigmatic studies. Readers will rapidly become familiar with these few studies (the money-cure study, the lie-detection study, etc.), which should allow them to focus more on new statistical concepts as they are presented without being distracted by details associated with a different study. The spreadsheet exercises are a powerful learning tool and allow you to conduct basic analyses. When data sets are large or more complex analyses are necessary, however, it is useful to know how to use one of the more powerful statistical packages. To this end, the chapters include exercises that walk the reader through the procedures necessary to conduct the analyses using either SPSS or SAS. SPSS exercises are presented in the chapters, with corresponding SAS exercises in an appendix. Key terms and symbols, which usually are italicized when they first appear in the text, are defined in boxes throughout the chapters and are collected together into a glossary at the end of the book. In addition, most chapters contain considerable graphic and tabular information, including spreadsheet printouts and figures. Finally, a CD is provided that contains all of the spreadsheets used in the book, SPSS and SAS output and data files, and practice data sets.
PREFACE
xiii
ACKNOWLEDGEMENTS Several classes of graduate students at Georgia State University have read and used earlier versions of this book, and we have benefited greatly from their comments and their experience. In fact, without them—without their efforts, their needs, and their anxieties—we would probably never have come to write this book. Among our former students, we would especially like to thank Anna Williams, Kim Deffebach, Robert Casey, Carli Jacobs, and P. Brooke Robertson for their diligence, good humor, and many insightful comments. Carli Jacobs also assisted with creating the answers appendix and the SAS exercises, and Brooke Robertson assisted with the indexes. We would also like to thank our colleagues and friends for their many helpful comments and support (Roger Bakeman would especially like to thank Lauren B. Adamson, Josephine V. Brown, Kenneth D. Clark, Alvin D. Hall, Daryl W. Nenstiel, and James L. Pate, and Byron Robinson would especially like to thank Carolyn B. Mervis, Irwin D. Waldman, and Bronwyn W. Robinson). Roger Bakeman & Byron F. Robinson
This page intentionally left blank
Understanding Statistics in the Behavioral Sciences
This page intentionally left blank
1
Preliminaries: How to Use This Book
In this chapter you will: 1. Be introduced in a broad, general way to the uses of statistics in the behavioral sciences and the scope of this book. 2. Be introduced to a simple tool—a spreadsheet program (e.g., Microsofts's Excel)—that can perform statistical computations in a way that is both conceptually meaningful and practically useful. 3. Be introduced to an integrated approach to basic statistics, one that relies on multiple regression and emphasizes a few powerful ideas common to a wide range of basic statistical analyses. 4. Be introduced to some of the assumptions, notations, and strategies for learning used in this book. 1.1
STATISTICS AND THE BEHAVIORAL SCIENCES What is the effect of early experience with the mother on an infant's subsequent development? What sort of person makes a good manager? What kinds of appeals are most likely to affect a person's attitudes? Why do people buy a certain kind of soap? Why do some children do better in school than others? Are there programs that can affect children's academic performance? Are there therapies that can affect an adult's mental health? When seeking answers to questions like these, behavioral scientists almost always resort to statistics. Why is this true? Why should science in general, and the behavioral sciences in particular, be so dependent on statistics?
Why Behavioral Scientists Need Statistics Simplifying considerably, assume for the moment that the goal of any science is to understand a phenomenon well enough so that such characteristics as when it occurs and how often it occurs can be predicted with some precision. Some phenomena studied by scientists (e.g., simple physical events) might arise from only a few causes. If this were so, then it would be relatively easy to include in one study all the important causes, making it at least easier to understand and predict that phenomenon. Other phenomena (especially phenomena studied by
1
2
PRELIMINARIES: How To USE THIS BOOK behavioral scientists), however, might result from a multitude of causes. If important causes were not identified, or for any other reason were not included in a study, then prediction would suffer. But prediction could suffer for another reason as well. It may be that at least some complex phenomena are affected by genuinely random processes. For scientists who hold a deterministic view of human behavior, this is a nightmarish possibility, as William Kessen (1979) noted in a speech to the American Psychological Association (Kessen was speaking to child psychologists but what he said also applies to students of human behavior in general): To be sure, most expert students of children continue to assert the truth of the positivistic dream—that we have not yet found the underlying structural simplicities that will reveal the child entire, that we have not yet cut nature at the joints—but it may be wise ... to peer into the abyss of the positivistic nightmare—that ... the variety of the child's definition is not the removable error of an incomplete science, (p. 815) Kessen raised two possibilities that students of human behavior should consider: Predictions of human behavior may be imprecise because the behavioral sciences are not yet mature enough to identify all determining causes, or—and this is the "positivistic nightmare"—precise predictions may be impossible in principle because of random or chaotic elements in human behavior. An ancient and important philosophic question echoes in these two possibilities, but statistics are neutral in this debate. Prediction may fail to be perfect because important causes of a completely determined phenomenon were not taken into account. Or prediction may fail because indeterminate elements shape the phenomenon. In either case, the observed association of the presumed causes (or explanatory factors) with the resultant phenomenon will be less than perfect. When prediction is essentially perfect, statistics are not necessary. When prediction is not perfect, statistical techniques help us determine whether our observations should be ignored because they probably reflect just a chance occurrence, or whether they should be emphasized because they probably reflect a real effect. In other words, when results are ambiguous, statistical analysis allows us to determine whether or not the effects we observe are statistically significant. Thus, the need for statistical analysis is not a matter of whether it is physical or social phenomena that are under investigation, but rather a matter of how strongly effects are linked to their presumed causes, that is, of how well the phenomenon under study can be predicted. For example, when studying the effect of certain toxins on the body, predictability can approach 100%. If the toxin is introduced into a person's body, that person dies. In such cases, statistics are hardly needed. The effects of psychoactive drugs often appear to occupy a middle ground between complete determinism and complete chaos. Even though there are broad regularities, different drugs in different concentrations affect individuals differently—a state of affairs that can complicate both research and clinical practice. Behavioral regularities of the sort students of human behavior study are typically weaker than the effects of toxins or psychoactive drugs. Usually the rules uncovered by behavioral research suggest probabilities, not certainties, which can be quite frustrating for judges, clinical psychologists, and others who must make decisions in individual cases. The behavioral regularities may be real, but often they are not all that powerful and there will be many exceptions to the general rules. And, from the point of view of the researcher, many behavioral
1.1 STATISTICS AND THE BEHAVIORAL SCIENCES
3
regularities may not be at all obvious to the naked eye. These regularities are probabilistic and the evidence for them may appear ambiguous, so skeptics will naturally suspect that an invested researcher who claims effects is led more by desire than evidence. Statistical techniques are important, indeed essential, because they provide ways to resolve such ambiguities. Thus, to return to the question posed at the beginning of this section, scientists need statistics when they deal with phenomena that can be predicted only imperfectly. Behavioral scientists make a career of dealing with imperfectly predicted phenomena and thus often have greater need of statistics than other kinds of scientists. But any scientist, when faced with ambiguous results, needs techniques that control for the all-too-human tendency to see what we want or expect to see. Happily, statistical techniques provide ways of making decisions about ambiguous evidence, ways that insulate those decisions from an investigator's potential bias. No wonder, then, that statistics has come to play an especially important role in the thinking and research of behavioral scientists.
How Much Statistical Knowledge Do Behavioral Scientists Need? Few would deny that behavioral scientists need some knowledge of statistics. However, there is room for a considerable diversity of opinion as to how much knowledge is enough. Certainly they need to be able to read journals in their field and analyze their data. At a minimum, diligent readers of this book, who read the material and faithfully do the exercises, should be able to understand much of what they find in the results sections of journal articles and should be able to perform and present competent data analyses of their own. In no sense is this book a compendium of all the statistical knowledge a behavioral scientist will ever need, but it will provide a basis for learning about more advanced topics like multivariate analysis of variance, advanced multiple regression, advanced experimental design, factor analysis, log-linear analysis, logistic regression, and the like later on. This book is perhaps best viewed as an advanced-level introductory text. We assume that many readers will have had an undergraduate introductory statistics course but they are not now comfortable applying whatever they learned earlier. In fact, as noted in the preface, that earlier material may be remembered only in the most vague and shadowy of forms and as a result cannot form the basis for confident practice. Reading this book and working its exercises should allow previous knowledge to coalesce with present learning and provide many "moments of recognition" when terms and concepts heard before suddenly make sense in a way they had not previously. One caveat is necessary: It is as important for us to realize what we do not know as it is to understand what we do know. An attempt is made to present topics in this book in as straightforward and simple a manner as possible. Yet any one of them could be elaborated and, if probed, might reveal subtleties, complexities, and depths not even hinted at here. Such successive refinement seems central to academic work and to scholarly endeavor generally. We hope this book provides you with a sufficiently elaborated view of basic statistics to allow you to realize what a small corner of the statistical world you have conquered. 1.2
COMPUTING STATISTICS BY HAND AND COMPUTER We learn new skills—from broad jumping to piano playing to statistics—best by doing. Watching others perform any of these activities may be helpful initially,
4
PRELIMINARIES: How To USE THIS BOOK but in the end there is no substitute for individual exercise. Thus, watching an instructor perform statistical computations on a blackboard may be helpful at first, but at some point we need to work through computations ourselves, at our own speed. The question then is, what is the best way to practice the required computations? And, once mastered, what is the best way to do the computations required for a real data analysis? Traditionally, pencil and paper (or something similar) have been used. Later, mechanical calculators offered some relief from the tedium and error-prone ways of hand methods. These have now been almost totally replaced by amazingly inexpensive and seemingly ubiquitous hand-held electronic calculators. At the same time, personal computers have become as commonly used by students as typewriters once were, and computer labs have sprouted apparently overnight on most college campuses. It is now reasonable to assume that students will have easy access to microcomputers. In fact, this book assumes that readers will use a personal computer for computations, sometimes when reading the text, certainly when doing the exercises. In addition to accuracy and speed, personal computers offer advantages not easily realized even with electronic calculators. Historically, considerable space in statistics texts, and considerable student time, was devoted to deriving computational from theoretical formulas. The computational formulas were then used for all subsequent exercises. In other words, the formulas used in exercises were designed for hand omputation but were not necessarily conceptually meaningful. The only justification for this state of affairs was practical. Given earlier technology, theoretical formulas were simply too cumbersome to use. But with the advent of personal computers, this has all changed. It is simply no longer necessary for anyone (other than specialists who develop the general-purpose computer packages the rest of us use) to know computational formulas. For basic work, the definitional methods presented in this book can be used, and for more advanced work, several excellent packages are available, such as SPSS. As a happy result, less rote learning is required and concepts, not mechanics, can be used to solve both the exercises presented in this book and practical problems as well. The advantages for the reader are considerable. Free from the need to spend time mastering unnecessary details, the same degree of statistical mastery now takes less time. But it is the personal computer, specifically a personal computer running a spreadsheet program, that makes such an approach practical.
Spreadsheets: A Basic Computing Tool for Statistics The computer programs that spurred so many sales of personal computers initially were electronic spreadsheets and today, after word processors, they remain the second most common kind of application run on personal computers. Although designed initially for business applications, it turns out that spreadsheets are extremely well-suited for statistical use as well. The idea behind spreadsheets—like so many innovative ideas—is astonishingly simple. A spreadsheet consists of rows (usually labeled 1, 2, 3, etc.) and columns (usually labeled A, B, C, etc.) that together form a field or matrix of individual cells (identified as A1, A2, B1, etc.; see Fig. 1.1). Among other things, cells can contain formulas or arithmetic rules. For example, a cell might contain a formula for adding together all the numbers in the column above it. That cell would then display the total for the column. Several entries in a row might all contain formulas for column totals and an additional formula in that row might sum the column totals, providing a grand sum of all the numbers in the table.
1.2 COMPUTING STATISTICS BY HAND AND COMPUTER
_5
Rows can represent such diverse items as expense categories for a small business, lines in a tax form, participants in an experiment, or students in a class. Columns can represent months or years, interest or other kinds of rates, or test scores or scores of any kind. The whole might then represent a model for a small business, a tax computation, a statistical analysis, or a class grade book. Whatever is represented, spreadsheets typically consist of linked and interdependent computations, which means that a change in one cell will affect the values of all the other cells that depend on it. For example, if cell B3 in Fig. 1.1 were changed from 12 to 22, the column total in cell B5 would change from 52 to 62, the row total in cell F3 would change from 67 to 77, and the total of the column totals in cell F5 would change from 232 to 242. Electronic spreadsheets ensure that making such changes, whether they result from incorrect entries or simply a desire to experiment with different values, does not necessitate a series of frantic erasures and error-prone hand recalculations. Instead, the effects of a change in the value of one cell can ripple through the spreadsheet seemingly instantaneously as the computer performs the required recalculations. This means that a template (a particular set of labels and formulas) can be developed for a series of calculations (e.g., a particular statistical analysis) and then reused with different data by entering the new data into the cells in place of the old data. As though this were not enough, the use of spreadsheets for statistical analysis offers yet another distinct advantage. For whatever reason, many students encountering statistics for the first (or even second) time find the usual sort of statistical notation—subscripts, superscripts, big and little Greek letters, various squiggles—intimidating or worse. Too often, notation seems to form a barrier that impedes the understanding of too many students. Much of the notational burden is carried by the spreadsheet itself because of the graphic way spreadsheets are laid out with rows representing subjects (i.e., participants or, more generally, cases) and columns representing various statistical elements. As a result, in this book the need for elaborate statistical notation is greatly reduced and the notation remaining is relatively straightforward and its meaning easily tied to the spreadsheet layout. Moreover, given mastery of a spreadsheet approach to statistics as presented in this book, readers should then have little trouble understanding and applying more complex notation in subsequent, more advanced courses. Another advantage of a spreadsheet approach to basic data analysis should be mentioned. Although there is already a limited graphic aspect to data laid out
1 2 3 4 5 6 7
A
B
C
D
E
F
Name Alex Betty Carlos Total N Mean
Test 1 21
Test 2
Test 3
Test 4
Total
22 16 18
19 20 21
23 19 22
85 67 80
56 3
60 3
64
232
12 19 52 3
17.3
18.7
20.0
3
21.3
3
77.3
FIG. 1.1. An example of a simple spreadsheet. For example, cells in row 1 and column A contain labels, cell B3 contains the value 12, cell B5 contains a formula for the sum of the three cells above it, and cell B7 contains a formula for dividing cell B5 by B6 to get the mean.
6
PRELIMINARIES: How To USE THIS BOOK in a spreadsheet format, other kinds of graphic portrayals can considerably enhance understanding of our data. Fortunately, most spreadsheet programs have some graphing capability, and in subsequent chapters we demonstrate just how useful various graphs produced by spreadsheet programs can be for understanding the concepts as well as the results of data analysis.
A Brief Introduction to Spreadsheet Programs It is characteristic of tools, from simple hammers to high-speed electronic calculators, that advantages are reaped only after an entry price is paid. The advantages of spreadsheets when learning and doing statistics is clear. They automate calculations, reinforce concepts, and simplify notation. Yet to gain these advantages, first the tool needs to be mastered. In the case of spreadsheets, this entry cost is neither unduly high nor completely negligible. It takes some effort to learn how to use spreadsheets, more than for a hand-held calculator, but less than for a typical word processor. But the payoff can be considerable, not just for statistics but for a host of other calculating tasks as well. Many of you are probably already reasonably expert with spreadsheets, but if you are a complete novice, do not despair. Spreadsheets are not very difficult to master. Almost all spreadsheets programs are remarkably similar, far more similar than word-processing programs. Whatever spreadsheet programs you use, your first task is to become familiar with it, a process that is usually aided by the tutorial programs provided with many programs. Nonetheless, some general orienting comments that apply to spreadsheets in general may be helpful. Specific examples are also provided. These examples follow the conventions of Microsoft's Excel, which is one of the most widely available spreadsheets. But even if you use a spreadsheet with somewhat different conventions, this combination of general discussion and specific examples should prove helpful. As already noted, a spreadsheet consists of a matrix of rows and columns. Columns are identified with letters, rows with numbers. Each cell in the matrix has an address that identifies that cell. For example, A1 is the address for the cell in the upper left-hand corner and cell C5 is over two columns and down four rows from A1. When the program is first invoked, one cell will be highlighted. The highlight can be moved by pressing the directional arrows. A few moments of experimentation with your spreadsheet program should demonstrate how this works. Initially, all cells of the spreadsheet are empty, but potentially any cell can contain a (1) value, (2) label, or (3) formula. In other words, cells can contain numbers, identifying information, or computational rules. To enter something in a cell, first highlight that cell. Then simply type. If the first letter you type is a number, most spreadsheets assume you mean a value. If the first letter you type is a letter (or a quote mark), most spreadsheets assume you mean a label. A label can be any identifying information you want and is usually used to make the spreadsheet readable. Conventions for formulas can vary, but the usual assumption is, if the first character is an equals sign a formula follows.
1.2 COMPUTING STATISTICS BY HAND AND COMPUTER
7
Spreadsheet Formulas and Functions Whatever the convention used for your spreadsheet, formulas are very general and powerful and it is important to understand how they work. If a number is entered in a cell, that number is displayed on your spreadsheet. For example, in Fig. 1.2 you see the number 12 in cell B3. Similarly, if a label is entered in a cell, that label is displayed on your spreadsheet, for example, "Alex" in cell A2. But if a formula is entered in a cell, the spreadsheet displays the value rather than the formula. For example, in Fig. 1.2 we have indicated the formula that is in cell B5, =SUM(B2:B4), but in fact the value 52 and not the formula would be displayed in cell B5 on your screen (when you highlight a cell that contains a formula, the formula is displayed, usually in a special box at the top of your screen). If the value of any cell used in computing that formula is changed subsequently, the formula will be recomputed and the new value displayed, which, as noted earlier, gives spreadsheets a notable advantage over pencil-and-paper techniques. A formula can indicate simple arithmetic. For example, =B5/B6 entered in cell By divides the cell two above By by the cell immediately above it (see Fig. 1.2). A formula can be as simple as a pointer to another cell. For example, =F7
entered in cell F11 would cause cell F11 to display the value of cell F7. Or a formula can combine values in other cells, constants, or both as required. For example, =F6-3 entered in cell G6 would subtract a constant value of three from the value of. the cell immediately to the left of G6. Or, =(G5*3.14)/F6 entered in cell G6 would multiply the value of the cell immediately above by a constant value of 3.14 and would then divide the product by the value of the cell
A
1 2 3 4 5 6 7
B
Name Test 1 Alex 21 Betty 12 Carlos 19 Total =SUM(B2:B4) N=COUNT(B2:B4) Mean =B5/B6
C
D
E
F
Test 2
Test 3
Test 4
Total
22 16 18 56 3 18.7
19 20 21 60 3 20.0
23 19 22 64 3 21.3
85 67 80 180 3 60.0
FIG. 1.2. The spreadsheet shown in Fig. 1.1 with Column B expanded to show the formulas in Cells B5, B6, and B7. Row 1 and Column A contain labels, Cells B2 through E4 contains numbers, Rows 5-7 and Column F contain formulas.
PRELIMINARIES: How To USE THIS BOOK
8
immediately to the left of G6. In other words, formulas can combine cell address and constants with operators. Operators include: 1. 2. 3. 4. 5.
+ / * ^
(for addition) (for subtraction) (for division) (for multiplication) (for exponentiation)
Left and right parentheses are used to group addresses and constants as needed to indicate the order of computation. For example, =A7+B7/B8 is not the same as =(A7+B7)/B8. A formula can contain any address except that of its host cell, because this would result in a circular definition. A formula can also indicate a function, or can combine functions with cell addresses, constants, or both. Functions are predefined and the particular functions included vary some among spreadsheets. Only three functions are required for the exercises in this book: 1. =SUM(range) 2. =COUNT(range) 3. =SQRT(cell) although others you may find useful include: 4. =AVERAGE(range) 5. =STDEV(range) (more on the standard deviation function later). For example, =SUM(D2:D4) sums the numbers in column D in rows 2 through 4. And =COUNT(D2:D4) counts the number of cells containing numeric values, which in this case (assuming there is a value in each of the cells in the range indicated) would be 3 Finally, =SQRT(H23)or=H23^.5 computes the square root of whatever value is in cell H23. (Recall that the square root of a number is the same as that number raised to the one-half power. Thus =SQRT(H23) and =H236^ .5 give identical results.) One final note: We have shown functions here with capital letters to make them stand out, but function names are not case sensitive. =SUM and =sum have the same effect. Some functions operate on a single cell, like the square root function; others, like the summation function, operate on a range of cells. For example, C1:C10 specifies a range incorporating 10 cells in column C. Again, D8:H8
1.2 COMPUTING STATISTICS BY HAND AND COMPUTER
_9
specifies a range of five cells in row 8. Similarly,
A1:D5 indicates a rectangular range, four columns across by five rows down, containing 20 cells. The ability to specify a range turns out to be very useful. Not only can a range be specified for a function in a formula, a range, like an individual cell, can also be copied or moved (more on this later). Further, you do not need to type in the range explicitly. For example, you might point to a cell, select it with a left mouse click, begin typing "=sum(" and then, holding the left mouse burton down, "paint" a range of cells. This range (whether a singe column, a single row, or a rectangle of cells) will be entered into the cells for you.
Relative Versus Absolute Addressing in Spreadsheets Spreadsheets distinguish between relative and absolute addresses, for both individual cells and ranges. This distinction may seem rather academic at first, but in fact the flexibility and power of spreadsheets depends in large measure on the way addresses are copied, preserving relative addresses. When a formula is copied from one cell to another, the addresses in the new cell are relocated, preserving the relative, not the absolute address. For example, if the formula in cell By, which indicates that the value in cell B5 is to be divided by the value in cell B6 (=B5/B6), is copied to cell C7, the addresses are relocated so that the formula in cell C7 is now =C5/C6, exactly as you would like if you wanted to copy this formula for computing the mean (see Fig. 1.2). What is preserved is the relative address (like the old formula, the new formula refers to the two cells immediately above it), not the absolute address (B5 and B6 in the original formula). This allows for considerable efficiency in defining spreadsheets. For example, a formula that operates on other values in its row can be copied to cells in the same column, and each of those formulas will then operate on the values in the appropriate row. In addition, if a formula in cell C12 indicates summation for cells Ci through C10, and if the formula in C12 is copied to cells D12 through G12, then the formulas in cells D12 through G12 will indicate summation for rows 1 though 10 for that column. All of this is extremely helpful for statistical analysis. Raw data for behavioral science studies often consist of scores for different subjects. (When discussing spreadsheets, we use the briefer term subjects, although usually participants would be used when writing results for publication.) Each row could represent a different subject. Subject identification numbers might then be entered in column A and raw data for these subjects in column B. Data for the first participant might be on row 1, for the second participant on row 2, and so forth. Various statistical manipulations of the raw scores could be accomplished in various columns, and summations for all columns could be entered in an appropriate row after the data for the last subject. Formulas need be entered fully only once and then copied. Taking advantage of the way copying works allows spreadsheet users to rapidly create a spreadsheet template that performs the required statistical computations. Sometimes, of course, we want an address in a formula to be copied absolutely, not relatively, which is easy enough to do. An address, like D13, is always understood to be relative, unless the column letter and row number is preceded by a dollar sign, like $D$13. For example, the formula, =B1-$B$24
_10
PRELIMINARIES: How To USE THIS BOOK entered in cell D1 indicates that the value in cell B24 is to be subtracted from the value in the cell Bl, the cell two columns to the left of D1. If this formula is copied to cell D2, the formula in cell D2 will be =B2-$B$24 The address for Bl was relocated—the first term in the initial formula and in the copied formula points to a cell two to the left of the host cell—but the address for B24 was not relocated (because of the dollar signs). In other words, the initial formula in cell D1 indicates relative addressing for the first term but absolute addressing for the second. So far we have talked about copy, but move is a different matter. When a cell or a range is copied, addresses are relocated, preserving their relative position (unless the formula being copied indicates an absolute address). When a cell or a range is moved, on the other hand, the addresses are not relocated, but the initial addresses are preserved instead. Furthermore, and consistent with what the words copy and move usually mean, after a copy two versions exist, the initial cell or range and the copied cell or range. After a move, however, only the moved cell or range exists; the initial cell or range is erased. Copying is very useful when you want to expand a formula throughout a table, whereas moving is useful when you want to reorganize the layout of your spreadsheet, perhaps to make room for new variables or formulas. The number of significant digits maintained internally by spreadsheet programs for a particular cell can vary, depending on the particular computer, but typically the internal representation is accurate to several digits. Consequently, accuracy is not usually a concern of spreadsheet users. The number of digits displayed on your screen, however, will depend on the column width you allow and the format you have specified. For example, in Figs.1.1i and 1.2, we specified a numeric format with one digit after the decimal point for row 7. Consequently cell C7 displays 18.7, but the number held internally, and used for any computations, is 18.6666... Again, it is important to distinguish between what a cell contains and what a cell displays. If your answers to the exercises in this book do not agree exactly with the answers given in the figures or in the Answers to Selected Exercises section, do not be unduly alarmed. The discrepancy may be due simply to the format of the display. The brief introduction to spreadsheets contained in the preceding paragraphs is far from exhaustive and most spreadsheets have many more capabilities than those mentioned here. Moreover, we have not discussed how to save a spreadsheet once defined or how to retrieve that spreadsheet for subsequent use, nor have we discussed machine-specific or systems-level matters, for example, how one invokes the spreadsheet program in the first place. Still, we have mentioned most of the spreadsheet capabilities necessary for the exercises and demonstrations contained in this book. As a next step, and as preparation for material presented later, readers unfamiliar with spreadsheets should now spend some time familiarizing themselves with whichever spreadsheet they plan to use. Exercise 1.1 Using a Spreadsheet Program This first exercise provides brief and introductory practice in using a spreadsheet program. If you are not sure how to perform any of these operations, ask someone to show you or experiment with the help provided by the program.
1.2 COMPUTING STATISTICS BY HAND AND COMPUTER
11_
1. Invoke your spreadsheet program. Experiment with moving the highlight around using the cursor control keys (marked with arrows). 2. Enter five numbers in rows 2-6 of column B. Enter five different numbers in rows 2-6 of columns C and column D. Enter a label for column B (like "Test 1") in row 1 of column B; likewise for columns C and D. Enter labels in column A as appropriate. 3. Enter a formula in cell B7 that sums the five numbers in B2 through B6. Copy the formula from cell B7 to cells C7 through D7. Are the computed values in row 7 correct? 4. Enter a formula in cell E2 that sums the values in row 2. Copy this formula from cell E2 to cells E3 through E6. Are the totals in these cells correct? Change one of the numbers in the range B2:D6. Are the totals updated correctly? 5. Enter a formula in cell B8 that counts the numbers in B2 through B6. Copy the formula from cell B8 to cells C8 through D8. Are the computed values in row 8 correct? Delete one the numbers in the range B2:D6. Is the count updated correctly? 6. Insert three blank rows between rows 5 and 6 (this moves row 6 to row 8). Enter numbers in some of the blank cells in what are now rows 5 and 6. Are the sums and counts updated correctly? 7. Print a hard copy of your spreadsheet (assuming that a printer is attached to your microcomputer). 8. Save the template you have created, giving it some meaningful name. Exit the spreadsheet program.
A Brief Introduction to SPSS A spreadsheet is a powerful tool for manipulating data and conducting basic statistical analyses. Spreadsheets also allow students to explore the inner workings of meaningful definitional formulas and avoid the drudgery of hand calculations using opaque computational formulas. All of the computations described in this book can be accomplished by most spreadsheet packages with a regression function. As the analyses you wish to conduct become larger (e.g., contain many cases) and more complex (e.g., analysis of repeated measures), however, it will be to your advantage to learn a computer package dedicated to statistical analysis. To this end we have included exercises using SPSS (and SAS) that parallel and extend the spreadsheet exercises. SPSS is a system of commands and procedures for data definition, manipulation, and analysis. Recent versions of SPSS use a Windows-based menu and a point-and-click-driven interface. We assume the reader has basic familiarity with such Windows-based program features. SPSS includes a data interface that is similar in many respects to a spreadsheet. Rows represent cases and columns represent different variables. You can manipulate data using cut, paste, and copy commands in a manner similar to spreadsheets. SPSS also provides the ability to import spreadsheet files directly into the SPSS data editor. The SPSS exercises will provide enough information to navigate the commands and procedures necessary to complete each exercise. You should, however, consult the SPSS manuals and tutorial to gain a more thorough background in all of the options and short-cuts available in SPSS. The following exercise will familiarize you with the SPSS interface and basic commands (comparable SAS exercises for this and subsequent SPSS exercises are contained in an appendix).
12
PRELIMINARIES: How To USE THIS BOOK Exercise 1 .2 Using SPSS This first exercise provides brief and introductory practice in using SPSS. If you are not sure how to perform any of these operations, ask someone to show you or experiment with the help provided by the program. Prior to importing data into SPSS, open the spreadsheet form Exercise 1.1, delete lines 5-7 (leaving just data), save the file with a new name. 1. Invoke SPSS. Open the spreadsheet file you just created (data from Exercise 1.1). From the main menu, select Flle->Open->Data. In the Open File dialog box find the Files of Type window and change the file type from an SPSS (.sav) file to and Excel (.xls) file. Find your spreadsheet file and open it by either double clicking on the filename or highlighting the file and clicking Open. In the open data source window, make sure that Read variable names in the first row of data is checked and then click on OK. Confirm that the variable names and data are the same as your spreadsheet. 2. Change the name of the variable in column A. At the bottom of the SPSS data editor, click on the Variable View tab. In variable view, experiment by changing the name of one of the variables. Note that SPSS variables are limited to eight characters. You can, however, click on the Label column and provide a more descriptive label that will appear in the output. Labels can be up to 256 characters in length, but be aware that labels that are too long will make the output difficult to read. 3. Create a new variable by entering a new name in the Name column. Designate the number of decimal places you would like to display and give the new variable a meaningful label. 4. Enter data for the new variable. Click on the Data View tab and enter five values in the rows below the new variable name. 5. From the main menu, select Analyze-> Descriptive Statistics ->Descriptives. In the Descriptives window, highlight all three variables in the left-hand window and move them to the right hand window by clicking on the right-pointing triangle. Click on the Options button, check the box next to Sum. Click Continue and then OK. This should open the output viewer and display basic descriptive statistics. Confirm that the N (i.e., count) and sum are the same values you obtained in Exercise 1.1. 6. Return to the data editor and change some of the values. Run descriptive statistics again. Were the descriptive statistics updated correctly? 7. Save the output and data files. To save the output file, make sure you are in the SPSS output viewer, and then select File->Save from the main menu. Give the file a meaningful name and click Save. The output will be saved in a file with an .spo extension. Now return to the data editor and save the data using a meaningful name. SPSS data files are saved with a .sav extension. Exit SPSS.
1.3
AN INTEGRATED APPROACH TO LEARNING STATISTICS To a large extent, statistical practice has developed separately in a number of different academic disciplines or areas, many of which hardly spoke the same language. Often students in some areas would learn only analyses of variance and students in other areas would learn only multiple regression. And even when
1.3 AN INTEGRATED APPROACH TO LEARNING STATISTICS
13_
students learned about both, the two would typically be presented as quite unrelated topics. As a result, students in different areas learned seemingly separate statistics in different ways. Even introductory statistical texts often presented statistics in a piecemeal fashion, as a series of disparate topics. During the last several decades there has been a growing realization that considerable commonality underlies seemingly different statistical approaches, that much of what constitutes basic statistical analysis can be understood as applications of a general linear model. This has fortunate implications for the student new to statistics. Formerly, for example, multiple regression was typically taught as a predictive technique, appropriate for correlational studies in education and business, whereas the analysis of variance was taught as a way to analyze experimental results in agriculture and psychology. Increasingly, we now understand that both use the same concepts and computations, and analysis of variance is simply a specific application of multiple regression, which is the more general or overarching approach. As Cohen and Cohen (1983) noted in their excellent text on multiple regression, multiple regression/correlation is a general data-analytic system, "a versatile, all-purpose system for analyzing the data of the behavioral, social, and biological sciences" (p. 4). Reflecting this new understanding, correlation, multiple regression, and the analysis of variance for standard experimental designs are treated here in a single unified and integrated way. This results in considerable economy. Students confront fewer separate topics, but those they do encounter possess broad generality and application, which is consistent with the general "less-is-more" philosophy on which this book is predicated. Earlier in this chapter we argued that only theoretical, not computational, statistical formulas need be learned. It may not be necessary to learn many of the traditional theoretical formulas either—at least not in an introductory course—because an integrated approach renders many of them unnecessary. Thus, an integrated approach reduces the amount to be learned but does not sacrifice understanding of basic statistical concepts or limit the kinds of analyses that can be performed. This may sound like an extravagant claim to you now (although certainly appealing), but once you understand the material and work the exercises in this book, you should be in a position to judge for yourself.
Multiple Regression: A Second Basic Computing Tool The conceptual framework for the statistical analyses presented in this book is provided mainly by multiple regression. Both concepts and computations for particular analyses are embodied in spreadsheets. The two make for a happy marriage, almost as though spreadsheets were invented to make a multipleregression approach to basic statistical analysis comprehensible and practical. As demonstrated repeatedly throughout this book, the layout of spreadsheets allows the logic of multiple regression to be expressed faithfully and, as noted earlier, with a minimum of intimidating notation. By way of introduction, it seems worthwhile at this point to describe multiple regression in a very brief and general way. In introducing multiple regression/correlation, Cohen and Cohen (1983) wrote that it "is a highly general and therefore very flexible data-analytic system that may be used whenever a quantitative variable (the dependent variable) is to be studied as a function of, or in relationship to, any factors of interest (expressed as independent variables)" (P- 3). An amazing number of research questions in the behavioral sciences fit this simple description. Usually there is something we want to explain, or account for (like IQ scores, income, or health), which we want to account for in terms of
14
PRELIMINARIES: How To USE THIS BOOK various factors. To be sure, a study that attempts to account for a categorical instead of a quantitative dependent variable (these terms are defined in chap. 4) does not fit this description, but techniques for analyzing categorical variables are mentioned in chapter 3. Moreover, many of the same concepts apply whether the dependent variable is categorical or quantitative. Thus the material presented here applies broadly to most simple statistical analytic techniques. Multiple regression, used as a basic computing tool, can be understood in black-box terms. Without even knowing what is inside the black box, or how it works, we can simply provide the black box with the appropriate information. We would tell it what we want to account for (the dependent variable or DV), what the factors of interest are (the independent variables or the IVs), and what the independent and dependent variable scores are for the subjects in our study. In other words, we would provide the multiple regression program with a rectangular data matrix. Rows would represent subjects (or some other unit of analysis like a dyad or a family), the first column would represent the dependent variable, and subsequent columns would represent independent variables. The matrix would then consist of scores for individual subjects for the various variables. For its part, the multiple-regression program would return to us coefficients or "weights" for each independent variable, indicating the relative importance of each, along with an overall constant and other statistics. The coefficients are then used as part of the basic machinery for carrying out the statistical analyses presented in this book. Multiple-regression statistics are discussed in greater detail in chapter 11. As a practical matter, and in order to perform the exercises throughout this book, the reader needs some way of determining multiple regression coefficients. If there is only one independent variable, the regression statistics can be computed directly in the spreadsheet according to formulas given in chapters 8 and 11. If there is more than one, it becomes much easier to let a general-purpose multipleregression program do the computational work. Happily, most spreadsheet programs have an internal or built-in ability to do multiple regression. (In Excel, specify Tools | Data Analysis, usually after a onetime Tools | Add-ins specifying the Analysis ToolPak.) All you need do is request regression and then specify the range for the dependent variable, the range for the independent variable or variables, and the place in the spreadsheet where you wish the regression output placed. And for more advanced cases, and also as a check on the spreadsheet exercises and operations, you will use the multipleregression procedure in a statistical package such as SPSS. However, do not let the technical details of how, or with which program, you will compute multiple-regression statistics obscure the power and sweep of the argument being made here. Two basic computing tools have been discussed. Multiple regression provides coefficients. These coefficients are then incorporated in spreadsheets, which complete the necessary statistical computations. As you read this book and come to understand the integrated approach pursued here, you will also be developing specific spreadsheet templates that allow you to carry out the analyses you have learned. In order to perform competently what is presented here, in the future you could use either these templates or a specific statistical package. In fact, when you are done, and as a necessary consequence of doing the exercises in this book, you will have created a library of spreadsheet templates that can be, in effect, your own personal statistical package.
1 .3 AN INTEGRATED APPROACH TO LEARNING STATISTICS
_15_
A Note About Statistical Notation As noted earlier, spreadsheets provide a graphic notation of sorts and allow notation to be simplified, but notation is hardly eliminated entirely. Some notation is needed. Yet notation can be a tricky subject that raises passions seemingly out of proportion to its importance. Part of the problem may be prior experience. Once we have learned one set of notation, we are often reluctant to learn another, just as we are likely to favor the first word-processing program we learned and reject subsequent ones as ridiculous or unnatural. This presents problems. What sort of notation should be used here? Is it possible to select symbols that seem useful, clear, consistent, and reasonably natural to most readers? Moreover, for this book we want notation that can easily be used for labeling spreadsheets, which rules out a fair number of nonstandard symbols, including something as simple as a bar over a letter. Consider, for example, one of the most common statistics, the arithmetic mean. When raw scores are symbolized with X, their mean is often symbolized as an X with a bar above it (read "X-bar"), but there is no easy way to enter X-bar as a spreadsheet label. Another symbol often used to designate the mean is M, and because of its typographical simplicity, we decided to use it in this book. Thus MX (read "mean-of-X") indicates the mean of the X scores, My indicates the mean of the Y scores, and so forth. For purposes of labeling spreadsheets, you may use either MX and My (no italics and no subscripts) or MX and My, whichever you find easier with your spreadsheet. Some might prefer X-bar and Y-bar, but what is easy for a pen often becomes complex for a computer. Our use of notation in this book has been guided by a few simple principles. Whenever possible, and recognizing that there is some but by no means universal standardization of statistical notation among textbooks, we have usually opted for forms in common use. However, as mentioned in the preceding paragraph, usability as a spreadsheet label has affected our notational choices. We have also tried to avoid complex forms. For example, if no ambiguity is introduced, we often avoid subscripts, and thus X may be used instead of Xij to indicate a particular datum if context (and location in spreadsheet) indicate clearly which datum is under discussion. Above all, we have endeavored to be consistent within this book and to define all notation when first introduced. Statistical symbols and key terms are set off from the text in boxes (labeled Notes) and, for ease of reference, statistical symbols and key terms are collected together in a glossary at the back of the book. Finally, and in order to facilitate comparison with other notation the reader may have learned or may encounter, we usually mention any common alternative forms.
The Importance of Spreadsheet Exercises This book is intended to be one leg of a tripod. The reader and the computer form the other two legs. When using this book, we assume that readers will have their computer at hand and they will use it to check examples and perform all exercises given in the text. Answers for most exercises are provided, either in the text itself, in figures in the text, or in the Answers to Selected Exercises section at the end of the book. This should be understood as a device to provide immediate feedback, not as an invitation to short-circuit the kind of learning that comes only by doing things for yourself. Tripods are useful because their three legs provide stability, but all three legs working together are required. When deprived of one of its legs, a tripod becomes unstable and falls. Likewise, unless you ground your work firmly on the three legs we intend— the material presented in this book, yourself, and your computer— your progress could be unstable and your
16_
PRELIMINARIES: How To USE THIS BOOK understanding might fall short. The moral is clear: You need to read thoughtfully and carefully, do all exercises, and think about how the exercises and the written material relate. Readers will come to this text with different degrees of spreadsheet experience, including none, which is why detailed instructions are provided for many of the exercises, especially those earlier in the book. For many readers, such instructions will prove welcome. But for readers who begin with more experience, and for readers who become experienced in the course of doing these exercises, detailed instructions can become more of an annoyance than a help. For that reason, general instructions are provided for all exercises, allowing more practiced or venturesome readers the option of working more on their own. Many of the exercises build on previous problems, modifying previous spreadsheets for new purposes. This is advantageous because it minimizes the tedium of data entry and demonstrates in a clear way how your work, and your learning, are accumulating. Nonetheless, we recommend that you save a copy of each spreadsheet template that you develop, even if only a slight modification of an earlier template. You may want to print a copy of each different template for future reference and you will want to save a copy on disk or other storage medium. To avoid confusion, give each template that you save a name that relates to the exercise, one that allows you to identify and retrieve it later. As you will soon discover, you will often have occasion to refer back to previous spreadsheets and you will want to retrieve earlier attempts for later modification. Now, on to the work at hand. The next chapter begins where researchers typically begin, with a research question, some research data, and the need to evaluate the hypothesis that stimulated the research in the first place.
2
Getting Started: The Logic of Hypothesis Testing
In this chapter you will: 1. 2. 3. 4. 5. 6. 7.
Learn the difference between descriptive and inferential statistics. Learn the difference between samples and populations. Learn the difference between statistics and parameters. Be introduced to the logic of statistical tests. Learn about type I errors (false claims). Learn about type II errors (missed effects). Learn about the power of statistical tests to detect real effects.
Students in the behavioral sciences study statistics for a variety of reasons: the course is required, they need to analyze their thesis data in order to get a degree, and so forth. Thinking positively, let us assume that learning how to analyze, understand, and interpret research data is the paramount motivation. Then, rather than starting with long digressions into descriptive statistics and basic probability theory, it makes sense to begin immediately showing how statistics can be used to answer research questions. 2.1
STATISTICS, SAMPLES, AND POPULATIONS In discussing our need for statistics, imagine as our starting point a very simple study. Assume that a maverick psychotherapist decides to try an unusual treatment. The next 10 people who show up at her office are included in the study. Each is referred to an outside consultant for an initial evaluation. Then appointments are scheduled weekly for the next 3 months, but instead of talking to the clients, the psychotherapist simply gives them $50 and sends them on their way. After 3 months, the patients are again evaluated by an outside consultant who knows nothing of this unusual treatment. Based on the consultant's reports, the therapist determines that 8 of 10 clients improved. Her data are shown in Fig. 2.1. The question is, should we be impressed by any of this? After all, all 10 did not get better. And some would probably have gotten better just by chance alone. Is 8 out of 10 just a chance happening? Or do these results merit some attention?
17
18
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING Subject
Outcome
1
+
2
+
3 4 5 6 7 8 9 10
+ + + + + +
FIG. 2.1. Data for the money treatment study: + = improved, - = not improved. Specifically, do they suggest that the "money cure" the therapist attempted is effective? A knowledge of statistics lets us answer questions like these and reach conclusions even when our knowledge about the world is incomplete. In this case, all we know is what happened with a sample of 10 people. But what can we say about the effect of the money cure on people in general?
Generalizing From Sample to Population Ultimately we want to know not just the datum for a single subject (whether the subject improved) and not just the group summary statistic (8 of 10 improved), but what happens to therapeutic clients in general. What we really want to know is, for the population of all possible clients, how many would get better—that is, what is the probability that the money cure would prove effective in general? Assessing all possible clients to find this out is a practical impossibility. Even if we could identify potential clients, the number would be overwhelming. So instead we select a limited but manageable number to study and hope that this sample represents the population at large.
The Importance of a Random Sample The study sample and how it is selected are the foundation on which the logic of statistical reasoning rests. In the most narrow sense, the results of the moneycure study tell us only about the 10 people actually studied. Yet normally we want to claim that these results generalize to a wider population. Such claims are justified if our sample is truly a random sample from the population of interest. We would first need to define the population of interest (e.g., all potential psychotherapy clients), put their names in a hat, and then draw 10 names. This would be a random sample because each and every name has an equal chance of being selected. In practice, however, the requirement of random sampling is almost never met in behavioral science research. Some survey research studies that use random-digit dialing of respondents come very close. But almost always psychological studies use "samples of convenience" and then argue that the sample, if not randomly selected, at least appears to represent middle-class mothers, college sophomores, happily married couples, and so forth. This rather
2.1 STATISTICS, SAMPLES, AND POPULATIONS
19
casual and after-the-fact approach to sampling is less than ideal from a statistical point of view, and although commonly condoned, it does place a burden on individual researchers to justify any generalizations.
What Are Statistics Anyway? The term statistics is used in two quite different ways. In a narrow sense, a statistic is simply a number that characterizes or summarizes group data. The statistic of interest for the present example is a count or tally— the number of people in the study who improved— but other statistics for the group could be the average age of the subjects or the variability of their income. In a broader sense, statistics, as a body of knowledge and as an academic discipline, comprises concepts and techniques that allow researchers to move beyond the information provided by one study and generalize, or make inferences about, a larger population. Thus statistics, in the broad sense, makes extensive use of statistics, in the narrow sense.
Descriptive and Inferential Statistics It is common to distinguish between descriptive statistics and inferential statistics. Descriptive statistics are statistics in the narrow sense. Examples include the ratio of men to women in a graduate statistics class, the average age of students at our university, the medium income of households in the United States, and the variability of Scholastic Aptitude Test (SAT) scores for 2004. Descriptive statistics are simply summary scores for sets of data and as such characterize various aspects of the units studied. In psychological studies, units are often people (e.g, students in a graduate statistics class or the people who took the SAT in 2004) and are often referred to as participants in research reports. But other kinds of units (e.g., a mother-infant dyad, a family, a school, a city, a year, etc.) are possible. Inferential statistics, on the other hand, implies using statistics in a broader sense. The term refers to a body of reasoning and techniques, called statistical decision theory or hypothesis testing, that allow researchers to make decisions about populations based on the incomplete information provided by samples. Thus the techniques of inferential statistics are designed to address a practical problem. If we cannot assess all units (individuals, dyads, schools, states, etc.) in the population, how can we still make decisions about various aspects of the population that we believe are important and interesting?
Populations and Parameters, Samples and Statistics Aspects of interest are identified with population parameters. For example, one parameter could be the probability that a client in the population would improve. Another could be the average age of clients in the population. Another could be the strength of the association between clients' age and their income, again in the population, not just in the sample. We assume that population parameters are quantifiable and, in theory at least, knowable in some sense even if not directly observable. Some writers, for example, regard populations as essentially infinite by definition, which means that determination of parameter values by direct measurement would forever elude us. But even if we cannot assess population parameters directly, we can at least estimate values for them from sample data instead. Indeed, statisticians spend considerable care and time deriving appropriate ways to estimate different parameters and demonstrating that those estimates have a variety of desirable properties.
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING
20
FIG. 2.2. Schematic representation of the relations between statistics, samples, parameters, and populations. In the sample, aspects of interest are identified with statistics (in the narrow sense). In other words, a statistic is to a sample as a parameter is to a population (see Fig. 2.2). Often (but not always) a sample statistic can be used to estimate a population parameter directly. For example, the population mean is usually represented with u (the Greek lower case mu), the sample mean is often represented with M (as discussed in the last chapter), and as it turns out the value of u can be estimated with the sample M. In other words, "estimated = M
(2.l)
Simplifying considerably, inferential statistics implies: 1. Drawing a random sample from a population. 2. Computing statistics from the sample. 3. Estimating population parameters from the sample statistics. In the next section we discuss ways of comparing values of population parameters estimated from a sample with values assumed by a particular theory. As you will see, this is the way we test our research hypotheses.
Note 2.1 M
Roman letters are usually used to represent sample statistics. For example, M is often used for the sample mean.
u
Greek letters are usually used to represent population parameters. For example, u (lower case Greek mu) is often used for the population mean.
2.2 HYPOTHESIS TESTING: AN INTRODUCTION 2.2
21
HYPOTHESIS TESTING: AN INTRODUCTION With reference to the money-cure study described at the beginning of this chapter, is 8 out of 10 clients improving a significant result or just a chance happening? Just by chance we could have selected 8 out of 10 patients who improved even though in the population from which those subjects were selected the money cure has no effect on therapeutic outcome. If we assume that the money cure has no effect—that patients, left to their own devices, are as likely to improve as not—then the value for the population parameter of interest (the probability of improving) would be .5 (or 1 in 2). This is not a matter of computation, but simply an assumption, based on the theory that the money cure has no effect. But even if the true value of the population parameter were .5, it is still possible, just by chance alone, to draw a sample in which the value of the relevant statistic is .8. It may be unlikely, but it could happen. Even so, if an observed result (.8) is sufficiently unlikely, naturally we question whether or not the assumed value for the population parameter (.5) is reasonable. Over the course of the past century, a method of inference termed statistical decision making or hypothesis testing has been developed, largely by Fisher but with generous assists from Neyman and Pearson. Hypothesis testing provides a way of deciding whether or not postulated or assumed values for various population parameters are tenable given the evidence provided by the statistics computed from a particular sample. Five elements central to this process, listed in the order in which they need to be addressed during hypothesis testing, are: 1. The null hypothesis, which provides a theoretical basis for constructing a sampling distribution for a particular test statistic. 2. The sampling distribution, which allows us to determine how probable particular values of a test statistic would be if in fact the null hypothesis were true. 3. The alpha level, a probability value that is used for the statistical test. 4. The test statistic, the value for which is computed from the sample. 5. The statistical test, which allows us to determine whether it is probable, or improbable (the exact probability value is the alpha level), that the sample was drawn from a population for which the null hypothesis is true. The Null Hypothesis The null hypothesis (usually symbolized HO) is one guess about the true state of affairs in the population of interest. It is contrasted with the alternative hypothesis (usually symbolized H1). The null and alternative hypotheses are usually formulated so that, as a matter of logic, they are mutually exclusive (only one of them can be true) and exhaustive (one of them must be true). In other words, if one is false, the other necessarily must be true. Thus if the null hypothesis states that it is not raining, then the alternative hypothesis states that it is raining. Textbooks in statistics often begin by introducing hypotheses about population means. For example, if the null hypothesis states that the population mean is zero, then an alternative hypothesis could be that the population mean is not zero. Symbolically:
22
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING In this case, the alternative hypothesis commits to a mean different from zero no matter whether that mean is smaller or larger than zero. Thus this H1 is nondirectional (sometimes called two-tailed). Another null hypothesis could state that the population mean is greater than zero whereas the alternative hypothesis could be that the population mean is not greater than zero. Symbolically: H0: u < 0 . H 1 : u>0. Thus this H1 is directional (sometimes called one-tailed} because it commits only to differences from zero that, in this case, are larger than zero. Usually, the alternative hypothesis corresponds to the investigator's hunch regarding the true state of affairs; that is, it corresponds to what the investigator hopes to demonstrate (for the current example, that the money treatment has an effect). However, it is the tenability of the null hypothesis that is actually tested (that the money treatment has no effect). If the null hypothesis is found untenable, then as a matter of logic the alternative must be tenable, in which case investigators "reject the null" and, for that reason, "accept the alternative." When encountered for the first time, this way of reasoning often seems indirect and even a bit tortured to many students. Why not simply demonstrate that the alternative hypothesis is tenable in the first place? This is partly a matter of convention, but there is good reason for it. There is a crucial difference between null and alternative hypotheses. In its typical form, a null hypothesis is exact. It commits to a particular value for a population parameter of interest. The alternative hypothesis, on the other hand, usually is inexact. It only claims that the parameter is not the value claimed by the null hypothesis but does not commit to a specific value (although, as we just saw, it may commit to a particular direction). (Exceptions to the foregoing exist, but they are rare.) In order to construct a sampling distribution for the test statistic, which is the next step in hypothesis testing, an exact value for the population parameter of interest is needed. In most cases, it is provided by the null hypothesis. Thus we test, and perhaps reject, not the hypothesis that embodies our research concern and that postulates an effect, but another hypothesis that postulates exactly no effect and that, from a substantive point of view, is less interesting. According to the no-effect or null hypothesis for the present example, exactly half of the clients should get better just by chance alone. If this null hypothesis were true, then the population parameter for the probability of improving would be .5. This may or may not be the true value of the population parameter, but at least it provides a basis for predicting how probable various values of the test statistic would be, if the true value for the population parameter were indeed .5.
The Sampling Distribution Once the null hypothesis is defined, the sampling distribution appropriate for the test statistic can be specified. Sampling distributions are theoretical constructs and as such are based on logical and formal considerations. They resemble but should not be confused with frequency histograms, which are based on data and indicate the empirical frequency for the scores in a sample. Sampling distributions are important for hypothesis testing because we can use them to derive how likely (i.e., how probable) a particular value of a test statistic would be, in theory at least, if the null hypothesis were true. The binomial, which is described in chapter 3, is the appropriate sampling distribution for the current example.
2.2 HYPOTHESIS TESTING: AN INTRODUCTION
23
The phrase "in theory" as it applies to sampling distributions is important. The probabilities provided by a sampling distribution are accurate for the test statistic to the extent that the data and the data collection procedures meet the assumptions used to generate the theoretical sampling distribution in the first place. Assumptions for different tests vary. And in practice many assumptions can be violated without severe consequences. One assumption, however, called independence of measurements (or simply, independence), is basic to most sampling distributions and, if violated, raises serious questions about any conclusions. This key assumption requires that during data collection scores are assigned to each unit in the sample (subject, dyad, family, and so forth.) independently. In other words, each score must represent an independent draw from the population. For the present example, this means that the evaluation of one client cannot be linked, or affected by, the evaluation given another client. To use the classic example, imagine an urn filled with tickets, each of which has a number printed on it. We shake the urn, reach in, remove one ticket, and note its number. We then replace the ticket and repeat the procedure until we have accumulated N numbers, the size of our sample. (If the population of tickets is large enough, it may not matter much whether we draw with, or without, replacement of tickets previously drawn.) This constitutes an independent sample because presumably tickets pulled on previous draws do not influence which ticket we pull on the next draw. We would not, however, select two tickets at a time because then pairs of tickets would be linked and the total of all tickets drawn would not constitute an independent sample. And if the assumption of independence is violated, then we can no longer be confident that a probability value derived from a theoretical sampling distribution provides us with the correct probability value for the particular test statistic being examined.
The Alpha Level The alpha level is the probability value used for the statistical test. By convention, it is usually set to .05 or, more stringently, to .01. If the probability of the results observed in the sample occurring by chance alone (given that the null hypothesis is true) is equal to or less than the alpha level, then we declare the null hypothesis untenable.
The Test Statistic In contrast to the sampling distribution, which is based on theory, the value of the test statistic depends on the data collected, and thus in general terms a test statistic is a score computed from sample data. For example, if our null hypothesis involved the age of our clients, the average age might be used as a test statistic. In theory, any of a wide array of summary statistics could be used as test statistics, but in practice, statistical attention has focused on just a few. For the present example, the test statistic is the probability that clients who received the money cure would improve and its value, as determined from the sample of 10 subjects, is .8.
The Statistical Test Given a particular null hypothesis, an appropriate sampling distribution, and a value for the appropriate test statistic, we are in a position to determine whether or not the result we observed in our sample would be probable if the null hypothesis is in fact true. If it turns out that our result would occur only rarely,
24
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING given that the null hypothesis is true, we may decide that the null hypothesis is untenable. But how rare is rare? As noted a few paragraphs back, by convention and somewhat arbitrarily, 5% is generally accepted as a reasonable cutoff point. Certainly other percentages could be justified, but in general behavioral scientists are willing to reject the null hypothesis and accept the alternative only if the results actually obtained would occur 5% of the time or less by chance alone if the null hypothesis were true. This process of deciding what level of risk is acceptable and, on that basis, deciding whether or not to reject the null hypothesis constitutes the statistical test. For the present example (testing the effectiveness of the money cure), the appropriate test is called a sign test or a binomial test. In the next chapter we describe this test and demonstrate its use.
2.3
FALSE CLAIMS, REAL EFFECTS, AND POWER
Type I Error: The Risk of Making a False Claim Earlier in this chapter we claimed that knowledge of statistics allows us to make decisions from incomplete information. Thus we may make decisions about a population based only on a sample selected from the relevant population. For the money-cure example, only 10 subjects were examined—which falls far short of a complete survey of psychotherapy clients. Yet based on this sample of 10 subjects (and pending presentation of the sign test in the next chapter), we might conclude that the money cure affected so many clients in the sample positively that the null hypothesis (the hypothesis that the clients were selected from a population in which the money cure has no effect) is untenable, which leads us to conclude that, yes, the money cure does have a beneficial effect. Basing decisions on incomplete information entails a certain amount of risk. What if, for example, in the population from which our subjects were selected, the money cure had no effect even though, in this one study, we just happened to select a high proportion of clients who got better? In this case, if we claimed an effect based on the particular sample we happened to draw, we would be wrong. We would be making what is called a type I error, which means we would have rejected the null hypothesis when in fact the null hypothesis is true. We would have made a false claim. Given the nature of statistical inference, we can never eliminate type I errors, but at least we can control how likely they are to occur. As noted earlier, the probability cutoff point for rejecting the null hypothesis is called the alpha level. If we set our alpha level to the conventional .05, then the probability that we will reject the null hypothesis wrongly, that is, make a type I error, is also .05. After all, by setting the alpha level to .05 for a statistical test we commit ourselves to rejecting the null hypothesis if the results we obtain would occur 5% of the time or less given that the null hypothesis is true. If we did the same experiment again and again, and if in fact there is no effect in the population, over the long run 95% of the time we would correctly claim no effect. But 5% of the time, just by the luck of the draw, we would wrongly claim an effect. As noted earlier, by convention most behavioral scientists find this level of risk acceptable.
Type II Error: The Risk of Missing a Real Effect Making a false claim is not the only error that can result from statistical decision making. If, for the population from which our subjects were selected, the money cure indeed has an effect but, based on the particular sample drawn, we claimed that there was none, we would be making what is called a type II error. That is,
2.3 FALSE CLAIMS, REAL EFFECTS, AND POWER
25
we would have failed to reject the null hypothesis when in fact the null hypothesis is false. We would have missed a real effect. Under most circumstances, we do not know the exact probability of a type II error. The probability of a type II error depends on the actual state of affairs in the population— which we do not know exactly, and not on the null-hypothesis assumed state of affairs— which we define and hence know exactly. A type I error occurs when the magnitude of the effect in the population (indexed by an appropriate population parameter) is zero (or some other specific value) and yet, based on the sample selected, we claim an effect (thereby making a false claim). The probability that this will occur is determined by the alpha level, which we set, and depends on the sampling distribution for the test statistic under the null hypothesis, which we assume. Hence we can specify and control the probability of type I error. In contrast, a type II error occurs when the magnitude of the effect in the population is different from zero (or another specific value) by some unspecified amount and yet, based on the sample selected, we do not claim an effect (thereby missing a real effect). The probability of a type II error can be determined only if the exact magnitude of the effect in the population is known, which means that, under most circumstances, we cannot determine the probability of a type II error exactly. However, even if we do not know the probability of a type II error, we can affect its magnitude by changing the alpha level. If we select a more stringent alpha level (.01 instead of .05, for example), which has the effect of decreasing the probability of making a false claim, we necessarily increase the probability of missing a real effect. It is a trade-off. Decreasing the probability of a type I error increases the probability of a type II error and vice versa. If we are less likely to make a false claim, we are more likely to miss a real effect. If we are less likely to miss a real effect, we are more likely to make a false claim.
Power: The Probability of Detecting Real Effects Perhaps too pessimistically, the preceding paragraphs have focused on the ways statistical decision making can go wrong. But just as there are two ways we can be wrong, there are also two ways we can be right. (All four possibilities are shown schematically in Fig. 2.3.) One way that we can be right occurs when there genuinely is no effect in the population and we claim none. The probability of making a false claim is alpha (a), so the probability of correctly claiming no effect is 1 - alpha. If there is no effect, if our alpha level is .05, and if we conduct study after study, over the long run 95% of the time we will be right when we claim no effect. A second way that we can be right occurs when we claim an effect and there genuinely is one. Just as alpha (a) is the probability of making a false claim (Type I error), so the probability of missing a genuine effect (Type II error) is often symbolized as beta (B). And just as the probability of correctly claiming no effect is 1 - alpha, so the probability of correctly claiming an effect is 1 - beta. This probability, the ability to detect real effects, is called the power of a statistical test. The power of statistical tests is affected by three factors. First, the magnitude of the effect in the population affects power. Other things being equal, bigger effects are more likely to be detected than smaller effects and hence when effects are large we are more likely to claim an effect. Second, as is discussed in chapter 17, we are more likely to detect an effect of a particular magnitude when sample sizes are larger than when they are smaller. Third, alpha level affects power. If the alpha level is made less stringent (changed from .05 to .10, for example),
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING
26
FIG. 2.3. Possible outcomes of the statistical decision-making process and their associated probabilities (P symbolizes probability).
power will be increased. We will be more likely to detect real effects if they exist but we will also be more likely to make false claims if H0 is true. Conversely, if we make the alpha level more stringent, power will be decreased. We will be less likely to make false claims if H0 is true but we will also be less likely to detect real effects if they exist. Of these three, only sample size is under our control and so the only practical way to increase power is to increase sample size. Alpha is almost always set at .05 (or less) by convention, so choosing a less stringent alpha level is usually not a practical way to increase power; in any case, such increases would be balanced by the concomitant increase in type I error. And the magnitude of the effect in the population is hardly under our control. In fact, as we noted earlier, normally we do not even know its actual value and thus we cannot compute a value for either beta (the probability of missing a real effect) or power (the probability of detecting a real effect).
Note 2.2 Type I Error
Making a false claim, or incorrectly rejecting the null hypothesis (also called an alpha error}. The probability of making a Type I error is alpha.
Type II Error
Missing a real effect, or incorrectly accepting the null hypothesis (also called a beta error). The probability of making a Type II error is beta.
Power
The probability of detecting a real effect, or the probability of correctly rejecting the null hypothesis. The power of a test is one minus the probability of missing a real effect, or 1 - beta.
2.3 FALSE CLAIMS, REAL EFFECTS, AND POWER
27
Alpha, Beta, and Power: A Numerical Example Under normal circumstances, we do not know the true state of affairs for the population. All we know (because we assume it) is what the sampling distribution for the test statistic would be if the null hypothesis were true. For example, imagine that we have developed a test statistic whose sampling distribution under the null hypothesis is as shown in the first histogram (top) of Fig. 2.4. Altogether there are 20 possible outcomes, each represented by a square. One of the outcomes is 1, which is why there is one square or box above the value 1. Two of the outcomes are 2, which is why there are two boxes above the value 2. Similarly, three of the possible outcomes are 3, four are 4, four are 5, three are 6, two are 7, and one is 8. Area is proportional to probability, and thus the probability that the outcome actually will be 1 on any one trial is 1/20, that it will be 6 is 3/20, and so forth. The most likely outcomes for the test statistic are either 4 or 5. For each, the probability is 4/20 or .2; hence the probability of either a 4 or a 5 is .4 (because the probability of at least one of a set of events occurring is the sum of their separate probabilities). Symbolically, P( 4 or 5 ) =P(4) +P(5) = .2 + .2 = .4.
(Recall that P symbolizes probability.) Imagine further that we have decided beforehand that we will reject the null hypothesis only if our study yields a test statistic whose value is larger than expected (this specifies a directional or one-tailed test as opposed to a nondirectional or two-tailed test) and that we have set our alpha level to an unconventionally generous .15. Note that the theoretical probability of a 7 is 2/20 or .10 and the probability of an 8 is 1/10 or .05; hence the probability of either a 7 or 8— the largest values in this sampling distribution— is the sum of .10 and .05 or .15. Symbolically,
P(7 or 8) = P(7)+P(8) = .10 + .05 = .15. Thus if we conduct a study and discover that the value of the test statistic is 7 or larger, we would reject the null hypothesis. This is, after all, a relatively unlikely outcome, given the sampling distribution portrayed at the top of Fig. 2.4. If we performed the study hundreds of times, and if the null hypothesis were true, the value for the test statistic would be as big as 7 or 8 only 15% of the time. In order to demonstrate the relations between alpha, beta, and power, it is convenient (in fact, necessary) to assume that we actually know the true state of affairs for the population. For example, imagine that we know the true value of the appropriate population parameter and it generates a sampling distribution like the one shown in the second histogram (bottom) of Fig. 2.4. Given this information we can compute beta. The sampling distributions shown in Fig. 2.4, like all sampling distributions, indicate the likelihood of various outcomes. For example, the first histogram (top) indicates that if the null hypothesis is true then a test statistic as large as 7 or 8 would occur only 15% of the time or less. Each time we do a study, of course, only one outcome is produced. If that outcome were 6 or less, we would not reject the null hypothesis. After all, outcomes of 6 or less would occur 85% of the time (17 times out of 20; top histogram). In other words:
28
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING
P(1 or 2 or 3 or 4 or 5 or 6) = P(1) + P2) + P3) + P(4) + P5) + TO) = .05 + .10 + .15 + .20 + .20 + .15 = .85
However, if the population probabilities for the test statistic are as indicated in the second histogram (bottom), then outcomes of 6 or less would actually occur 6 times out of 20, or just 30% of the time:
P(4 or 5 or 6) = P(4)+P(5)+n6) = .05 + .10 + .15 = .30 If we conducted the study repeatedly, over the long run 30% of the time we would not reject the null hypothesis (because the value of the test statistic would be 6 or less). In other words, 30% of the time we would fail to detect the effect. Hence, if the true state of affairs is as indicated in the bottom histogram, and our null hypothesis is based on the top histogram, then the probability of a type II error (of missing the effect) would be .3. However, 70% of the time we would correctly claim an effect. In this case the power of the statistical test would be .7.
FIG. 2.4. Sampling distributions for a hypothetical test statistic assumed by the null hypothesis (top) and actually occurring in the population (bottom).
2.3 FALSE CLAIMS, REAL EFFECTS, AND POWER
29
The Region of Rejection Sampling distributions like those portrayed in Fig. 2.4 show the relations among alpha, beta, and power graphically. Consider the unshaded and shaded areas. For the null-hypothesis-generated sampling distribution (top), and an alpha level of .15, values in the shaded area to the right would cause us to reject the null hypothesis (in this case, values of 7 or more), whereas values in the unshaded area to the left would not cause us to reject the null hypothesis (in this case, values of 6 or less). Recall that for a sampling distribution, area is equivalent to probability. Thus, for an alpha level of .15, 15% of the area is shaded. This area is called the region of rejection or the critical region. If values computed for our test statistic fall within this rejection region (in this case, if our test statistic is 7 or greater), we would reject the null hypothesis. In the top panel, 85% of the area under the null-hypothesis-generated sampling distribution is unshaded, to the left. This proportion is 1 - alpha, the probability of correctly making no claim if the null hypothesis is true. But if the null hypothesis is false, and if we know that the true state of affairs is as depicted in the bottom panel, then matters are different. The unshaded 30% of the area to the left represents beta, the probability of missing an effect (the probability of a type II error), whereas the shaded 70% of the area to the right is 1 - beta and represents power. In this case, we have a 70% chance of detecting an effect as significant, given an alpha level of .15. The shaded area can help us visualize the relation between alpha and beta. Imagine that we reduce the alpha level to .05, reducing the shaded portion to values of 8 or more. This makes alpha (the probability of making a false claim, a type I error) smaller (top panel). But at the same time, beta (the probability of missing an effect, a type II error) becomes larger while 1 - beta (the power of the test or the probability of correctly detecting a genuine effect) necessarily decreases (bottom panel). Exercise 2.1 Type I Errors, Type II Errors, and Power The problems in this exercise refer to Fig. 2.4. They provide practice in determining the probability of making false claims (type I error), of missing real effects (type II error), and of detecting real effects (power). All questions assume that if the null hypothesis is true, the test statistic is distributed as shown in the top panel. 1. If the alpha level were .05, instead of .15, what values for the test statistic would fall in the critical region, that is, lead us to reject the null hypothesis? 2. If the alpha level were .30, what values for the test statistic would fall in the critical region? 3. If the alpha level were .05, and the true state of affairs were as depicted in the bottom panel, what is the probability of missing a real effect? What is the probability of correctly claiming an effect? 4. If the alpha level were .30, and the true state of affairs were as depicted in the bottom panel, what is the probability of missing a real effect? What is the probability of correctly claimin'g an effect?
GETTING STARTED: LOGIC OF HYPOTHESIS TESTING
30
2.4
WHY DISCUSS INFERENTIAL BEFORE DESCRIPTIVE STATISTICS? The differences between samples and populations, between statistics and parameters, and between descriptive and inferential statistics were discussed in this chapter. Hypothesis testing, which represents an important application of inferential statistics, was presented and the ways a statistical decision could be either correct or incorrect were noted. Many textbooks spend the first several chapters detailing descriptive statistics—a topic not addressed in earnest in this book until chapter 5. Descriptive statistics usually seem more concrete than inferential statistics (there are no type I and type II errors to puzzle about and no parameters to estimate, for example), and as a result many students find presentations of descriptive statistics less challenging and easier to learn. Nonetheless, we prefer to begin with material that is more abstract and certainly more intellectually exciting (hypothesis testing, for example) because it immediately and clearly demonstrates the usefulness of statistical knowledge and techniques. Continuing in this vein, a common and useful statistical test, the sign test, is presented in chapter 3.
Note 2.3 Alpha Level
Maximum acceptable probability for type I error (claiming an effect based on sample data when there is none in the population). Conventionally set to .05 or .01.
Rejection Region
Designates values for the test statistic whose probability of occurrence is equal to or less than the alpha level, assuming that the null hypothesis is true. If the test statistic assumes any of these values (falls in the region of rejection), the null hypothesis is rejected.
3
Inferring From a Sample: The Binomial Distribution
In this chapter you will: 1. Learn about a simple sampling distribution, the binomial. 2. Learn how to perform a simple statistical test, the binomial or sign test. This chapter continues the discussion of hypothesis testing begun in chapter 2 and illustrates how a common statistical test, the binomial or sign test, can be used to answer simple research questions. The sign test makes use of one of the easiest distributions to understand, the binomial. 3.1
THE BINOMIAL DISTRIBUTION Interest in the binomial distribution began with Blaise Pascal in the 17th century and has continued to be a source of interest for mathematicians and statisticians ever since. The binomial is easy to understand (at least now, after several brilliant mathematicians have shown us how), easy to generate, and lends itself to a clear and straightforward demonstration of hypothesis testing. Moreover, a simple and useful statistical test—the sign test—is based on the binomial. For all these reasons, the first sampling distribution introduced in this book is the binomial. The binomial is the appropriate sampling distribution to consider whenever a trial results in one of two events—that is, whenever the results of a single trial (or treatment) can be categorized in one, and only one, of two ways. For example, in the money-cure study described in the last chapter, clients were categorized as either improved or not improved. In more general terms, whenever a subject (or a family, a coin toss, or whatever constitutes the sampling unit) can be assigned to one of two possible categories, the binomial distribution can be used. From it, we can figure out how probable the results for a particular study would be if the null hypothesis (and not the alternative hypothesis) were in fact true. Consider the money-cure study. If we examine only one subject (symbolized S), then there are only two possible outcomes for the study—the events defined for a single trial (because, in fact, this study consists of only one trial):
31
32
INFERRING FROM A SAMPLE: THE BINOMIAL 1. The subject improves. 2. The subject does not improve. If we assume that both events are equally likely, then the probability that the subject improves is 1/2 or .5. The probabilities for both events must sum to 1, so the probability that the subject does not improve must be 1 minus .5. Symbolically: P - P(S improves) = .5 Q - P(S does not improve) = 1 - .5 - .5 (P is often used to represent the probability for the first event, Q for its complement; thus Q =1- P.) However, if we examine two subjects (S1 and S2), then there are four possible outcomes for the study: 1. 2. 3. 4.
Both subjects improve. The first improves but the second does not improve. The first does not improve but the second does improve. Neither subject improves.
The probability that the first subject will improve is .5 and the probability that the second subject will improve is also .5; therefore, because the probability that all events in a set will occur is the product of their separate probabilities, the probability that both subjects will improve is .25. Symbolically (imp = improves), P(both improve) = P(S1 imp and S2 imp) = P(Sl imp) x P(S2 imp) - 0.5 x 0.5 = 0.25 Similarly, because the probability of a subject not improving is also .5, the probability for the second, third, and fourth outcomes will also be .25. This makes sense, because the probabilities for all outcomes must sum to 1. Although there are four different possible outcomes for a study with two subjects, there are only three outcome classes: 1. Two improve (outcome 1). 2. One improves (outcomes 2 and 3). 3. None improve (outcome 4). The probability for the first outcome class is .25. This class contains only one outcome, outcome 1, whose probability was computed in the previous paragraph. Likewise, the probability for the third class is also .25, again because this outcome class contains only one outcome, outcome 4, whose probability is .25. But the probability for the second outcome class is .5. This class contains two outcomes, outcomes 2 and 3, both of whose probabilities are .25. Therefore, because the probability of any one of a set of events occurring is the sum of their separate probabilities, the probability that either will occur is twice .25 or .5. Symbolically,
3.1 THE BINOMIAL DISTRIBUTION
33
.P(one improves) =P(S 1 imp and S2 not) + P(S1 not and S2 imp) - P(S1 imp) x P(S2 not) + P(S1, not) x P(S2 imp) = 0.5 x 0.5 + 0.5 x 0.5 = 0.25 + 0.25 = 0.5 This formula illustrates the two basic probability rules we have been using: (a) The and rule, which states that the probability that two events will both occur is the product of their individual probabilities; and (b) the or rule, which states that the probability that either of two events will occur is the sum of their individual probabilities. Often discussions of the binomial are cast in terms of coins. After all, under normal circumstances a tossed coin must come up either heads (H) or tails (T). Continuing our intuitive introduction to the binomial, if we toss a coin three times (which is equivalent to examining three subjects), then there are eight different possible outcomes for a three-trial study (T = tails, H = heads): 1. 2. 3. 4. 5. 6. 7. 8.
TTT TTH THT THH HTT HTH HHT HHH
and four outcome classes: 1. 2. 3. 4.
0 heads (1 above) 1 head (2, 3, and 5 above) 2 heads (4, 6, and 7 above) 3 heads (8 above)
A convenient way to represent all of the possible outcomes is with a branching tree diagram (see Fig. 3.1). The first branch at the top represents the first toss, which could be tails or heads. The second set of branches represents the second toss, and so forth. If you tried to list all possible outcomes without some mnemonic device, you might miss a few. If you use the branching diagram, you should be able to list all without omission.
FIG. 3.1. The eight possible outcomes if a coin is tossed three times.
34
INFERRING FROM A SAMPLE: THE BINOMIAL
At this point you may begin to see (or remember) a pattern. With one toss there are two outcomes and two classes. With two tosses there are four outcomes and three classes. With three tosses there are eight outcomes and four classes. And in general, if there are N tosses, there will be N + 1 classes (e.g., for 7 tosses there will be 8 classes: 0 heads, 1 head, 2 heads, ..., 7 heads) and there will be 2N outcomes (because each toss doubles the number of outcomes for the previous toss). Thus for 2 tosses there are 4 outcomes, for 3 tosses, 8 outcomes, for 4 tosses, 16 outcomes, for 5 tosses, 32 outcomes, and so forth (see Fig. 3.2). Exercise 3.1 Tree Diagrams This exercise provides practice in generating tree diagrams and in determining theoretical probabilities for different outcomes. 1. Generate a tree diagram for 4 tosses. Count the number of outcomes in each outcome class. Have you generated the correct number of outcomes and outcome classes? If this seems easy, generate a tree diagram for 5 or 6 tosses. 2. How many outcomes, and outcome classes, are there for 7 tosses? For 8 tosses? For 10 tosses? For 20 tosses?
The principles described in the preceding several paragraphs form the basis for generating a binomial distribution, which is then used as the sampling distribution for the sign test. If we know the number of subjects (N) in a study— and if it is appropriate to assign each subject to one of two categories—then we know that there are 2N different outcomes or 2N different ways the study might turn out. If according to our null hypothesis the two events for a single subject are equally likely (i.e., if the null hypothesis states that the population parameter for the probability of one of two events is .5—which means that the probability for the other outcome must also be .5), it follows that each of the 2N possible outcomes for the study will be equally likely also. For example, if a coin is tossed two times, then there are four possible outcomes (TT, TH, HT, HH), and if the assumed probability for a head is .5, then the probability for any one of the four outcomes is .25. In general, if the null hypothesis assumes that P - .5, and if there are N trials (subjects, coin tosses, and so forth.), then in order to compute
FIG. 3.2. Relation between number of trials, number of outcomes, and number of outcome classes.
3.1 THE BINOMIAL DISTRIBUTION
35
probabilities for the different outcome classes we need only know how many of the 2N outcomes fall in each outcome class. For example, as described previously, with three subjects there are eight possible outcomes. Three of the possible outcomes involve two heads (THH, HTH, HHT); thus the probability of tossing two out of three heads is 3/8 or .375. One of the possible outcomes involves three heads (HHH), thus the probability of tossing all three heads is 1/8 or .125. Putting these together, four of the outcomes involve two or more heads (THH, HTH, HHT, HHH) and so the probability of getting at least two heads in three tosses is 4/8 or .5. But remember, these are all theoretical probabilities based on the null hypothesis that the coin is fair.
Pascal's Triangle For just five or six subjects, you could generate a list of all the simple outcomes using a tree diagram and count the number of outcomes in each class, but with larger numbers of subjects this becomes unwieldy. Fortunately, there is a simple way to compute the number of outcomes in each outcome class. Named for its discoverer, this device is called Pascal's triangle, and although its role in the history of mathematics has been profound, our use of it here is quite simple. Exercise 3.2 Pascal's Triangle This is the second exercise to require a spreadsheet program. The purpose is to develop a spreadsheet template that generates values for Pascal's triangle. Due to its triangular shape, this spreadsheet is unique, unlike any other in the book. The formulas in the cells of the triangle are essentially alike. Instead of laboriously and tediously entering similar formulas in different cells, you should use this exercise to acquaint yourself with the use and utility of the spreadsheet copy command (see chap. 1). General Instructions 1. Let columns represent different number of trials. Use values beginning with 1 and continuing through 10. Label the columns appropriately. 2. Indicate the number of simple outcomes that would occur for each number of trials. 3. Normally Pascal's triangle is represented with the top of the triangle pointing up. In this case, lay the triangle on its side, letting the "top" point to the left. The 1-trial column will have two entries (because there are two outcome classs), the 2-trials column will have three entries (because there are three outcome classes), and so forth. Entries in each column will be staggered, which means that no cell will have an entry immediately to its left or right. Instead, entries will be located on what are, in effect, diagonal lines. 4. Enter formulas that will generate the correct values for Pascal's triangle. Remember that the value of a target entry is the sum of the two entries to the left, the one diagonally above and the one diagonally below the target entry. Once all formulas are correctly entered (which can be done by copying formulas to the appropriate cells), enter the "seed value" of 1 in the apex or top cell of the triangle. Observe the way the values for Pascal's triangle ripple through your spreadsheet. 5. Sum the entries (number of simple outcomes per outcome class) in each column of the triangle. The sum of the entries in each column should be
36
INFERRING FROM A SAMPLE: THE BINOMIAL exactly the same as the number of simple outcomes you computed previously in step 2. Why? 6. The numbers in the 10-trials column represent the number of outcomes per outcome class (0 heads, 1 head, 2 heads, ..., 10 heads). Compute the probability for each outcome class. 7. Sum the probabilities for these 11 outcome classes. This sum must be 1. Why? Detailed Instructions 1. We want to display columns A-L, so set the default column width to 5. Set the width for column L to 8. 2. Label cell A1 "N' for number of trials. Label cell A2 "N + 1" for number of classes. Label cell A3 (and A27) "2N,, for number of outcomes. 3. Put the formula "=B1+1" in cell C1. (This is the Excel convention. Other spreadsheets may have different conventions for entering formulas.) This causes C1 to display a number one larger than the number in the cell immediately to its left. Copy this formula from cell C1 to cells D1 through K1. Now put the value 1 in cell B1 and watch the results "ripple" through the first row. Cells B1-K1 should now display the values 1, 2, 3, ..., 10. 4. Put the formula "=B1+1" in the cell B2. This causes B2 to display a number one greater than the value contained in B1. Copy this formula from cell B2 to cells C2 through K2. 5. Put the formula "=B3*2" in cell C3. This causes C3 to display a number double the number in the cell immediately to its left. Copy this formula from cell C3 to cells D3 through K3. Now "seed" cell B3 with the value 2. Cells C3-K3 should now display the values 4, 8, 16, ..., 1024. 6. Put the formula "=A13+A15" in cell B14. This causes cell B14 to display the sum of the cells diagonally up to the left and diagonally down to the left. Copy this formula to cell B16. When formulas are copied, the cells pointed to are relative to the cell copied. Therefore cell B16 will display the sum of A15 and A17. And because no values have yet been put in the these A-column cells, B14 and B16 will, for now at least, display zeros. 7. Copy the "sum of up and down left diagonal cells" formula to cells C13, C15, C17 (omitting C14 and C16); to D12, D14, D16, D18 (omitting D13, D15, D17); to E11 through E19 (omitting even rows); to F10 through F20 (omitting odd rows); ...; to K5 through K25 (omitting even rows). At this point, your spreadsheet should display a triangle of zeros. The triangle is lying on its side. Its base spans cells K5 through K25 and its apex, cell A15, has yet to be filled in. 8. Put the value 1 in cell A15. Observe the way the values for Pascal's triangle ripple through your spreadsheet. 9. Put a formula for the sum of cells B5-B25 in cell B27. Copy this formula to cells C27-L27. Rows 2 and 28 should now display the same values. Why? 10. In cells L5, L7, L9, ..., L25, put a formula for dividing the cell immediately to the left by cell K27 (absolute). This gives the probabilities for each outcome class when there are 10 trials. (You may want to format column L to have four decimal places.) The value in cell L28 should be 1. Why?
37
3.1 THE BINOMIAL DISTRIBUTION
At this point your spreadsheet should look the one displayed in Fig. 3.3. Each column provides you with the number of outcomes in each outcome class for a particular number of trial. These are the binomial coefficients for that number of trials. For this exercise, you stopped at 10 trials, but in principle you could keep generating more of the triangle forever, simply by copying columns to the right. In practice, of course, you would be limited by the maximum dimensions allowed by your spreadsheet program. But in theory, you should now understand how to generate the number of outcomes in each outcome class for a binomial sampling distribution involving any number of trials. For example, given five trials and a fair coin, your spreadsheet should indicate there are 32 different outcomes and six outcome classes (column F in Fig. 3-3)- The binomial coefficients in this column (1, 5, 10, 10, 5, 1) represent the number of outcomes in each class; thus there is 1 outcome that contains zero heads, 5 that contain one head, 10 for two heads, 10 again for three heads, 5 for
A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
B N
N+1
2N
C 1 2 2
E 2 3 4
F
4 5 16
5 6 32
I
H
K 6 7 8 9 10 9; 10 11 8 7 64 128 256 512 1024
G
J
L
1 0 0010 1
10 0 0098
1 1 1 1 1 1
21
2
6
1
4
20
70 35
15 5
1
0 2461
210 0 2051
28
120
0 1172
36
7
45 0.0439
8
1
252
84
21
1
0 2051
126
56
6
210 126
35
10
1
84 56
15 10
120 0 1172
28
6
4
45 0 0439 36
7
5 1
9 8
1
9 1
10 0 0098 1 1 0 0010
2
4
16
32
64 128 256 512 1024
1
FIG. 3.3. Pascal's triangle: number of outcomes in each outcome class for 1 to 10 trials.
38
INFERRING FROM A SAMPLE: THE BINOMIAL four heads, and 1 for all five heads. The probability, given a fair coin (P = .5), of getting five heads is 1/32 (.03125), of getting four heads is 5/32 (.15625), of getting four or more heads is 6/32 (.1875 or .15625 + .03125), and of getting either no heads or all heads is 2/32 (.0625 or .03125 + .03125). This sampling distribution for five trials is displayed graphically on the left side of Fig. 3.4. Exercise 3.3 Binomial Probabilities The problems in this exercise refer to Fig. 3.3. They provide practice in computing theoretical probabilities using the binomial. 1. The sampling distribution given a fair coin for six tosses is portrayed on the right side of Fig. 3.3. (The coefficients are listed in column G of Fig. 3.3.) What is the probability of getting all heads? Of getting either all heads or all tails? Of getting five or more heads? Of getting either two or three or four heads? Of getting five or more all the same (either five or more heads or five or more tails)? 2. Draw a sampling distribution for eight tosses. What is the probability of eight heads? Of either eight heads or eight tails? Of seven or more heads? Of six or more heads? Of seven or more all the same? Of six or more all the same? 3. Given N trials, there are N ways of getting just one head. Why does this make sense? 4. If your spreadsheet program can produce graphs, use it to graph the binomial sampling distributions for 8, 9, and 10 trials. You may want to produce both bar and line graph versions. Which is more correct? Why?
FIG. 3.4. Binomial sampling distributions for five (left) and six (right) tosses, given a fair coin.
3.1 THE BINOMIAL DISTRIBUTION
39
Binomial Parameters The binomial distribution can be thought of as a family of possible distributions, only two of which are displayed in Fig. 3.4. The binomial is often specified with three parameters: P is the probability of the first outcome, Q the probability of the second outcome, and N is the number of trials. However, only two parameters are necessary. Because Q= 1 - P necessarily (the probability of the second outcome must be one minus the probability of the first outcome), a particular instance of the binomial is specified completely by P and N. In other words, two parameters are associated with the binomial distribution. Given values for these parameters, a particular member of the family can be generated. The examples given in this chapter have all assumed P = .5 because this is the usual case. But imagine that P - .2 and N - 3. If according to our null hypothesis the probability of a head is .2 (and thus the probability of a tail is .8), then how likely are we to toss three heads, three tails, or any of the other possible outcomes? The probabilities for each of the eight possible outcomes are: P(TTT) P(TTH) P(THT) P(THH) P(HTT) P(HTH) P(HHT) P(HHH)
= P(T) x P(T) x P(T) = .8 x .8 x.8 = P(T) x P(T) x P(H) = .8 x .8 x .2 = P(T) x P(H) x P(T) = .8 x .2 x .8 = P(T) x P(H) x P(H) - .8 x .2 x .2 = P(H) x P(T) x P(T) = .2 x .8 x .8 = P(H) x P(T) x P(H) = .2 x .8 x .2 = P(H) x P(H) x P(T) = .2 x .2 x .8 = P(H) x P(H) x P(H) = .2 x .2 x .2
=.512 = .128 = .128 = .032 = .128 = .032 = .032 = .008
Hence the probabilities for the four outcome classes are: P(0 heads) P(1 head) P(2 heads) P(3 heads)
= P(TTT) = P(TTH) + P(THT) + P(HTT) = P(THH) + P(HTH) + P(HHT) = P(HHH)
= 1 x .512 = .512 = 3 x .128 = .384 = 3 x .032 = .096 = 1 x .008 = .008
Note that the probabilities for the eight outcomes and for the four outcome classes add to one, as they must. Note also that you are far more likely to toss three tails than three heads, assuming that the true probability of tossing one head is .2. Given this one example, you should be able to generate binomial distributions for various values of P and N (for additional examples and formulas, see Loftus & Loftus, 1988). More to the point, you should have a sense for how binomial distributions with various values for their P and N parameters are generated in the first place. 3.2
THE SIGN TEST This simple and common statistical test is called the sign test because, as commonly presented, it requires that a sign (usually a + or - sign, as in Fig. 2.1) be assigned subjects on the basis of some assessment procedure. It is based on the binomial distribution, consequently it is also called the binomial test. It is useful for determining if there are more (or fewer) plus signs than would be expected, if subjects were sampled from a population in which the probability of assigning a + were some specified value, usually .5. Consider the money-cure study. There were 10 subjects in the sample, and 8 improved. The null hypothesis purports that subjects were as likely to improve as
40
INFERRING FROM A SAMPLE: THE BINOMIAL
not (which is analogous to assuming that a coin is fair) and thus the values for the relevant population parameters are P = .5 and N- 10. This would generate the sampling distribution with binomial coefficients as shown in column K, and probabilities as shown in column L, of the spreadsheet portrayed in Fig. 3.3.
One-Tailed and Two-Tailed Tests For the sign test (and for certain other statistical tests too), it is important to distinguish between one- and two-tailed tests. If a coin is fair, we would expect the number of heads to be one-half the number of trials— a value in the center of the distribution. Any extreme score for the test statistic— for example, one head and nine tails or nine heads and one tail— would suggest that the coin might not be fair. Note that extreme values fall at either the left or the right ends of the distribution (see, e.g., Fig. 3.4), under one or the other of the distribution's two "tails." If we reject the null hypothesis whenever the test statistic is either considerably less or considerably more than expected, then the test is called nondirectional or two-tailed. Symbolically:
H0 : H1:
P=.5. P=.5.
On the other hand, if we think that only more heads than expected, or more clients improving than expected, is of interest, and so choose to reject the null hypothesis only when values of the test statistic fall at one end of the distribution but not the other, then the test is called directional or one-tailed. Symbolically:
H0: HI:
P<.5P > .5.
Note that null and alternative hypotheses are formulated somewhat differently for one- and two-tailed tests. For example, based on the values shown in Fig. 3.3 for N - 10 trials and
P=.5:
1. The probability of all heads is .001 (one-tailed) but the probability of either all heads or all tails is .002 (two-tailed). 2. The probability of nine or more heads is .011il (one-tailed) but the probability of nine or more heads or nine or more tails is .021 (two-tailed). 3. The probability of eight or more heads is .055 (one-tailed) but the probability of eight or more heads or eight or more tails is .109 (twotailed). These probabilities were computed using the or rule, which means that individual probabilities were summed. Thus .001 + .010 + .044 = .055. If you try to recreate these values on a hand-held calculator from those displayed in Fig. 3.2, and you round at intermediate steps, your answers may vary slightly from those given here. Numeric results given here and throughout the text were computed with a spreadsheet and intermediate results were not rounded. Only the final scores are rounded and then usually three significant digits are retained. If the alpha level were .05, if the test were two-tailed, and if the value of the test statistic were 0, 1, 9, or 10, we would reject the null hypothesis. These outcomes, which are termed critical values and constitute the region of rejection or the critical region (see chap. 2), would occur 2.1% of the time or less by chance
3.2 THE SIGN TEST
41_
alone. If the alpha level were again .05, but the test one-tailed, again we would reject if the value of the test statistic were 9 or 10. (Arbitrarily, and just as a convenient convention, if a test is one-tailed we assume values that might cause rejection are in the right-hand tail unless explicitly stated otherwise.) These outcomes would occur 1.1% of the time by chance alone, whereas values of 8 or higher would occur 5.5% of the time—just a little too often for a value of 8 to be included in the rejection region for an alpha level of .05, one-tailed. Although it is true that the probability of tossing exactly 8 heads with a fair coin is .044, regions of rejection are constructed by accumulating probabilities from the extremities of the tails inward. If the alpha level were .10, however, critical values for a one-tailed test would be 8, 9, or 10. These outcomes would occur 5.5% of the time by chance alone. Critical values for a two-tailed test would be 0, 1, 9, or 10. These outcomes would occur 2.1% of the time, as noted earlier. The critical values would not include 2 and 8. The probability of tossing 8 or more heads or 8 or more tails is .109, which is too high for values of 2 and 8 to be included in the critical region for an alpha level of .10, two-tailed. Finally we can answer the question posed at the beginning of chapter 2: Is 8 out of 10 clients improving worthy of attention? If our alpha level were .05, it would not matter whether the test was one- or two-tailed—we would not reject the null hypothesis. However, if our alpha level were .10, we would reject for a one- but not for a two-tailed test. A test statistic as high as 8 (one-tailed) would occur 5.5%, and a test statistic as extreme as 8 (two-tailed) would occur 10.9% of the time if the null hypothesis were true. This is not strong enough evidence to reject the null hypothesis at an alpha level of .05, one- or two-tailed, or at an alpha level of .10, two-tailed. But a value of 8 falls in the critical region for an alpha = .10, one-tailed sign test. But note, an alpha level of .10 is used in this paragraph largely for illustrative purposes. In research reports published in scientific journals, alpha levels are almost always .05 or less. Exercise 3.4 Critical Values for the Sign Test This exercise provides practice in determining critical values for the sign test. 1. Verify that all probabilities and percentages given in the previous several paragraphs are correct (if you were not already doing this.) 2. Using Pascal's triangle, compute the number of outcomes in each class, along with their associated probabilities, for N = 11 and N= 12. 3. Construct a table of critical values for the sign test. (A critical value is one that causes you to reject the null hypothesis.) Include nine rows, one each for number of trials = 4 through 12. Include four columns, two for one-tailed and two for two-tailed tests. Within each kind of test, include a column for alpha = .05 and for alpha = .10. Indicate with an "x" combinations for which no values exist that would justify rejecting the null hypothesis. For example, the N = 10 row would look like this: ...
10
One-tailed
Two-tailed
a = 05
a = .10
a = .05
a =.10
9-10
8-10
0-1,9-10
0-1,9-10
42
INFERRING FROM A SAMPLE: THE BINOMIAL
Using the Sign Test The table you have just constructed (Exercise 3.4, part 3) works as long as there are no more than 12 subjects in a study. But if the need arose, you know enough so that, in principle at least, you could lengthen this table. More importantly, the process of computing values for this table should have helped you gain insight into how such tables of critical values for sampling distributions are constructed in general. In practice, of course, investigators rarely compute critical values. Instead, they consult tables constructed by others and indeed such a table is provided here for the sign test (see Table A in the statistical tables section). When contemplating a statistical test, the first step is to decide which test is appropriate. The sign test is appropriate if a binomial (literally, bi = two, nomin = name) code can be meaningfully assigned each subject (e.g., assigning each subject a + or a - based on some assessment procedure) and if the assessments are made independently so that the outcome of one cannot affect the outcomes of others. Assuming that a sign test is appropriate, the next steps require that you commit to an alpha level, decide on an appropriate null and alternative hypothesis (which will imply certain values for the relevant population parameters), and (as a related matter) decide whether a one- or two-tailed test is appropriate. The final step is the statistical decision. From the appropriate table, determine which values of the test statistic would occur rarely (their combined probabilities cannot exceed your preselected alpha level) if your null hypothesis were true. If the test statistic derived from your sample is one of those rare values (i.e., if it falls in the critical region) then you reject the null and necessarily accept the alternative hypothesis. Exercise 3.5 Applying the Sign Test This exercise provides practice in applying the sign test. You will need to refer to Table A in the Statistical Tables section. 1. An expanded study testing the efficacy of the money cure contains 40 subjects. By chance, you expect 20 to respond to treatment, but in fact 30 do respond. You decide beforehand that you will reject the null hypothesis that treatment has no effect only if more subjects improved than expected and only if the specific result would occur less than 5% of the time if the null hypothesis were in fact true. Do you reject the null hypothesis? Why or why not? 2. What if only 26 improved? Would you reject the null hypothesis? Why or why not? 3. What if 30 improved, but you had decided beforehand to reject the null hypothesis if the number of subjects was different from expected, either more improving or more worsening than expected. Keeping the alpha level at .05, would you reject the null hypothesis now? Why or why not? Given these same circumstances, what would your decision be if 26 improved? Explain your decision. 4. Given 50 subjects, an alpha level of .05, and a two-tailed test, what is the smallest number of subjects who, even if they all improved, would not allow you to reject the null hypothesis? What is the largest number of subjects who, even if they all improved, would not allow you to reject the null hypothesis?
3.2 THE SIGN TEST
43
5. Given 50 subjects, an alpha level of .05, but a one-tailed test, what is the largest number who, even if they all improved, would not allow you to reject the null hypothesis? What is the smallest number that would allow you to reject the null hypothesis? 6. Now answer question 4, but for an alpha level of .01. 7. Now answer question 5, but for an alpha level of .01. At this point you are now able to use a simple test, the sign test, and the logic of hypothesis testing in order to answer research questions like the one posed at the beginning of chapter 2: How many patients would need to get better before we could claim statistical significance for the result? The next exercise will instruct you in how to conduct a sign test using SPSS. Exercise 3.6 The Sign Test in SPSS This exercise provides practice in using the sign test in SPSS. 1. Invoke SPSS and create two new variables in the Variable View window. Name the first variable "outcome" and the second variable "freq." Set the number of decimal places for both variables to 0. 2.1 Provide appropriate value labels for the outcome variable. In row one, click on the values cell and then click again on the grey box that appears to the right of the cell. This will open the Value Labels dialog box. Enter "1" and "improved" in the Value and Value Label windows, respectively. Then click on Add to enter the information in the bottom window. Now do the same, but this time enter "0" and "no improvement". After you enter labels for "0" and "1", click on OK. Now, the labels improved and no improvement will appear in the SPSS output, instead of the rather cryptic codes 1 and 0. 3. Weight the cases by the frequency variable. Select Data->Weight Cases from the main menu. In the Weight Cases dialog box, select the Weight cases by radio button and move the freq variable to the Frequency variable window. Click on OK. This command tells SPSS to weight each value for outcome (i.e., improved or no improvement) by the number indicated in the freq variable. You could simply enter a 1 or 0 for each case in the outcome column, but doing so would become tedious when N is large. 4. In Data View enter a 1 in the first row of the outcome column and a 0 in the second row. In the freq column enter 30 in the first row and 10 in the second row. Because you weighted the cases by the freq variable in step 3, this is the same as entering 30 ones and 10 zeros in the outcome column. 5. Select Analyzed->Nonparametric Tests->Binomial from the main menu. Move the outcome variable to the Test Variable List box. Note that the Test Proportion window reads .50 by default. This indicates that you expect 50% of the outcomes to be positive. Click on OK. 6. Look at the output. Is the N correct? Look at the Observed Prop. column. What proportion improved? What was the expected proportion based on the Test Prop. column? Now look at the final column labeled Asymp. Sig (twotailed). This tells you the probability of finding 30 out of 40 positive outcomes if you expected only 20. If the value is less than alpha, then you reject the null hypothesis that that treatment has no effect. 7. Change the values in the freq column to reflect that 26 improved and 14 did not. Would you reject the null hypothesis with an alpha of .05?
44
INFERRING FROM A SAMPLE: THE BINOMIAL 8. What if 30 improved, but you expected 80% to improve? Keeping the alpha level at .05, would you reject the null hypothesis? You are now able to conduct a simple sign test by hand and using SPSS. In subsequent chapters other tests for statistical significance are described—those based on the normal, t, and F distributions instead of the binomial—but first some matters of basic terminology (chap. 4) and descriptive statistics (chap. 5) need to be discussed.
Note 3.1 Critical values
Values of a test statistic that would occur 5% of the time or less (assuming an alpha level of .05) if the null hypothesis were true. Depending on the test statistic used and how the null hypothesis is stated, these values could fall only at the extreme end of one tail of a distribution or at the extreme ends of both tails. If a test statistic assumes a critical value, the null hypothesis is rejected. For that reason, critical values define the critical region or the region of rejection of a sampling distribution.
4
Measuring Variables: Some Basic Vocabulary
In this chapter you will: 1. Learn the difference between qualitative and quantitative variables. 2. Learn the difference between nominal, ordinal, interval, and ratio scales. 3. Learn how to describe a study design in terms of independent and dependent variables. 4. Begin to learn how to match study designs with statistical procedures. In talking with others, whether describing your study to fellow researchers or attempting to gain statistical advice, it is important to use relatively standard, clear vocabulary. This chapter describes some basic terms and concepts used to talk about research studies. Some of these terms deal with the variables under study and how they are measured, and others deal with how the variables are related to each other in the study design. 4.1
SCALES OF MEASUREMENT All studies, including simple studies like the money-cure study used as a running example in chapters 2 and 3, incorporate one or more variables. The variables— so named because, unlike constants, they can take on various values at different times—identify the concepts the investigator thinks important for a given study. The variable of interest for the money-cure study was outcome: Did patients get better or not. The design included only one kind of treatment (and so this was constant), but an expanded study might well have defined two different kinds or levels of treatment (e.g., the money cure and a standard talk cure, or the money cure and no treatment). In this case, two variables would be identified, type of treatment and outcome, and the research question would involve the connection, if any, between the two. An important attribute of variables is their scale of measurement. Knowing the kind of scale used to assign values to variables helps the investigator decide which statistical procedures are appropriate. Following a scheme first introduced by Stevens (1946), four kinds of scales are identified:
45
46
MEASURING VARIABLES: SOME BASIC VOCABULARY 1. 2. 3. 4.
Nominal (or categorical). Ordinal. Interval. Ratio.
Nominal or Categorical Scales Nominal variables assume named values, that is, their permissible values are qualitative instead of quantitative. And although variables can be assigned different categories or levels, there is no inherent reason to order the categories in any particular way. Sex (female/male), religion (Catholic/ Jew/ Muslim/ Protestant/Other), and outcome (improved/did not improve) are examples.
Ordinal Scales Values for ordinal variables are also named. In other words, measurement results in assigning a particular name, level, or category to the unit (subject, family, trial, etc.) under consideration. But unlike nominal variables, for ordinal variables there is some rationale for ordering the categories in a particular way. Examples are freshman/sophomore/junior/senior, and first/second/third, and so forth. There is no reason, however, to assume that the distance between the first and second categories is in any way equivalent to the distance between the second and third pair, or any other adjacent pair, of categories. In sum, ordinalscale values represent degree, whereas nominal-scale values represent only kind. Ordinal data can be regarded as quantitative (in a relatively loose sense), but nominal data remain forever qualitative.
Interval and Ratio Scales Like ordinal scales, both interval and ratio scales reflect an underlying and ordered continuum, but unlike ordinal, these scales are characterized by equal intervals. That is, an interval anywhere on the scale (e.g., the interval between 4 and 5) is assumed equivalent to any other interval (e.g., the one between 12 and 13. Interval and ratio scales differ in only one respect. For an interval scale the placement of zero is arbitrary, whereas for a ratio scale zero indicates truly none of the quantity measured. Thus if the ratio of two numbers is 3:1, the second value represents three times more of the quantity measured only for ratio but not interval data. Examples of interval scales are IQ, scores on most attitude and personality tests, and temperature measured on the Fahrenheit or Celsius scales. Examples of ratio scales are age, weight, and temperature measured on the Kelvin scale (for which zero implies no molecular movement). The scale of measurement can itself be viewed as a nominal (or, arguably, ordinal) variable. It can assume any of four values and the value for a particular case can be determined by asking, at most, three yes/no questions (later on, in chap. 10, we will learn to say that three degrees of freedom are associated with the four categories). These questions or distinctions, and their permissible values, can be represented with a tree diagram (see Fig. 4.1).
47
4.1 SCALES OF MEASUREMENT
FIG. 4.1. Tree diagram for the four types of measurement scales as represented by three binary questions. In practice, the distinction between nominal (qualitative) and other (quantitative) scales of measurement is more important than the distinctions among ordinal, interval, and ratio scales. For most statistical procedures, the difference between interval and ratio variables is not consequential. In addition, under many circumstances it is often regarded as acceptable to apply statistical procedures that were developed for equal-interval to ordinal data. However, it is never acceptable to apply statistical procedures designed for equal-interval data to any numbers arbitrarily associated with nominal categories (with the possible exception of binary categorical variables, if one category can be regarded as representing more of some attribute than the other).
Note 4.1 Nominal
The levels of a nominal or categorical scale are category names, like Catholic | Jew| Muslim [Other for religion or male | female for sex. Their order is arbitrary; there is no obviously correct way to order the names.
Ordinal
The values or levels of an ordinal scale are named but also have an obvious order, like first| second third or freshman | sophomore |junior | senior. However, there is no obvious way to quantify the distance between levels or ranks.
Interval
The intervals of an interval scale are equal, no matter where they fall on the measurement continuum. The placement of zero, however, is arbitrary, like zero degrees Fahrenheit.
Ratio
The intervals of a ratio scale are also equal, but zero indicates truly none of the quantity, like zero degrees Kelvin. Thus the ratio of two numbers is meaningful for numbers measured on a ratio but not an interval scale.
48 4.2
MEASURING VARIABLES: SOME BASIC VOCABULARY DESIGNING A STUDY: INDEPENDENT AND DEPENDENT VARIABLES In addition to the distinction between qualitative and quantitative variables, a second distinction, likewise important because it helps the investigator decide which statistical procedures are appropriate, is between independent and dependent variables. Whether a variable is regarded as independent or dependent depends on the thinking of the investigator. Thus, a particular variable might be viewed as independent in one study but dependent in another. The dependent variable (also called the criterion or response variable) identifies the variable the investigator wants to account for or explain, whereas the independent variable (also called the predictor or explanatory variable) identifies the variable (or variables) that the investigator thinks may account for, or affect, the different values the dependent variable assumes. Strictly speaking, it is not always necessary that variables in a study be segregated into these two classes. In some cases an investigator may be interested only in the association among a set of variables and may not think of any variables as prior in some way to others. More typically, however, investigators are concerned with explanation. Reflecting this concern, the studies used as examples in this book all contain variables that can be regarded either as independent (i.e., logically prior and so independent of other variables named in the study) or as dependent (i.e., their values presumably depend on values for the independent variable or variables). The distinction between independent and dependent variables is more important than the actual words used to express it. We could just as well refer to explanatory and response variables instead. These terms have the merit of being more general but are less used (except when discussing log-linear analyses). Or, we could refer to predictor and criterion variables, which are terms commonly used in the multiple regression literature. Still, the terms independent variable (IV) and dependent variable (DV) have the merit of being the most widely used— and, as used by most writers, no longer have the once exclusively experimental connotation.
Experimental and Observational Studies In traditional usage the terms independent variable and dependent variable have been reserved for true experimental studies. That is, values or levels (usually called treatment conditions) for the independent variable would be determined (manipulated) by the experimenter, subjects would be randomly assigned to a particular level or condition, and then the subjects' scores for the dependent variable would be measured. As students in introductory statistics courses learn, experimental studies support causal conclusions in a different way than merely correlational or observational studies. In such studies, variation among variables is merely observed; there is no manipulation of independent variables and no random assignment of subjects to treatments. As a result, if some association is observed between independent and dependent variables, there is always the suspicion that their covariation reflects, not a causal link from one variable to the other, but the influence of a third variable, unmeasured in the study, that affects them both. In an experimental study, on the other hand, because values for any possibly influential third variables should be distributed by random assignment equally among the experimental conditions, the third variable alternative to a causal interpretation is rendered unlikely.
4.2 INDEPENDENT AND DEPENDENT VARIABLES
Dependent
Independent
49
Note 4.2 The variable a researcher wants to explain or account for is called the dependent variable or DV. It is also called the criterion variable or the response variable. Variables a researcher thinks account for or affect the values of the dependent variable are called independent variables or IVs. They are also called predictor variables or explanatory variables.
In contemporary usage, the terms independent and dependent variable indicate not necessarily experimental manipulation, but simply how investigators conceptualize relations among their variables. Such usage, however, should not lull us into making stronger causal claims than our procedures justify. In general, only experimental studies can justify strong causal claims. 4.3
MATCHING STUDY DESIGNS WITH STATISTICAL PROCEDURES The qualitative versus quantitative and independent versus dependent distinctions are used when describing a study's design. The question "What is the design of your study?" means: What variables do you think are important for this study? Which do you regard as dependent, which independent? What is their scale of measurement? Answering these questions, along with one or two more involving how independent variables are related and whether or not dependent variables involve repeated measurements, both specifies the design and aids in selecting statistical procedures appropriate for that design. In the following paragraphs, a number of common design considerations are described and appropriate statistical procedures named. Almost all of the remaining chapters in this book then describe how the more basic of these procedures are put into practice. For example, the two variables identified as important for the money-cure study were treatment and outcome. Outcome is viewed as the dependent variable, treatment as the independent variable. Both are nominal. There were two values for outcome: improved or not. However, there was only one value for the treatment—that is, all subjects received the same treatment. As we saw, for such a design (a single group, a binary dependent variable) a sign test was appropriate. However, this is a relatively weak design. A stronger design would include a control or comparison group. For example, the independent variable could be represented by two levels: the money treatment and a standard talk treatment. This would result in a design involving qualitative independent and dependent variables, both scored with more than one level or category. Data for such designs are the counts associated with the cells of the design—that is, the number of subjects who received the standard treatment who improved, the number who did not, the number who received the money treatment who improved, and the number who did not—resulting in a two by two table of counts. These counts are typically analyzed with chi-square statistics for two-dimensional tables (like the present example) and with loglinear approaches (Bakeman & Robinson, 1994; Bishop, Fienberg, & Holland, 1975) when tables include two or more dimensions (each categorical variable defines a dimension of the cross-classification or contingency table). As noted in
50
MEASURING VARIABLES: SOME BASIC VOCABULARY the preface, such analyses, which require computational tools not provided by spreadsheets, are discussed in Bakeman & Robinson (1994). Perhaps the most common design, insofar as there is one in the behavioral sciences, involves a quantitative dependent variable and one or more qualitative independent variables. Such data are analyzed with the analysis of variance (ANOVA), a statistical approach that has received considerable attention and has been much used in the behavioral sciences. If the independent variables were quantitative, correlation or multiple regression would be used instead. And if the independent variables were a mixture of qualitative and quantitative, again multiple regression would be used. In fact, as noted in chapter 1, the analysis of variance can be presented as one straightforward application of multiple regression, although not all texts emphasize this simplifying fact. For completeness, logistic regression and discriminant function analysis should be mentioned because they are appropriate techniques to use when the dependent variable is qualitative and the independent variable quantitative. Multivariate analysis of variance (MAN OVA) and multivariate regression should also be mentioned in recognition of the fact that some designs involve multiple dependent quantitative measures. These can be important techniques, but their use is regarded as an advanced topic in statistics and hence beyond the scope of this book. For a clear explication of these and other multivariate techniques, see Tabachnick and Fidell (2001). The preceding discussion (summarized in Fig. 4.2) has emphasized the scale of measurement for the independent and dependent variables (and has mentioned the number of dependent variables), but other distinctions are important and are often made. For example, it is common to distinguish between parametric and nonparametric tests. Parametric tests require assumptions about the distributions underlying the data, whereas nonparametric tests do not require such assumptions. In practice, nonparametric tests are typically applied to categorical or ordinal data, and parametric tests, which are emphasized here, to quantitative data. The traditional reference for a number of nonparametric techniques is Siegel (1956). Another distinction commonly made concerns the nature of the research question. Many authorities (e.g., Tabachnick & Fidell, 2001) find it convenient to distinguish questions that ask about relations among variables (for which correlation and regression are appropriate), from questions that ask about the significance of group differences (for which analysis of variance is appropriate), from questions that attempt to predict group membership (for which discriminant function analysis is appropriate), from questions that ask about the underlying structure of the data (for which factor analysis is appropriate). Explication of the techniques required for the first and second kinds of questions constitutes the bulk of this book, but in practice— because of the integrated multiple regression approach adopted here— the distinction between the two is somewhat blurred. Technically, however, the assumptions required for tests of association and tests of group differences are somewhat different (again see Tabachnick & Fidell, 2001). As noted previously, the statistical techniques required for the third and fourth kinds of questions constitute an advanced topic in statistical analysis and are not covered here.
51
4.3 MATCHING DESIGNS WITH STATISTICAL PROCEDURES
Independent Variables Single categorical
Dependent variable Categorical
Quantitative
Chi-square
ANOVA. Hest
Multiple categorical
Log-linear analysis
ANOVA
Single quantitative
Discriminant function
Simple correlation
Multiple quantitative
Discriminant function
Multiple regression
Categorical and Quantitative
Logistic regression
Multiple regression
FIG. 4.2. Appropriate statistical techniques for studies with categorical and quantitative dependent variables and various kinds and numbers of independent variables. Techniques appropriate for analyzing quantitative dependent variables are emphasized in this book. One technique that usually appears in introductory texts but is not discussed here in any detail is Student's t test. The t test is appropriate for designs with a quantitative dependent variable and a single binary independent variable. Thus it can be used to answer questions like, "Is the mean IQ at 3 years of age different for preterm and full-term infants?" The t test was developed earlier than the analysis of variance and is limited to two groups. The analysis of variance can analyze for differences between two groups and more besides. In the interest of economy (why learn two different tests when only one is needed?), we have decided to present only the more general ANOVA technique for analyzing problems that earlier would have been subjected to a t test. As Keppel and Saufley (1980) noted: The t test is a special case of the F test. ... If you were to conduct a t test and an F test on the data from the same two-group experiment, you would obtain exactly the same information. ... The two statistical tests are algebraically equivalent, that is,
Due to this equivalency, we decided not to develop the t test. . . . The F test can be applied to almost any situation in which the t test can be used, but it can also be applied in situations where the t test cannot be used. (p. 109) Do not misunderstand. We are not claiming that the t test is of no use and should not be used. Moreover, in order to understand much of the literature, its application needs to be understood. And pedagogically, there is much to be gained by a thorough explication of the t test and its variants. In a longer volume, it would have a place. Our intent in writing this book, however, was to present basic statistics in as economical and efficient a manner as possible, which means that concepts and techniques that have wide application (like multiple regression
52
MEASURING VARIABLES: SOME BASIC VOCABULARY and analysis of variance) are emphasized and other topics with more limited application (like chi-square and the different kinds of t tests) are omitted. The purpose of this chapter was first to introduce some basic vocabulary that will be used in this book, and second, to introduce the names of several useful statistical techniques with which you should be familiar, even though explication of these techniques is beyond the scope of this book. It is important, of course, to know how to apply particular statistical techniques, but it is even more important to know which techniques should be applied in the first place—and in order to do that, readers need to at least be aware of a wide range of possibilities. The importance of matching statistical procedures to study designs has been emphasized in the last several paragraphs. It is a key issue that will occupy us throughout the remainder of this book. In the next chapter, however, we discuss some simple and basic ways to describe data.
5
Describing a Sample: Basic Descriptive Statistics
In this chapter you will: 1. Learn how to summarize a sample (a particular set of scores) with a single number or statistic, the arithmetic mean. 2. Learn what it means to say that the mean is the best fit, in a least-squares sense, to the scores in the sample. 3. Learn how to describe the variability in a sample and in a population using the variance and the standard deviation. 4. Learn how to compare scores from different samples or populations using standard scores (Z scores). 5. Learn how to identify scores so extreme that they might possibly be measuring or copying mistakes. If you never wanted to generalize beyond a sample, inferential statistical techniques like hypothesis testing would be unnecessary. But very likely you would still want to describe what your sample was like. Basic descriptive statistical techniques, like those described in this chapter, allow you to do exactly that. In the last chapter we distinguished among variables measured on nominal, ordinal, interval, and ratio scales. Descriptive statistics for nominal data, as the running example of the money-cure study demonstrates, are extremely simple, consisting of little more than counts (or proportions), for example, the number (or percentage) of people who improved. Descriptive statistics for ordinal data (unless we think it justified to regard differences between ranks as equal) are no more complex. Again, we simply report the numbers and proportions of subjects who fall in the different categories. Describing interval or ratio variables offers more possibilities. In this chapter we discuss the mean, the variance, and the standard deviation, which are basic descriptive statistics appropriate for summarizing quantitative data.
53
54_ 5.1
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS
THE MEAN For economy of presentation, if for no other reason, it is often useful to characterize an entire set of scores with just one typical score. There are several different ways to compute this typical, or average, score. For example, the mode, the median, the arithmetic mean, and the geometric mean all involve different computations and often result in different values. Each of these typical scores has its uses, as students in introductory statistics courses learn, but by far the most commonly used average score in statistical work is the arithmetic mean. In fact, whenever the word mean is used in this book, it is safe to assume that it means the arithmetic mean.
The Mean Defined Almost certainly you learned how to compute the arithmetic mean long ago. The rule is (a) add the scores together and (b) divide their sum by the number of scores. Symbolically (using I, the Greek upper case sigma, to indicate summation, Y to represent a raw score, and N to indicate the number of scores), the mean for the Y scores is
We could leave matters there, but there is something to be learned if the definition of the mean is probed a bit more. You may find this probing rather overwrought for something so basic, but our purpose is to introduce an important concept (the method of least squares) within the simplest context possible. In developing this definition of the mean as a best-fit statistic in a least-squares sense, we make use of data derived from a simple (imaginary) study, which is described next and is used as an example in this and subsequent chapters.
An Example: The Lie Detection Study Imagine that we invite 10 subjects into our laboratory for a polygraph examination. We provide them with the script of the questions the examiner will ask and we instruct them to lie when answering certain of those questions. Different subjects provide false answers to different questions, but all subjects tell the same number of lies. Then we ask the examiner to identify untruthful responses. Thus the dependent variable (DV) for this study is the number of lies detected by the examiner. The data for this study are given in Fig. 5.1 and, as you can see, the number of lies detected ranged from two to nine. The next exercise shows you how to manipulate these data using a spreadsheet. Although this exercise uses the data from the lie detection study, it could easily be adapted for use with data from other studies as well.
Exercise 5.1 The Mean The spreadsheet template developed during this exercise computes basic descriptive statistics for the lie detection study. Moreover, it will form the basis for subsequent exercises. Give some attention to the notation used and defined in this exercise. It is used throughout the book.
5.1 THE MEAN
55
General Instructions 1. Setup and label five columns. Each row represents a subject and the first column contains that subject's number (1-10). 2. The second column contains values for the dependent variable, which is the number of lies detected for each subject as given in Fig. 5.1. Label this Y, which represents a generic DV. 3. The third column contains predicted scores. Label this Y' (read as "V-prime") and point each cell in this column to a cell that contains 5 (use an absolute address). You could enter the value 5 (which is an arbitrary value selected for this exercise) into the cells in this column directly, but if you point the cells to another, single cell you have the option of later changing the predicted values for all subjects by just changing one cell. 4. In the fourth column, enter a formula for the differences between the raw and the predicted scores. This difference score is symbolized with a lower case y, and is often called a deviation or residual score. Label this column Y-Y'. 5. In the fifth column, enter a formula for the square of the difference scores. Label it y*y (where y = Y-Y1). 6. On separate rows, enter formulas that compute the sum, the count, and the mean for the raw, the predicted, the difference, and squared difference scores. You may use Fig. 5.2 as a guide if you wish. Detailed Instructions 1. In row 2, label the columns as follows: Label
Column
Meaning
s
A
Y
B
Y'
C
The subject number or index. In this case it ranges from 1 to 10. In general, the subject could be an individual, a dyad, a family, or any other appropriate unit of analysis. The number of lies detected, actually observed. In general, Y indicates an observed score. For the present example, seven lies were detected for the fifth subject. The number of lies detected, predicted by you (read as "Vprime"). Usually the basis for prediction will be clear from context.
0 .. . Subject
No. lies detected
1
3
2 3 4 5 6 7 8
2 4 6 6 4 5 7
9 10
7 9
FIG. 5.1. Data for the lie detection study.
56
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS The difference between observed and predicted. This is the amount left when the predicted score is subtracted from the observed score. This difference score is symbolized with a lower case y, and is often called a deviation or residual score. y*y E The square of this difference, that is, Y-Y' multiplied by Y-Y'. The sum of the entries in this column is called a sum of squares (symbolized as SS) because the difference scores are squared and then summed. 2. To remind ourselves that the sum of the column E entries is the sum of the Y deviation scores squared, enter the label "SSy" in cell E1. In addition, enter the labels "Lies" in cell B1 and "y=" in cell D1. 3. In column A, label rows 13-16 as follows:
4. 5. 6. 7. 8. 9. 10. 11. 12.
Y-Y'
D
Label
Row
Meaning
Sum= 13 The sum of the scores in rows 3-12. N= 14 The number of scores in rows 3-12. Mean= 15 The mean of those scores. a= 16 The predicted score. Enter the values 1 through 10 in column A, rows 3-12. Enter the observed scores in column B, rows 3-12. Put a value in cell B16. For now, make that value 5. Point the cells in the Y' column (cells C3-C12) to the predicted cell (B16), that is, put the address B16 in C3-C12. Cells C3-C12 and B16 should now all display the same value. Enter a formula for subtracting Y' (the column C entry) from Y (the column B entry) in cells D3-D12. These are the deviation scores or the residuals. Enter a formula for multiplying Y-Y' (the column D entry) by itself in cells E3E12. These are the deviations squared. In cells B13-E13, enter a function that sums the entries in rows 3-12. In cells B14 and C14, enter a function that counts the entries in rows 3-12. In cells B15 and C15, enter a formula that computes the mean (the sum divided by N) for that column. At this point, your spreadsheet should look like the one portrayed in Fig. 5.2.
Predicting the Mean At this point, your spreadsheet should look like the one portrayed in Fig. 5.2. It shows the raw data, the arithmetic mean for those data, and the sum of the squared deviation scores, where a deviation consists of the difference between the raw score and the value 5. You probably wonder why we choose 5 as the predicted value in Exercise 5.1. We took a quick look at the numbers, noted they ranged from 2 to 9 but that there were two 2s. Then hurriedly and somewhat arbitrarily, we decided to guess that a typical value for this set of 10 numbers was 5. This was only a first guess, selected for purposes of exposition; we will revise it shortly. Actually, we had in mind a formal model for these scores:
Yi = a + e/
(5.2)
This particular model states that the score for the ith subject in the population, Yi, consists of some population parameter, represented with a (lower case Greek
57
5.1 THE MEAN
alpha), plus an error component, Y' (lower case Greek epsilon), specific to that subject. This is a very simple model. It suggests that the value of a score is determined by a single parameter and nothing more. Not all scores will have exactly the same value as the parameter, which is what the error component signals, but if asked to predict the value for any score, our prediction would always be a, the value of the parameter. If Y' (readY -prime, for Y-predicted) represents the predicted value for Y, and if a represents the estimated value for the population parameter a, then the prediction equation associated with this model is:
Yi' = a
(5.3)
For Exercise 5.1, we guessed that a was 5; thus the specific prediction equation became Yi' - 5 (or, omitting the subscript, simply Y - 5). For this prediction model, characteristics of individual subjects are not considered. The same prediction is applied to all, which is why all the predicted values in column C (see Fig. 5.2) are the same. In general, a model should indicate concisely how we think a particular dependent or criterion variable is generated. In that sense, it is an idealization of some presumed underlying reality, and presumably it reflects the mechanism that produces the criterion variable. Usually, a model contains one or more population parameters (like the a in Equation 5.2). In addition, and unlike Equation 5.2, it often contains one or more independent or predictor variables as well and so indicates how we think those predictor variables are related to the criterion variable under consideration.
D
y=.
Lies Y
Y'
Y-Y1
y*y
3 2 4
5 5 5
-2 -3 -1
4 9 1
5
1
1
6 7 8 9 10
6 6 4 5 7 7 9
5 5 5 5 5 5
1 -1 0 2 2 4
.1 1 0 4 4 16
Sum= N=
53 10
50 10
3
41
Mean=
5.3 5
5
s
1 2 3 4 5
a=
SSy
FIG. 5.2. Spreadsheet for the lie detection study using V"= 5 as the prediction equation.
_58
__
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS
The Equation 5.2 model makes sense because we have not yet really thought about the lie detection study. We have not yet considered how characteristics of individual subjects or the circumstances surrounding their testing sessions might have affected their responses. In other words, we have yet to identify research factors (independent variables) that we want to investigate. Thus, this simple model represents a starting point. Later on we will ask whether other, more complex models allow us to make better predictions than the initial model posed here, and it will prove useful then to have this simple model as a basis for comparison. But even if we provisionally accept this simple model and its associated prediction equation, have we found the best estimate for the parameter? The value 5 was after all a rather offhand guess. Can we do better? Is there a best value? The answer is yes, if we first define what we mean by best. A common criterion for best, one that is often used in statistics, is the least-squares criterion. According to this guide, whatever value provides us with the lowest (or least) possible value for the sum of the squared error scores (or squares) is best. The Equation 5.2 model, rewritten using estimates instead of parameters, is: Yi = a + ei
Substituting Yi' for a (because Y,' = a, Equation 5.3), this can be rewritten as: Yi=Yi' + ei Rearranging terms, this becomes: ei=Yi-Yi' In other words, the error score for each subject is the difference between that subject's actual score and the predicted score for that subject. In the last exercise the error scores (also called deviation scores or residuals because they indicate what is left after the predicted is subtracted from the observed) were symbolized with a lower case y (omitting subscripts, y = Y - Y'). When we used 5 as our predicted value, the sum of the squared error scores, which is the quantity we wish to minimize, was 41 (see Fig. 5.2). If no other value yields a sum of squares (as the sum of the squared residuals is usually called) lower than this, then we would regard 5 as the best estimate for the parameter, in the least-squares sense. As you have probably already guessed, 5 is not the best estimate. For pedagogic purposes, we deliberately choose a number that was close but not best. The best estimate, based on our sample of 10 scores, is 5.3, which is the mean value for these scores. Statisticians have proved to their satisfaction that the mean provides the best-fit, in the least-squares sense, to a set of numbers. When the mean is used to predict scores (when Y,' = M), the sum of the squared deviations between observed and predicted scores will assume its lowest possible value. This is easy to demonstrate, as the next exercise shows. Exercise 5.2 The Sum of Squares The purpose of this exercise is to demonstrate that the minimum value for the sum of the squared deviations is obtained when the mean is used to form deviation scores.
5.1 THE MEAN
59
1. Using the spreadsheet from the last exercise, enter different values in the cell that now contains 5, the predicted value or the value for the parameter a. For example, enter 3.5, 4, 4.5, 5, 5.5, 6, and so forth, and note the different values that are computed for the sum of squares, that is, the sum of the squared deviation scores. You should observe that the minimum value is achieved only when the mean (in this case, the value 5.3) is used. 2. Verify that using the mean indeed yields the smallest value for the sum of squares by entering values like 5.31, 5.32, 5.29, 5.28, and so forth. When the value 5.3 is used to form deviation scores, the sum of squares (usually abbreviated SS) should be 40.1, which is its lowest possible value for these data, and your spreadsheet should look like the one given in Fig. 5.3. The principle of least squares—which, once presented, may seem like an obvious way to minimize errors of prediction—was first articulated in 1805 by a French mathematician, Adrien Marie Legendre. He wrote, "We see, therefore, that the method of least squares reveals, in a manner of speaking, the center around which the results of observations arrange themselves, so that the deviations from that center are as small as possible" (cited in Stigler, 1986). The presentation here may have seemed like a long and involved way to present something as seemingly simple as the arithmetic mean, but consider what you have learned. On the practical side, you have had practice setting up and using a spreadsheet for a simple study. You have also learned to view the mean as the value that best represents or fits a set of numbers, using the least-squares criterion. This concept will serve you well later when the least-squares criterion is used in more interesting and more complex contexts. And, as you will discover in the next two sections, the groundwork required for computing the variance and the standard deviation is already insinuated in the spreadsheet you have just developed.
A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
B s 1 2 3 4 5 6 7 8 9 10
Sum=
N= Mean=
a=
Y 3 2 4 6 6 4 5 7 7 9 53 10 5.3 5.3
E
D
C Lies
Y' 5.3 5.3 5.3 5.3 5.3 5.3 5.3 5.3 5.3 5.3 53 10 5.3
y=
SSy
Y-Y' -2.3 -3.3 -1.3 0.7 0.7 -1.3 -0.3 1.7 1.7 3.7 0
5.29 10.89 1.69 0.49 0.49 1.69 0.09 2.89 2.89 13.69 40.1
y*y
FIG. 5.3. Spreadsheet for the lie detection study using Y' = 5.3 for the prediction equation.
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS
60
Note 5.1 Y
An upper case Y is used to indicate a generic raw score, usually for the dependent variable. Thus Y represents any score in the set generally, and Yi,indicates the score for the ith subject.
Y' Y' (read Y-prime) indicates a predicted score (think of "prime" as standing for "predicted"). This is relatively standard regression notation, although sometimes a Y with a circumflex or "hat" (i.e., a ^) over it is used instead, Y. Usually the basis for prediction will be clear from the context. Individual predicted scores are symbolized Y,-', but often the subscript is omitted. Y-
Y'
SS
5.2
The difference or deviation between an observed score and a predicted score is called a residual or error score. Again the subscripts are often omitted. Often lower case letters are . used to represent residuals. For example, in Fig. 5.3 a lower case y represents the deviation between a raw Y score and the mean for the Y scores. The sums of squares is formed by first squaring each residual, and then summing the resulting squares; thus it is the sum of the squared residuals. In other words, SS = E (Yi - Yi)2 where i = 1,N.
THE VARIANCE The mean is a point estimate because it identifies one typical score, whereas variability indicates range, which is an area within which scores typically fall. Consider the kind of practical problem that first caused mathematicians to ponder variability of scores. Imagine you are a ship's navigator at sea attempting to read the angle of the North Star above the horizon from a moving deck. In a desperate attempt at accuracy, you take successive readings, but each one is somewhat different from the others. Still, you reason, the best estimate you can make for the true value is the mean of your various measurements. You know there is only one true value, so you regard any deviation between that value and one of your measurements as error. Under these circumstances, variability might not be your major concern, unless, of course, you wanted to compare the variability of your measurements with someone else's. But now imagine you are concerned with various anthropometric measurements, for example, the heights of military recruits who come from different geographic areas. In this case, you might well find it interesting to see if the height of Scotsmen, for example, is more variable than that of Japanese. Or, given the lie detection study described earlier, you might want to know if the number of lies detected for the first five subjects (who might have been treated differently in some way) was more variable than the number for the last five. No matter whether scores are several measurements of the same subject, as for the navigation example, or single measurements of different subjects, as for the anthropometric or the lie detection study example, it can be useful to quantify the amount of variability in a set of scores. One common measure of variability is
5.2 THE VARIANCE
61_
the variance. The variance for a sample of scores is simply the mean squared deviation, that is, the average of the squared residuals. In other words, to compute the variance for a sample of scores: 1. 2. 3. 4.
Subtract the mean from each score. Square each difference. Sum the squared deviation scores. Divide that sum by the number of scores.
Symbolically (letting My represents the mean for the Y scores) the variance for the Y scores is:
Most of the necessary computations are already contained in the spreadsheet shown in Fig. 5.3. Exercise 5.3 The Sample Variance The purpose of this exercise is to modify the current template so that, in addition to the mean, it computes the variance for the sample data as well. General Instructions 1. Make sure that the predicted score is always the mean. In other words, point the cell that contains the predicted score to the cell that contains the mean of the raw scores. This allows you to change the raw scores, and hence the mean, insuring that deviation scores will remain deviations from the mean. 2. Add a formula for the sample variance to the current template. Provide an appropriate label. 3. Selectively change the values for the raw scores and note how these changes affect the value of the variance. First, change the raw scores so that they are all quite similar to each other. Then, make just one value very different from the others (e.g., most scores might be 3s and 4s whereas one score might be 94). Finally, select values that are somewhat different, and then a lot different, from each other. What do you observe? Detailed Instructions 1. Put the label "VAR=" for variance in cell D15. 2. Enter 10 (or a counting function) in cell E14. In cell E15 enter a formula for dividing the sum of the squared deviation scores (cell E13) by the number of scores (cell E14). This is the variance. 3. Insure that the predicted value will always be the mean of the raw scores, even if you change the raw scores (i.e., point cell B16 to cell B15). 4. The more scores are spread out around the mean, the greater the variance. This is easy to demonstrate. First replace the most extreme numbers of lies with numbers nearer the mean (in column B). For example, replace the 2 with a 4, the 7s with 6s, and the 9 with a 7. What is the variance for this new set of scores? How does it compare with the original variance of 4.01 ? 5. Next undo the changes you just made (restore the original raw scores) and replace numbers near the mean with more extreme scores. For example,
62
___
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS
replace the 4 with a 1, the 5 with a 2, and the two 7s with 9s. What is the variance now? Why should it be larger than the initial 4.01 ?
Sample and Population Variance For the last exercise you computed a quantity called the sample variance. According to Equation 5.4, this is the sum of squares divided by N. There is a second definition for the variance, which divides the sum of squares not by N, but by N - 1 instead. Symbolically:
Many writers call this the population variance. It is an estimate of the true value of the population variance, based on sample data, and is symbolized here as VAR' to distinguish it from VAR (the sample statistic). As a general rule, in this book addition of the apostrophe or prime indicates an estimate; many texts use the circumflex (i.e., the ^ or hat) over the symbol for the same purpose. Generations of students have been perplexed at first by the difference between the sample and population variance (and the sample and population standard deviation), which is not surprising once you realize that different writers define matters differently. One question concerns the N -1 divisor: Why is it N - 1 instead of N? For now accept that one of the divisors should be N -1 and wait until chapter 10, where we offer an intuitive explanation for the N -1. A second question concerns which divisor is used for the sample, which for the population variance: For example, should N or N-1 be used for the sample variance? Here we have defined the sample variance as the one with the N divisor and the population variance as the one with the N -1 divisor, but some texts and programs do just the opposite. The VAR function in Excel, for example, returns the sums of squares divided by N -1, whereas the VARP function (P for population) returns the SS divided by N. On one point there is no confusion: If one wants to describe the variance for a group, and if scores are available for all members of the group, then one divides the sum of squares by N. Some writers, usually those of a more empirical bent, think primarily in terms of samples, assume they can never know all the scores in the population, identify the group with a sample, and therefore call the SS divided by N the sample variance. If they divide the same SS by N -1, they call it the population variance, presumably because it is an estimate of population variance based on sample data. Other writers, usually those of a more formal or mathematical bent, think primarily in terms of populations, assume they can know all scores in the population, identify the group with the population, and therefore call the SS divided by N the population variance. If for some reason they do not have access to all scores in the population, they divide the SS by N -1 and call this the sample variance, presumably because it is an estimate of the population variance based on sample data. In this book, reflecting our preoccupation with empirical data analysis as opposed to formal mathematics, we side with the empiricists and call the SS divided by N the sample variance, and the SS divided by N -1 the population variance, or more correctly, the estimated population variance. But once you are aware of this distinction, and understand why some writers define matters one way whereas other writers reverse the definition, you should be able to understand different writers and use different computer routines correctly.
5.3 THE STANDARD DEVIATION 5.3
63
THE STANDARD DEVIATION A second useful measure of variability used in statistics is the standard deviation. To compute the standard deviation for a sample of scores, first compute (a) the variance and then compute (b) its square root. Symbolically, the sample standard deviation for the Y scores is:
The variance is an average sum of squares, so the units for variance are the units for the initial scores squared. For example, if the initial scores were feet, then variance would be measured in square feet and analogous to a measurement of area. Taking the square root of the variance, which is how the standard deviation is computed, means the units for the standard deviation are again the same as those used initially. Thus the standard deviation is like an average error, expressed in the same units as those used for the raw scores. The larger the deviations from the mean are (i.e., the more scores are spread out instead of clustering near the mean), the larger is the standard deviation. You may wonder why the square root of the variance has come to be called the standard deviation; why it is used to represent an average error? Certainly the average of the absolute values of the deviation scores (column D in the Fig. 5.3 spreadsheet) would be a logical candidate for a measure of the typical deviation or error. The reason the standard deviation has become standard has to do with its technical statistical properties. These properties are not shared by the mean absolute deviation. For now, and until you read further, accept this on faith but know that there are reasons for this that experts find acceptable.
Sample and Population Standard Deviation Equation 5.6 defines the sample standard deviation. The formula for the population standard deviation, estimated from sample data, is:
Again, as with the sample and population variance, you should be aware that some writers (and spreadsheets) reverse the definitions used here. Exercise 5.4 The Sample Standard Deviation The purpose of this exercise is to modify the current template so that, in addition to the mean and the sample variance, it computes the sample standard deviation as well.
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS
64
General Instructions 1. Add a formula for the sample standard deviation to the current template. Provide an appropriate label. 2. Again, selectively change the values for the raw scores and note how these changes affect the value of the standard deviation. What do you observe? 3. Finally, restore the raw scores to their initial values as given in Fig. 5.1. Detailed Instructions 1. Put the label "SD=" for standard deviation in cell D16. Then in cell E16, enter a formula for the square root of the variance (cell E15). This is the standard deviation. 2. Now change the raw data (cells B3-B12) and note the effect on the standard deviation (cell E16). First try the two alternative sets given in Exercise 5.2. Recall that one set was less spread out, and one more, than the initial data. Then try some unusually distributed data sets. For example, make one score 24, one 35, and all the rest 2s and 3s. Then make half the scores 3s and the other half 12s. What do you observe? 3. Finally, restore the original data as given in Fig. 5.1.
Note 5.2 M
The sample mean, which is the sum of the scores divided by N. The mean for the Y scores is often symbolized as Y with a bar above it (read Y-bar), the mean for the X scores as X with a bar above, and so forth. In this book, to avoid symbols difficult to portray in spreadsheets, My represents the mean of the Y scores, Mx the mean of the X scores, and so forth.
VAR
The sample variance, which is the sum of squares divided by N. Often it is symbolized S2 or s2, which makes sense because variance is a squared measure. If not clear from context, VARy indicates the variance for the Y scores, VARx the variance for the X scores, and so forth.
VAR'
The population variance as estimated from sample data, which is the sum of squares divided by N -1. Often it is symbolized with a circumflex (a ^ or hat) above s2.
SD
The sample standard deviation, which is the square root of the sample variance. Often it is symbolized S or s. If not clear from context, SDy indicates the standard deviation for the Y scores, SDx the standard deviation for the X scores, and so forth.
SD'
The population standard deviation as estimated from sample data, which is the square root of the estimated population variance. Often it is symbolized with a circumflex (a ^ or hat) above s.
' (prime)
In this book, a prime or apostrophe after a symbol indicates an estimate, for example, VAR' and SD'. Some texts use the circumflex (the ^ or hat) for this purpose.
5.3 THE STANDARD DEVIATION
65
After you have restored the raw data to the initial values, your spreadsheet should look like the one given in Fig. 5.4. There are two additional points concerning this spreadsheet. First, because Y, the predicted value, is the mean of the raw scores, the mean of the predicted scores will of course be the same as the raw score mean. Second, given that the predicted score is the mean, the sum of the deviation scores must be zero. This follows from the way the mean is defined. However, due to rounding errors, entries that should sum exactly to zero, like the sum of the deviation scores, may sum to an extremely small number instead, like 4E-16, which in scientific notation means 4 divided by a 1 followed by 16 zeros. Such values may appear if a general (G) format is used for a cell. If you specify a format with, for example, two decimal places, then the value ".00" will be displayed instead. Exercise 5.5 SPSS Descriptive Statistics The purpose of this exercise is to familiarize you with the Descriptives command in SPSS. 1. Invoke SPSS. Create two variables, s and Y. Give the Y variable the label "Lies". Enter the data from the lie detection study. You could do this by hand, or you could cut and paste from your spreadsheet as displayed in Fig. 5.4. 2. Select Analyze-> Descriptive Statistics-> Descriptives from the main menu. In the Descriptives window, move the Lies variable to the right-hand window. Click on the Options button, check the box next to Variance. Click Continue and then OK.
A
B
1
D
C
E
y
Lies
=
SSy
2
s
Y
Y'
Y-Y'
y*y
3 4
1
5.3
-2.3
5.29
2
3 2
5.3
-3.3
3. 4 5 6
4 6 6 4
5.3 5.3 5.3 5.3
-1.3 0.7 0.7 -1.3
10.89 1.69 0.49 0.49 1.69
5 6 7 8 9 10 11 12
7 8 9 10
5 7 7 9
5.3 5.3
-0.3 1.7
5.3 5.3
1.7
2.89 2.89
3.7
13.69
13 14 15 16 17
Sum= N= Mean=
53 10 5.3 5.3
53 10 5.3
0
40.1
a=
VAR= SD=
0.09
10 4.01 2.0025
FIG. 5.4. Spreadsheet for the lie detection study after variance and standard deviation calculations have been added.
66
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS 3. Examine the output. Do the values you obtained for N, the mean, variance, and standard deviation agree with the results from the spreadsheet you created in Exercise 5.4? 4. Save the SPSS data file.
5.4
STANDARD SCORES Adolphe Quetelet, a French 19th-century statistician, is noted for his work with the way various measurements are distributed. Among the scores he worked with were the chest circumferences of Scottish soldiers (see Stigler, 1986). Assume the mean is 40 inches and one particular soldier has a chest circumference of 45. Thus, this soldier's chest measured 5 inches more than the mean. Now from our lie detection study recall that the mean number of lies detected was five and for the loth subject nine lies were detected, which means that four more lies than average were detected for that subject. It makes little sense to ask who is more extreme, the subject with four detected lies above the mean or the soldier whose chest circumference is 5 inches more than the mean. The two scales, number of lies and inches, are as alike as goats and apples. What is needed is a common or standard scale of measurement. Then both scores could be rescaled and, once the transformed scores were expressed in the same units, these new scores could be compared. The common scale is provided by the standard deviations, and the transformed scores are called standard scores or Z scores. To compute a standard score, (a) subtract the mean from an individual score and (b) divide that difference by the standard deviation for the set of scores. Symbolically (letting My and SDy represent the mean and standard deviation for the Y scores), the Z score corresponding to the Y score for the ith subject is:
Equation 5.8 is expressed in terms of sample statistics. The corresponding definition expressed in terms of population parameters is:
Note that both the deviation (Yi - MY or Y -u) and the standard deviation (SDY or a) are measured in the units used for the initial scores. Thus when one is divided by the other, the units cancel, resulting in a unit-free score, neither lies nor inches. Actually, because deviation scores are divided by the appropriate standard deviation, we can regard the resulting standard scores as being expressed in standard deviation units. A standard score of 1.0, then, means the corresponding raw score is exactly one standard deviation above its mean, whereas a standard score of -1.0 indicates a raw score one standard deviation below the mean, a standard score of 2.5 implies a raw score two and a half standard deviations above the mean, and so forth.
5.4 STANDARD SCORES
67 Exercise 5.6 Standard Scores
This exercise modifies the current spreadsheet so that, in addition to the mean, variance, and standard deviation, it now computes standard scores as well. General Instructions 1. Add two columns to the current spreadsheet. In the first column enter formulas for Zscores and in the second column enter formulas for the square of the Zscores. 2. Enter formulas for the sum, count, and mean of the Zscores. Enter formulas for the sum, count, mean, and square root of the mean for the squared Z score. What is the mean Z score? Why must it be zero? What is the variance and standard deviation for the Zscores? Why must they be one? Detailed Instructions 1. Label column F (cell F2) with "Z" (for Zor standard score) and column G (cell G2) with "z*z." 2. Enter a formula in cells F3-F12 for dividing the residual (Y-Y' in column D) by the standard deviation (cell E16). This is the standard score. 3. Enter a formula in cells G3-G12 for multiplying the standard score (Z, the column F entry) by itself. (You may want to format columns F and G so that only two or three places after the decimal point are given.) 4. Copy the formulas from cells E13-E16 to F13-F16 and to G13-G16. 5. What is the mean Z score? Why must it be zero? What is the variance and standard deviation for the Z scores? Why must they be one? At this point, your spreadsheet should look the one given in Fig. 5.5. Note that the mean standard or Z score is zero. This follows from the fact that deviations from the mean must sum to zero. Note also that the standard deviation for the standard scores (the square root of the average squared Z score) is 1. This follows from the fact that differences were divided by the standard deviation; hence, a raw score that initially was a standard deviation above the mean would have been transformed into a standard score of 1. Exercise 5.7 More Standard Scores This exercise provides additional practice in computing standard scores. The current template is used, but with different data. This time chest circumference measurements are used. 1. Using the spreadsheet for the lie detection study as a base, create a new spreadsheet for the chest circumference of Scottish soldiers. All that needs to be changed is some labeling (e.g., "Chest" for chest circumference, instead of "Lies") and the raw data. Replace the number of lies with the values 34, 38, 39, 39, 40, 40, 41, 41, 43, and 45, respectively. These represent the chest measurements, in inches, for 10 Scottish soldiers. 2. Nothing else needs to be done. All formulas for computing residuals, means, variances, standard deviations, and standard scores are already in place'.
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS
68
Your new spreadsheet should look like the one shown in Fig. 5.6. You are now in a position to compare scores in the two distributions. Note that the standard deviation for lies was 2.28 (Fig. 5.5) and the standard deviation for chest measurements was 2.79 (rounded to two decimal places). This means that the chest measurements were somewhat more widely distributed about their mean than the number of lies were around their mean. Now examine the standard or Z scores. As you can see, the standard scores associated with 9 lies and 45 inches were 1.85 (Fig. 5.5) and 1.79 (Fig. 5.6), respectively, almost the same. This means that the two highest scores from these two distributions are almost equally extreme. Note also that many standard scores were between -1 and +1 and almost no scores were less than -2 or greater than +2. However, the soldier with the smallest chest circumference (34 inches) was more extreme in his distribution (Z score = -2.15) than was the subject who had only one lie detected (Z score = -1.65).
Exercise 5.8 Obtaining Standard Scores in SPSS Open the data file you created in exercise 5.5. Run Descriptive statistics for the Lies variable as you did in the previous exercise, but this time check the Save standardized values as variables box in the Descriptives window. After you run the Descriptives command, return to the Data editor. Notice that SPSS created a new variable called "zy." Why do these z-scores differ slightly from the scores you calculated with your spreadsheet?
A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
s 1; 2 3 4 5 6 7 8 9 10 Sum= N=. Mean= a-
B Lies Y 3 2 4 6 6 4 5 7 7 9 53 10 5.3 5.3
E
D
C
y= Y' 5.3 5.3 5.3 5.3 5.3 5.3 5.3 5.3 5.3 5.3 53 10 5.3
Y-Y' -2.3 -3.3 -1.3 0.7 0.7 -1.3 -0.3 1.7 1.7 3.7 0 10 VAR= SD=
F
G
SSy
y*y 5.29 10.89 1.69 0.49 0.49 1.69 0.09 2.89 2.89 13.69 40.1 10 4.01 2.0025
Z -1.15 -1.65 -0.65 0.35 0.35 -0.65 -0.15 0.85 0.85 1.85 0 10 0 0
z*z 1.3192 2.7157 0.4214 0.1222 0.1222 0.4214 0.0224 0.7207 0.7207 3.4140 10 10 1 1
FIG. 5.5. Spreadsheet for the lie detection study including standard or Z-score calculations.
69
5.4 STANDARD SCORES
Identifying Outliers Transforming or rescaling raw scores into standard scores is useful, not only because it allows us to compare scores in different distributions, but also because it can reveal comforting or disturbing information about a particular sample of scores. Sometimes a data recording instrument malfunctions or sometimes a person makes a copying or data entry mistake. If such mistakes result in a data point quite discrepant from other, legitimate data, then its standard score will be extreme. Opinions vary as to how extreme a score needs to be before it should be regarded as an outlier, a possibly illegitimate data point that should be modified or deleted. A common rule of thumb suggests that any data point whose standard score is less than -3 or greater than +3 should, at the very least, be subjected to scrutiny. Scores this extreme may be important and valid observations, but they also may indicate that something (a subject, a piece of recording apparatus, a procedure) may have malfunctioned, or the subject does not legitimately belong to the population from which the investigator intended to sample. Outliers can exercise undue weight on the statistics computed (recall Exercise 5.3) and can render suspect the assumptions required for parametric statistical tests. As a matter of course, then, it is good practice always to compute and examine standard scores and to consider whether any outliers represent influential and legitimate data or unwanted noise. The mean of standard scores will always be zero and their standard deviation and variance will always be 1 because of the way standard scores are defined. Note, however, that the shape of the distribution of the scores is unaffected by being standardized. If we begin with a sample of scores, and use the sample mean and standard deviation to compute standard scores, the shape of the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
A :
B Scots; Chests Y s 34 38 2 39 3 39 4 40 5 40 6 41 7 8 41! 43 9 10 45 Sum= 400 N= 10 Mean40 a= 5
E
D
C
SSy
Y-Y' Y-Y' 40 -6 -2 40 -1 40 -1 40 0 40 0 40 1 40 1 40 40 3 5 40 0 400 10 10
y*y
y
1:
•
F
=
40
VAR=
SD=:
36 4 1 1
o; 0;
11 1 9 25 78 10 7.80 2.79
Z -2.15 -0.72 -0.36 -0.36 0.00 0.00 0.36 0.36 1.07 1.79 0 10
G z*z 4.62 0.51 0.13 0.13 0.00 0.00 0.13 0.13 1.15 3.21 10 10 1 1
FIG. 5.6. Spreadsheet giving the chest circumferences in inches and their associated standard scores for 10 Scottish soldiers.
70
DESCRIBING A SAMPLE: DESCRIPTIVE STATISTICS distribution of the transformed scores will be identical to the shape of the distribution for the raw scores. If the raw scores are highly skewed, for example, the standardized scores will also be skewed. Many students come to a course in statistics believing that standardizing scores somehow makes them normally distributed. This is simply not so; their distribution is unaffected by standardizing. Exercise 5.9 Outliers For this exercise you again use the current template and modify the data. The purpose is to demonstrate the effect different distributions of raw scores, especially those that include outliers, can have on their means, standard deviations, and standardized scores. 1. Change the raw scores for chest measurements so that the scores are more spread out from the mean. Then change the raw scores to represent a skewed distribution, for example, one with many more low than high scores. In each case, how are the standard scores affected? 2. Change one or two of the raw scores for chest measurements to a very large number, like 88 (likely an outlier). Then change one or two of the raw scores to a very small number (an outlier in the other direction). In each case, how is the mean and standard deviation affected? How are the standard scores affected?
Descriptive Statistics Revisited This chapter has been concerned primarily with description, with statistics that tell us something about the scores in a sample (or in a population, if all scores in the population are available to us). Specifically we have learned that the mean number of lies detected for the 10 subjects in the lie detection study was 5.3, and if the mean were used as the best guess for the number of lies detected for each subject, a typical or average error (i.e., the standard deviation) would be 2.00. Likewise, we have learned that the average chest circumference for a sample of 10 Scotsmen was 40 inches, and if the mean were used as the best guess for each Scotsman's chest circumference, the standard deviation or error would be 2.79. We have also learned that the mean can be thought of as the typical score for a set of scores, whereas the variance and standard deviation indicate whether scores are widely dispersed or clustered more narrowly about that mean. The standard deviation, moreover, is used when computing standard scores, which provides a way of determining whether scores in different distributions are equally extreme. This may also aid in identifying measurement or data transcription errors. Descriptive statistics such as these can indicate to others exactly what we observed and measured; consequently, they are interesting and valuable in their own right. In addition, they can also be used as a basis for inference, as we begin to demonstrate in the next chapter.
Note 5.3
z
A standardized or z score. It is the difference between the raw score and the mean divided by the standard deviation.
6
Describing a Sample: Graphical Techniques
In this chapter you will: 1. Learn basic principles of good graphical design. 2. Learn how to create histograms, stem-and-leaf plots, and box plots. In the previous chapter you learned basic statistics designed to describe the characteristics of central tendency (e.g., the mean) and variability (e.g., the variance and standard deviation). These statistics help define two important aspects of a distribution. Indeed, they form the foundation for the analysis of variance. Before applying inferential statistics, however, it is always important to know the subtleties inherent in any data set. The mean and standard deviation do not provide a comprehensive view of the data. The application of good graphical techniques allows you to determine the shape of your data distribution, discover outliers, and uncover subtle patterns in your data that may not be readily detectable when viewing only numerical values. As Edward Tufte summarized in the epilogue of his classic text, The Visual Display of Quantitative Information (1983, p. 191), "What is to be sought in designs for the display of information is the clear portrayal of complexity... the task of the designer is to give visual access to the subtle and the difficult—that is, the revelation of the complex." At this point we should differentiate two purposes of graphing data. Most of you are likely familiar with the bar charts typically found in professional journals. The purpose of such figures is to present the reader with the results of an analysis and they are therefore designed to tell a specific story; perhaps to the exclusion of unimportant information. The graphical techniques in this chapter, on the other hand, are more in the spirit of exploratory data analysis described by John Tukey (1977)- These graphical techniques will allow you to get to know your data before you conduct confirmatory or inferential statistics to assure that you have selected the right models to test, and to help you determine if your data meet the assumptions (e.g., normality) for each of the inferential analyses you plan to conduct. These techniques can also be used at any time, before or after confirmatory analysis, to simply explore the data in search of new and interesting patterns that you may have not considered during study design. The techniques are also critical to data cleanup. Strange patterns in your data or outliers may 71
72
DESCRIBING A SAMPLE: GRAPHICAL TECHNIQUES become evident in your graphs. These may be due to true variability in the population, or they may be errors. Therefore, it is always important to check.
6.1
PRINCIPLES OF GOOD DESIGN Before proceeding immediately to the different types of graphs used in exploratory data analysis, it is helpful to present a few principles that will help you organize your data in the most efficient manner possible. Graphical techniques are useful for showing large amounts of data in one readily interpretable unit. What would take you many words and paragraphs to describe in written text can be presented in a single well-designed figure. Thus, it is important to produce graphs that present your data in a coherent format. At the same time, however, you should avoid covering up patterns in your data or, worse yet, distorting the data in ways that would lead to erroneous conclusions. If you follow a few simple principles, the graphs you produce should give you vital insight into your data that will help you make appropriate decisions concerning the more complex inferential analyses you will learn about shortly.
Have a Clear Goal Good expository writing always begins with a clear thesis and then uses the literary style most suitable for presenting that thesis. Similarly, when producing a graph, you should have a clear goal in mind, and then select the best technique for accomplishing that goal. For instance, a histogram may be the best format for getting an overall feel for the way univariate data are distributed, while a stemand leaf plot may be better when trying to uncover outliers or unusual clusters of values. When comparing two or more groups, box plots may be the best graphical form. In your initial attempts to explore a data set, you will need to use various different graphical techniques with different goals in mind. As your analyses progress, however, the goal of each figure should become clearer. Eventually, when presenting your data to others, you should ensure that the figures are clearly integrated with the text and written interpretations of your analyses. In later chapters we present graphs that are designed to explicate the results of your analyses.
Show the Data The preceding principle does not imply that a good figure can have only one point. In fact, as the "show the data" principle suggests, a single graph can, and should, present as many aspects of the data as can be viewed in a coherent fashion. Thus, a single graph could provide the viewer with information concerning central tendency, variability, the shape of the distribution, and even individual data points. By doing so, for instance, it is possible to provide information concerning group comparisons, but also how valid the comparisons are, and how to best interpret any differences and similarities between the groups. Consider the typical bar chart ubiquitous in many professional publications. If two groups are compared, then the figure conveys only two pieces of information—the mean of each group. In some cases you may also be provided with error bars that give you some idea of variability within the groups. If a graph is to be worth a thousand words, however, then it needs to contain more than four values (i.e., two means and two standard errors). The stem-and-leaf diagram (see the next section) is an excellent example of a graphical technique that
6.1 PRINCIPLES OF GOOD DESIGN
73
presents large amounts of data (in some cases all of the raw data itself) in a format that allows the quick apperception of summary characteristics.
Keep It Simple The explosion of relatively simple to use software packages such as Excel and Power Point in the 1990s gave great power to the user to conduct statistical analyses and prepare presentation quality graphics. Along with that power, however, comes responsibility. Just as you would not conduct a statistical analysis in SPSS or Excel simply because it was available to you as a menu item, you should not add graphical decoration to your figures just because the option is available. Charts in Excel frequently default to the use of garish color schemes, grid lines, and background color fills that, at best, direct attention away from the main point of the figure, and, at worst, obscure or distort the data. Some authors refer to this as "chartjunk" (Tufte, 1983; Wainer, 1984) Chartjunk should be kept to a minimum and information to a maximum. You can use some of the design options, such as holding one aspect of a figure to highlight an important point, but you should always ask yourself before selecting an option, such as threedimensional shadows for a bar chart, if it adds any informational value to the figure. 6.2
GRAPHICAL TECHNIQUES EXPLAINED The following section will present three graphical techniques, the stem-and-leaf plot, the histogram, and the box plot, that will help you get to know your data. Using a combination of these techniques will allow you to uncover characteristics of your data such as the shape of the distribution (i.e., skew), central tendency (the mean, median, and/or mode), and spread (i.e., variance). Some of the techniques are also useful for detecting gaps and clusters within the data and for finding possible outliers. It is also possible to adapt these techniques to aid in the comparison of distributions.
The Stem-and-Leaf Plot The stem-and-leaf plot, or stem plot for short, embodies the principle of "show the data." It is actually a hybrid of a text table and a true figure. The stem-andleaf plot organizes individual numerical data in ways that highlight summary features of the distribution. Consider the following set of numbers—40, 49, 47, 46, 64, 64, 46, 66, 48, 46, 46, 38, 64, 63, 45, 65, 34, 60, 57, 55, 5O, 41, 65, 47, 49, 63, 47, 46, 56, 40, 49, 60, 19, 47, 42, 46, 63, 61, 63, 37. Try to quickly identify the characteristics of this data set. What is the minimum value? What is the maximum value? Are there any clusters in the data? Are there any outliers? An unordered case listing of the- data is not very helpful. A first step to organizing the data is to sort it in numerical order—19, 34, 37, 38, 40, 40, 41, 42, 45, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 48, 49, 49, 49, 50, 55, 56, 57, 60, 60, 61, 63, 63, 63, 63, 64, 64, 64, 65, 65, 66. It is now readily apparent that the minimum value is 19 and the maximum is 66. Upon closer inspection there is a large number of values at 46. The careful reader may have also noticed that 19 may be an outlier. Thus, when data sets are relatively small (say, less than 50 or so cases), simply sorting your data in Excel or a statistical package and then scrolling through the values will give you a hint about some of the important features of your data. The shape of the distribution, however, is hard to extract from this sort of presentation.
74
DESCRIBING A SAMPLE: GRAPHICAL TECHNIQUES We could also group the data into a frequency table. To do so, we simply create class intervals and then count the number of values within each interval (see figure 6.1). When viewing the figure it is obvious that there also are two modes—one between 40 and 49, and the other between 60 and 69. The distribution also appears asymmetrical—there are fewer numbers in the lower ranges and more in the higher ranges (i.e., a negative skew). By creating a frequency table we have gained a better understanding of the shape of the data, but we have lost access to the original data. We don't know, for instance, if the minimum value is 10 or 19. We also don't know if the mode is due to the existence of 19 scores of 40, or if the 19 scores are distributed evenly throughout the 40-49 range. The stem-and-leaf plot provides the opportunity to organize the data into a frequency table, yet it maintains access to the original values. The plot is made up of stems, which represent the class intervals from a frequency table. Each leaf represents an individual value. To construct a stem plot you separate each number at a place value. The stem becomes the value to the left of the place value, and the leaf becomes the number to the right of the place value. In the previous example, because the values range between 10 and 100, it makes sense to create stems in the tens and let each leaf represent the ones place holder. Typically stems and leaves are represented by one-digit numbers, but depending on the range of your date, it is sometimes useful to have two-digit stems or leaves. To read the plot presented in Fig. 6.2, you combine the stem with the leaf to recreate each data point. The first value is 1 for the stem and 9 for the leaf; thus the 9 represents the value 19. Similarly, on the third line we combine a stem of 3 with the leaves 4, 7, and 8 to create 34, 37, and 38. The frequency column is optional, but provides a good check to make sure you have the correct N for your data set. The stem plot provides the same information as the frequency table, but retains the original data values. From this plot, we can still see that the data clusters between 40 and 50, but we also know that 46 is the true mode. An added advantage is that the stem plot is more graphical in nature. The shape of the distribution (remember the negative skew) is visually perceptible in this format. It should be noted that it is necessary to use a fixed font, such as Courier, rather than a proportional font, to create an accurate visual impression of horizontal bars extending from each leaf. The stem-and-leaf plot can be adapted in a number of ways. First, if your data are in a different range, say from 100 to 999, you could round or truncate the values so that the stem represents the hundreds place holder and each leaf would represent the tens. For instance, a value of 324 would have a stem of 3 and a leaf of 2. Of course, given this method, you could not differentiate between 324 and 327. If you desired to retain the full resolution of each number, you could use
FIG. 6.1. A frequency table.
5.2 GRAPHICAL TECHNIQUES EXPLAINED
75
FIG. 6.2. A stem-and-leaf plot.
a stem of 3 (hundred) and leaves of 24 and 27. If you have a large number of scores clustered on one stem, you could alter the number of stems by splitting each one in half. For instance, Fig. 6.2 could be recreated with each stem split into those values below five and those five and above. By doing so, the shape of the distribution may sometimes become more obvious (see Fig. 6.3)
The Histogram The stem plot is an excellent graphical technique for displaying all of the data in a fashion that allows the viewer to perceive the distributional characteristics with little effort. The histogram does not retain information about individual scores, but as it is more graphical in nature, it is particularly good for providing a feel for the distribution of data at a quick glance. Given the design principle of show the data, however, the stem plot is the preferred technique, but most statistical programs do not provide the flexibility in selection of interval sizes and the manipulation of the number of digits in the stem and/or leaves necessary to take full advantage of the features of the stem plot. It is for this reason that we also present the histogram. A histogram is a graphical presentation of a frequency table. Class intervals are plotted on the x axis and frequency is plotted on the y axis. Individual bars indicate the number of values contained within each class interval. A histogram looks very much like the stem plot turned on its side. Compare the stem plot in
FIG. 6.3. A stem-and-leaf plot with each stem split in two.
DESCRIBING A SAMPLE: GRAPHICAL TECHNIQUES
76
FIG. 6.4. An example of a histogram. Fig. 6.3 and the histogram of the same data presented in Fig. 6.4. The two modes are clearly visible, and so is the outlier. We can also see the negative skew in the shape of the distribution. What is missing is the ability to access individual data points. When creating a histogram it is important to be aware of the number and size of the bins (i.e., the class intervals). In Fig. 6.4 there are 10 bins, which seems a reasonable number given the range of the data. Fig. 6.5 shows the same data as Fig. 6.4, but with only five bins. The two modes are still apparent in this figure, but the outlier is no longer readily apparent and the gap between the two modes is not as dramatic. Some consider the optimum width of each bin or class interval to be the one that most closely resembles the probability distribution of the data. Scott (1979) suggested that this can be accomplished by calculating the width of each bin as
W=3.49 * SV* N-1/3
(6.1)
where W is the bin size, SD is the standard deviation, and N is the sample size. Given formula 6.1 and using the estimated standard deviation, the appropriate width of the bins for the data is 10.85. Given the range of the data this would produce a histogram very similar to the one in Fig. 6.5. Using Scott's formula will often result in noninteger bin widths, which might be confusing to some viewers. Therefore, histograms based on Formula 6.1 provide a good exploratory tool for
FIG. 6.5 A histogram with only five bins.
5.2 GRAPHICAL TECHNIQUES EXPLAINED
77
determining the overall shape of a distribution, but may not be as well suited for presenting your data to others. Moreover, such histograms may sometimes omit detailed information, such as a small number of outliers that do not have great influence on the overall distribution. It is important to play with the size of the bins to give yourself the opportunity to explore all aspects of a distribution. This can be easily accomplished in most statistical packages.
The Box-and-Whiskers Plot The box-and-whiskers plot, or box plot, contains less information than the histogram or stem plot, but provides a graphical representation of important descriptive statistics related to the distribution of a data set. The box plot is also a good graphical technique for comparing two or more distributions. It is usually comprised of a box, a line within the box, and whiskers, which are two lines emanating from the either side of the box. These features of the box plot describe five characteristics of a distribution—the 25th and 75th percentiles, maximum and minimum values (excluding outliers), and the 5Oth percentile (i.e., the median). When oriented vertically, the lower and upper edges of the box represent the 25th and 75th percentiles. These two values define the interquartile range (IQR), which is the 75th percentile minus the 25th percentile, or the middle 50% of scores. The whiskers extend to the maximum and minimum values, excluding outliers. A line within the box is also drawn to indicate the position of the median. The box plot uses the median instead of the mean as a measure of central tendency, and the IQR instead of the standard deviation as a measure of variability. This is because the median and IQR are more resistant to the effects of individual scores, especially extreme scores, than the mean and standard deviation. If we changed the 66 to 166 in the data set, the median and IQR would remain unchanged at 48.5 and 16.5, respectively. The mean and standard deviation, on the other hand, would change from 50.85 to 53.35 and from 10.64 to 21.00, respectively. Thus, when first exploring your data, the median and IRQ are more useful. Later, after you have cleaned your data and determined the appropriate inferential statistics, the mean and standard deviation become more important. Fig. 6.6 presents a box plot for the data set. It is readily apparent that 50% of values fall within the box between the mid 40s and the low 6os. The median is around 48, and the minimum and maximum values are in the mid 30s and 6os. Skew is also apparent in these data as the bottom whisker is longer than the top, and a larger number of scores within the box are found above the median. The dot at the bottom of the plot represents an outlier. In chapter 5 you learned that an outlier could be defined as any value that was greater than three standard deviations away from the mean. In fact, this outlier is 19 and has a z score of -2.99, so it falls just under the threshold of 3.00 absolute—although the definition of an outlier as ±3.00 standard deviations is just a rule of thumb and any value approaching 3 should probably be examined carefully. In the box plot, however, outliers are defined as values that lie 1 l /2 IQRs or more away from the median. SPSS also identifies extreme values as those that are 3 IQRs or more away from the median. If there were any extreme values in these data, SPSS would identify them with an asterisk. Not all statistical packages define outliers in the same manner, so it is important that you check. Occasionally, you may come across a box plot that uses the terms hinges and H spread. Hinges are approximations of the 25th and 75th percentiles designed to be easier to calculate. The H spread is the equivalent of the IQR and is defined as the upper hinge minus the lower hinge. Hinges were useful before the days of
78
DESCRIBING A SAMPLE: GRAPHICAL TECHNIQUES
FIG. 6.6. A box-plot representation of the data. statistical packages implemented on a computer. Today they are rarely seen, but if you do come across a box plot that uses hinges, they can be interpreted in the same way as plots based on percentiles and the IQR.
Comparing Distributions Graphically Thus far we have discussed the graphical representation of univariate data, or data based on a single sample. Often, however, it is useful to evaluate the distributions of two or more groups of data on the same figure. In such cases, the box plot allows for the quick comparison of distributions. The median appears prominently in a box plot and allows one to quickly perceive differences between groups with reference to central tendency. At the same time, however, information concerning variability and the shape of each group's distribution is maintained, making it a much better representation of group differences than a text table of means and standard deviations. The box plot is therefore an excellent graphical technique to use when your eventual goal is to apply inferential statistics that test group differences. In Fig. 6.7, Group 1 is the same sample we have been working with throughout the chapter. Compare Group 1 to Group 2. It is easy to see that the two groups have a similar median value, but there is more variability in Group 2. Group 2 also seems to be more symmetrical and has an outlier in the 120 range that should be checked. Group 3, on the other hand, has a median much higher than the other two groups. As with Group 1, the third group is negatively skewed—the bottom whisker is longer than the top whisker—but the skew is not as large as the first group. The previous example demonstrates the usefulness of side-by-side box plots for comparison of the distributional characteristics of different samples. The box plot for comparing groups can also be easily created in most statistical packages. Ease of computation is important, but you should also not limit yourself to only those graphical techniques that have already been implemented by the common statistical packages. For instance, you could use many small histograms in one figure to compare distributions. This is what Tufte (1983) referred to as the small multiple. It is also possible to adapt the stem plot to grouped data. Compare the side-by-side stem plot in Fig. 6.8 to the box plot in Fig. 6.7. From both plots we can immediately determine that the central tendency of the two distributions is about the same. The mode of both is in the 405. With small sample sizes it is also relatively easy to find the median of the two groups by counting leaves in order
5.2 GRAPHICAL TECHNIQUES EXPLAINED
79
FIG. 6.7. A box plot comparing the distribution of three groups. until you reach the midpoint of the sample. In this case, because there is an even number of scores, it is necessary to take the mean of the two scores closest to the center of the plot. From this figure, we can also see that Group 2's distribution is more symmetrical than Group 1's distribution, if you exclude the outliers. Finally, even though this figure may be a little busier than the box plot, it retains information concerning individual data points that allow you to extract characteristics of the samples, such as the outliers and the large number of 46s and 47s in Group 1. These variations of the graphical techniques can easily be created by hand, on a word processor (as with Fig. 6.8), or in common draw programs that allow you to manipulate the output from statistical packages such as SPSS. Although they may take a little more effort, such graphs give you insight into your data and uncover subtle patterns that may not be readily apparent using the graphical techniques already implemented in software packages. Remember the quote from Edward Tufte: "the task of the designer is to give visual access to the subtle and the difficult—that is, the revelation of the complex." Your ability to reveal the complex patterns in your data is limited only by your creativity and skill in
FIG. 6.8. The side-by-side stem plot.
DESCRIBING A SAMPLE: GRAPHICAL TECHNIQUES
80
applying good principles of exploratory data analysis and the armamentarium of graphical techniques that you have mastered. Exercise 6.1 Graphical Data Exploration This exercise provides practice in the creation and interpretation of stem plots, histograms, and box plots. You will use the data provided in the table here. These data come from a study of children with Williams syndrome, a syndrome that often results in mild mental retardation (Robinson, Mervis, & Robinson, 2003). Each child was given a vocabulary test (Voc) and a test of grammar (Gram). In this exercise you will explore the univariate distributions of the vocabulary and the grammar scores. You will also generate a box-plot to explore possible differences between the vocabulary and grammar scores. Sub 1 2 3 4 5 6 7 8 9 10
1. 2.
3.
4.
5.
6.
Voc 84 70 75 62 93 83 98 59 73 67
Gram 79 79 69 68 91 75 94 61 67 57
Sub 11 12 13 14 15 16 17 18 19 20
Voc 80 92 60 70 75 35 31 64 69 83
Gram 67 76 64 75 76 55 55 63 67 82
Sub 21 22 23 24 25 26 27 28 29 30
Voc 56 45 68 66 54 44 89 80 70 80
Gram 66 66 82 63 67 72 75 67
For the vocabulary data, sort them in numerical order. Without calculating any statistics, what are your first impressions of the distribution? What are the minimum and maximum values? Are there any possible outliers? Do the scores seem to cluster in any particular way? By hand, using graph paper, create a stem plot of the data. Were your first impressions concerning central tendency, variability, and outliers correct? What about the shape of the distribution? Is it skewed? If so, describe the skew. Now create a histogram of the vocabulary data using eight equally spaced bins. Compare and contrast the histogram to the stem plot. Which do you find more informative and why? Create another histogram by calculating the width according to Formula 5.1. What is the width? Compare this histogram to the one you created in part 3. How do your impressions of the data differ based on the histograms with differing bin widths? Redo parts 1-4 using the grammar data. You may do this by hand, or on a computer, if you are familiar with the graphing functions in Excel, SPSS (see the next exercise), or another statistical package. Create side-by-side box plots of the vocabulary and grammar data. How do the two sets of scores differ with respect to central tendency, variability, and the shapes of their distributions?
5.2 GRAPHICAL TECHNIQUES EXPLAINED
81
The preceding exercise gave you valuable experience in creating graphs by hand. In the next exercise, you will be introduced to some of the graphs available in the Explore procedure of SPSS. Statistical packages provide a quick and efficient means of summarizing graphical data, but it is often the case that graphing data by hand gives you a better "feel" for the data. Moreover, most statistical packages do not provide options for creating some of the graphs presented in this chapter (e.g., the side-by-side stem plot), nor do they provide you with the flexibility to change all of the parameters in any particular graph (e.g., the stem size of a stem plot). For these reasons you should not limit yourself to any one statistical package. Often, using a statistical package to create summary statistics that you then graph by hand, or using a draw program, is the most flexible option available for exploratory data analysis. Exercise 6.2 Graphing in SPSS In this exercise you will learn to use the Explore command to obtain descriptive statistics, histograms, stem-and-leaf plots, and box plots. 1. Open SPSS, create variables, and enter the data from Exercise 6.1. 2. Select Analyze->Descriptive Statistics->Explore from the main menu. 3. Move the vocabulary and grammar variables to the Dependent List box. Click on Plots and select the histogram option in the Explore: Plots dialog box. Click Continue. 4. Click on Options and select the exclude cases pairwise option and click continue. Note that some SPSS commands give you the option of excluding cases listwise or pairwise. Selecting listwise exclusion will cause SPSS to delete any case that is missing data on at least one variable. Pairwise exclusion, on the other hand, will exclude only those cases that have missing data on variables involved in the current analysis. 5 Click on OK to run the Explore command. 6. Examine the output. What are the Ns for the vocabulary and grammar variables? Why are they different? What would they be if you selected listwise exclusion? 7. Examine the Descriptives output. Are the mean, median, standard deviation, interquartile range, and skewness statistics what you would expect given your results from Exercise 6.1? 8. Examine the histogram for the vocabulary scores. Double click on the histogram to open the chart editor. In the chart editor you can change the bin widths. Select Chart->Axis from the main menu. In the intervals box, select custom and click on Define. You can now select either the number of intervals you would like to display, or the interval width. Change the number of bins and/or the bin width. Observe how doing so affects the shape of the histogram. Do this with the gram variable as well. 9. Examine the stem-and-leaf plots and the box plots. Do they look like the plots you generated by hand? 10. Change some of the values in the data and rerun the Explore command. See if you can predict how the plots will look based on the changed values. 11. Return the numbers you changed to their original values and save the data file.
This page intentionally left blank
7
Inferring From a Sample: The Normal and t Distributions
In this chapter you will: 1. Learn how the normal can be used as an approximation for the binomial distribution if N (the number of trials) is large. 2. Learn what the normal curve is, how it arose historically, and what kind of circumstances produce it. 3. Learn what the central limit theorem is and how the normal distribution can be used as an approximation for the distribution of sample means if N (the sample size) is large. 4. Be introduced to the t distribution and learn how it can be used for the distribution of sample means if N is small. 5. Be introduced to the standard error of the mean and learn how to perform a single-sample test, determining whether a sample was probably drawn from a population with specified parameters. 6. Learn how to determine 95% confidence intervals for the population mean. In chapter 3 you were asked to derive critical values for the binomial sampling distribution for larger and larger numbers of trials. This was not too difficult to do for small values like 10 or even 15, but you can see how tedious and errorprone this could become for larger numbers. This concerned at least some 18thcentury English and French mathematicians. Lacking today's high-speed electronic computers, they set out to reduce tedium in a different way, and so sought approximations for the binomial. The goal was to find a way for computing probability values that did not require deriving probabilities for all the separate outcomes of series of different numbers of binomial trials. The binomial is a discrete distribution. No matter the number of trials, the number of outcomes that contain, for example, no heads, 1 head, 2 heads, ..., N heads are always whole numbers. As a result, the distribution, portrayed correctly, will always be jagged and will not be described by a smooth, continuous line. Recall the binomial distributions graphed in chapter 3, but now imagine the distributions for ever larger values of N. As N becomes large, the discreteness matters less. The graphs for the distributions begin to look almost smooth, which 83
84
INFERRING FROM A SAMPLE: THE NORMAL AND t suggests that as N approaches infinity, the distribution might approach a smooth line, one that could be represented with a continuous function. And if a function could be found that gave frequencies for the various outcome classes, then the entire distribution could be generated relatively easily. Desired is a function of X, symbolized as Y=f(X), that would generate all possible values of Y. The function would be supplied with the binomial parameters and produce the appropriate binomial distribution, or at least a close approximation. It is doubtful that the function would be as simple as Y - 2 + .5X, which would generate a straight line, or Y= X2, a quadratic equation that would generate a parabola (or U-shaped curve). But whatever the correct function, once defined, it can be viewed as a generating device. Supplied with continuously changing values of X, it would produce (i.e., generate) the corresponding values of Y as defined by the particular function.
7.1
THE NORMAL APPROXIMATION FOR THE BINOMIAL The person usually credited with first describing a functional approximation for the binomial distribution is a French mathematician, De Moivre, although it is doubtful if he fully appreciated the implications of his discovery (Stigler, 1986). Writing in the 1730s, De Moivre defined a curve that has come to be called the normal. If the number of trials is large (over 50, for example, but an exact rule of thumb is discussed subsequently), then the discrete binomial can be approximated with the continuously varying curve specified by De Moivre. This has an immediate and practical implication. As you may have noted, Table A in the statistical tables appendix (Critical Values for the Binomial Distribution) does not give values for N greater than 50. The reason is, for values over 50 the normal approximation is usually quite sufficient. The normal distribution is well known and values for the standard normal distribution (this is defined in a few paragraphs) are widely available (see Table B, Areas Under the Normal Curve, in the statistical tables appendix). Thus it seems easier and more economical to use the normal distribution for any large-sample sign or binomial tests. The first step is to standardize the test statistic, which is the number of trials characterized by the first of the two possible outcomes (e.g., the number of coin tosses that came up heads) and is symbolized here as T. To standardize T, we need to know (or assume) values for the three parameters that define its binomial distribution: 1. N, is the total number of trials conducted. 2. P, the probability for the first outcome according to the null hypothesis under investigation (e.g., that the coin is fair, which means that P, the probability of a head, would be .5). 3. Q, the probability for the second outcome (e.g., the probability of a tail), which necessarily equals one minus P (i.e., Q = 1 - P). In standardizing T, we compare T against all possible outcomes represented by the binomial distribution whose parameters are the values given for N and P, hence we are dealing with a population. Recall from the chapter before last (Equation 5.9) that, for a population, the definition of a standard score (omitting subscripts, and understanding u and a to be the mean and standard deviation for the T scores) is
7.1 THE NORMAL APPROXIMATION FOR THE BINOMIAL
85
This is the standardized value for T, or the binomial test Z score. In order to compute Z, we need to know both u and a. For the binomial distribution, the mean is
This makes intuitive sense. If N were 6 and P were .5, then the mean or expected value would be 3, meaning three heads out of six tosses. Similarly, if N were 5 and P were .5, the mean value would be 2.5 or if N were 6 and P were .33, the mean value would be 2. The standard deviation for the binomial distribution is
This makes less intuitive sense. For now, accept that it is correct, but if you would like proof, check a mathematically oriented statistics text (e.g., Marascuilo & Serlin, 1988). In any case, we can now rewrite Equation 7.1, the binomial test Z score, as follows:
If N is large, statisticians have proved that the Z defined by Equation 7.4 will be distributed approximately normally. That is, if the population parameters are as claimed, then the sampling distribution for Z should be the normal curve first described by De Moivre. The parameter N is determined by our procedures, Q is determined by P, and P is determined by the null hypothesis we wish to test. Now, and this is the point of the foregoing discussion, we have a way of determining if the null hypothesis is tenable, given the value actually observed for T. For example, if N = 86 and P = .5, we would expect that half of the trials, or 43, would result in successes. If the actual number of successes were 53 then, applying Equation 6.1:
Five percent of the area under the normal curve is demarcated with a Z value of 1.96 (two-tailed), and 2.16 is bigger than that; thus we conclude that a value as extreme as 53 would occur by chance alone less than 5% of the time, if the present sample were drawn from a population whose value for P is .5. (The most common null hypothesis for binary trials assumes that P = .5 and for that reason our discussion of the sign or binomial test in chapter 3 was confined to such cases. Some null hypotheses may demand other values, however, and equation 7.4 can easily accommodate them.) As already noted, the normal approximation for Z requires a large value of N. But how large is large? It depends not just on N, but on P as well. The usual rule of thumb is provided by Siegel (1956). He suggested that if the value of NPQ is at least 9, then the approximation provided by the normal will fit the true binomial distribution adequately enough for statistical hypothesis testing. This means that
86
INFERRING FROM A SAMPLE: THE NORMAL AND t if P = .5, then N could be as low as 36 (36 x .5 x .5 = 9) but that if P = .8, then N should be at least 57 (57 x .8 x .2 = 9.12). This rule of thumb, however, only establishes minimum standards; the approximation becomes better of course for larger values of N. The preceding paragraphs have introduced some useful material. If a study consists of a series of trials (e.g., if several different individuals are assessed), and if there are only two possible outcomes for each trial (e.g., a patient gets better or fails to get better), then the tenability of a particular value for P can be evaluated. This value predicts how many subjects should improve according to the null hypothesis. The actual number of subjects who improved can then be compared against this predicted value. The discrepancy between observed and predicted is standardized (i.e., divided by its standard deviation) and the resulting Z score is compared against the normal distribution. The assumption that such Z scores are approximately normally distributed, however, is reasonable only if N is sufficiently large (specifically, the product NPQ should be greater than nine). If the computed Z score is extreme, that is, if it falls in the region of rejection as defined by the alpha level and type of test (one- versus two-tailed), then the null hypothesis is rejected. We conclude that the sample of study subjects was probably drawn from a population whose value for P is not the value we derived from the null hypothesis and then used to compute the Z score. As our alternative hypothesis suggests, probably the true population P is a different value. To return to an example used earlier, if 86 patients were tested, and if 53 got better, we would conclude that this excess of 10 over the expected 43 was not likely a chance happenstance. Similarly, if only 33 got better, 10 less than expected, we would again conclude that this was likely not just chance (assuming a two-tailed test). It seems reasonable to conclude, therefore, that the treatment made a difference. Exercise 7.1 The Binomial Test using the Normal Approximation This exercise provides practice in using the large sample normal approximation for the binomial test. 1. Draw a graph, using your spreadsheet program if you like (this would be what spreadsheet programs usually call an XY graph). Label the X axis with values of P from 0 to 1. Label the Y axis N. Using Siegel's rule of thumb, compute enough values of N for various values of P so you can draw the graph. There are an infinite number of values for P, you need only select enough to give you confidence that you have accurately portrayed the shape of the graph. In drawing lines between the points you plotted, what assumptions did you make? You probably drew a smooth graph; is this justified? Should it be jagged instead? Why? What is the shape of your graph? It should be symmetric; why must it be? 2. The critical values for the normal distribution are: Two-tailed, alpha = .05: Z= ±1.96 Two-tailed, alpha = .01: Z= ±2.58 Memorize these values; you will need them later as well as for this and the next few exercises. Now, if N = 40, P= .5, and T= 26, what is the value of Z? Would you reject the null hypothesis, alpha = .05, two-tailed? Why or why not? If T= 28? If T= 30? Again, why or why not?
7.1 THE NORMAL APPROXIMATION FOR THE BINOMIAL
87_
3. 4.
Now answer the same questions as in part 2, but for an alpha level of .01. If N = 40, P = .75, and T= 25, would you reject the null hypothesis, alpha = .05, two-tailed? Why or why not? If T = 35? If T = 23? If T = 37? 5. If P = .5, alpha = .05, two-tailed, and N = 80, what values of T would allow you to reject the null hypothesis? What values would not allow you to reject? 6. Now answer part 5, but for N = 100 and N = 200. 7. If P = .2, alpha = .05, two-tailed, and N = 80, what values of T would allow you to reject the null hypothesis? What values would not allow you to reject?
7.2
THE NORMAL DISTRIBUTION In the preceding section we referred to the normal distribution and showed one way it could be used, but we have yet to describe it in any detail. There is some historical justification. Only early in the 19th century did Pierre Simon Laplace and Carl Friedrich Gauss, in a series of seminal papers, integrate De Moivre's earlier work, Legendre's method of least squares, Laplace's central limit theorem (discussed in the next section), and the normal curve into a synthesis that provided the basis for modern statistics.
Historical Considerations Primarily problems of astronomy and geodesy (the geological science concerned with the size and shape of the earth), in addition to sheer intellectual pleasure, motivated these early statistical theorists. One practical problem was how to combine several observations of the same astronomical or geodetic phenomenon, and from consideration of this problem both the method of least squares (discussed in chap. 5) and the realization that errors were distributed in a particular way resulted. This distribution later became called the normal or Gaussian distribution. Intended at first as a description for errors of observation, only later in the 19th century did investigators realize how common, or normal, it was. Adolphe Quetelet, in particular, became entranced by the normal curve arid demonstrated repeatedly that it could be found throughout nature. His fame in statistics, Stigler (1986) wrote, "depends primarily on two hypnotic ideas: the concept of the average man, and the notion that all naturally occurring distributions of properly collected and sorted data follow a normal curve" (p. 201). As noted briefly in chapter 5, Quetelet analyzed chest measurements for Scottish soldiers. These were normally distributed, which Quetelet interpreted as evidence that nature was striving for an ideal type, the average man. The ideal is the mean; deviations from it can only be due to accidental causes, to "errors." Quetelet is hardly the first person in the history of science to become enamored of one simple idea to the detriment of others, and the notion that the mean has some mystical force or moral imperative can still be found. Fortunately, the normal curve can be understood without recourse to underlying mystical principles.
The Normal Function The normal curve is a particular curve. It is not any bell-shaped curve, as students sometimes think, but a curve defined by a particular function. The function is not especially simple. It involves pi (a), which equals approximately 3.1416, the base of the natural logarithm (e), which equals approximately 2.7183,
88
INFERRING FROM A SAMPLE: THE NORMAL AND t
and a negative exponent. Like the binomial, the normal curve is actually a family of curves. The parameters for the normal curve are mu (u) and sigma (a), the population mean and standard deviation, respectively, and each pair of parameters generates a particular instance of a normal curve. The function that defines the normal curve is:
(X can range from minus to plus infinity.) One instance of the normal curve is so useful and is used so often that it has its own name. When the mean is zero and the standard deviation is one, the resulting curve is called the standard normal. Substituting u = 1 and a = 0 in equation 7.5 simplifies it somewhat and yields the definition of the standard normal curve:
(Again, Z can range from minus to plus infinity.) Any normal distribution can be transformed into the standard normal just by rescaling the X axis, in effect forcing the mean to be zero and the standard deviation one. To standardize the values on the X axis, the difference between each value and the mean would be divided by the standard deviation. By definition, standardizing produces a distribution whose mean is zero and whose standard deviation is one. The curve defined by Equation 7.6 (i.e., the standard normal curve) is depicted in Fig. 7.1. The function for the normal distribution may appear complex or even intimidating. In practice, however, this formula is almost never used. Tables giving the area under the standard normal curve to the left and/or right of various values of Z are used instead. If you look at the standard normal table provided here (see Table B in the statistical tables appendix), you will find that the area to the left of Z = 1 is .1587 and to the left of Z = +1 is .8413. This means that the area to the right of Z = +1 is .1587, that the area either less than -1 or greater than +1 is .3174, and that the area between -1 and +1 is .6826. In other words, if scores are normally distributed, a little over two-thirds of them will be within a standard deviation of the mean, and just under one-third will be greater than a standard deviation from the mean.
u o
Note 7.1 As noted earlier, the population mean is usually represented with a lower case Greek mu and the sample mean with M or X-bar. The population standard deviation is usually represented with a lower case Greek sigma and the sample standard deviation with S or SD.
7.2 THE NORMAL DISTRIBUTION
89
FIG. 7.1. The standard normal distribution.
Underlying Circumstances The underlying circumstances that produce normally distributed numbers are not difficult to understand. Whenever the thing measured (e.g., the chest circumference of Scottish soldiers, the height of college freshmen, the length of bats' wings, the IQ of military recruits) is the result of multiple, independent, random causes, the scores will be normally distributed. That is, their distribution will fit the form of the function described in the preceding paragraph. This is not mysterious, but simply a straightforward mathematical consequence. These circumstances necessarily produce normally distributed scores. And because so many phenomena are indeed the result of multiple, independent, random genetic and environmental influences, normally distributed scores are frequently encountered. Occasionally students think that, for example, IQ scores are normally distributed because of the way IQ tests are constructed. This is not true. Although the mean and standard deviation can be manipulated, the shape of the distribution cannot be changed. As noted in chapter 5, standard scores are distributed the same way the scores were distributed before standardization. Standardization only changes the scale markers on the X axis, replacing the mean of the raw scores with 0 (100 for IQ scores) and labeling one standard deviation below and above the mean -1 and +1 respectively (85 and 115 for IQ scores). To reiterate, standardization does not change the shape of a distribution. Moreover, we should remember that not all scores are normally distributed. For example, the distribution for individual annual income is quite skewed: Many people make very little, and only a few people make a lot. However, when circumstances are right—when a phenomenon is the result of many independent, random influences—then scores will be normally distributed. Of course, this is a theoretical statement. The distribution of a small sample of scores might not
90
INFERRING FROM A SAMPLE: THE NORMAL AND t appear normal, just due to the luck of that particular draw or sampling error, but if enough scores were sampled, the empirical distribution would closely approximate the theoretically expected normal distribution.
Normal and Rectangular Distributions Compared Why is any of this useful? Playing a bit with popular imagination, imagine we have developed a test to detect aliens from outerspace, something like the tests used by British intelligence during World War II to trip up German spies who were attempting to pass themselves off as English. Assume we know the theoretical distribution of scores for terrestrials, that is, we know the shape of the distribution as well as its mean and standard deviation. Now, even if we do not know how aliens perform, confronted with a single score, we can at least determine how likely that score is if the person tested is a terrestrial. Assume, for example, that possible scores for the test are 80, 81, 82 ..., 120, and each of the 41 possible scores is equally likely. This represents not a normal but a rectangular distribution. In this case, the probability for a score as high as 120 would be 1/41 or .0244 (one-tailed) and the probability for scores as extreme as 80 or 120 would be 2/41 or .0488 (two-tailed). The mean for this distribution is 100 and its standard deviation is 11.8. Thus the standard score for a raw score of 120 is 1.69. If the scores for the test had been normally distributed instead, the probability of a standard or Z score as high as 1.69 would have been .0455 (onetailed) and the probability of a Z score as extreme as 1.69 would have been .0909 (two-tailed; see Table B in the statistical tables appendix). This example was introduced for three reasons. First, it demonstrates that a standard score by itself does not tell us how probable the corresponding raw score is: We need to know the shape of the distribution as well. In addition, it provides yet another demonstration of the logic of hypothesis testing, in which the probability for a particular value of a test statistic is determined with reference to the appropriate theoretical sampling distribution. Finally, it lays some groundwork for the next exercise and for discussion of the central limit theorem, which is presented in the next section. Exercise 7.2 The Normal Distribution Primarily, this exercise provides practice using the table of areas under the normal curve (Table B in the statistical tables appendix). 1. As in the example just presented, assume M = 100, S = 11.8, but that scores are normally distributed. What proportion of scores are greater than 105? Greater than 115? Less than 80? What proportion differ from the mean by more than 3? 2. For normally distributed scores, what proportion are greater than two standard deviations above the mean? Greater than three standard deviations? More extreme than two standard deviations from the mean? More extreme than three standard deviations? 3. Assuming normal distributions, what are the critical values of Z for alpha = .05 and for alpha = .01 for one-tailed tests? For two-tailed tests? 4. Why might it make sense to regard any standardized scores less than -3 or greater than 3 as outliers?
7.2 THE NORMAL DISTRIBUTION
91
5. For the rectangular distribution just presented, M = 100 and SD = 11.8. What proportion of the scores are greater than .678 of a standard deviation above the mean? Are within .678 of a standard deviation from the mean? 6. (Optional) For this rectangular distribution, whose scores ranged from 80 to 120, why does SD= 11.8?
7.3
THE CENTRAL LIMIT THEOREM What has come to be called the central limit theorem (central in this context means fundamental, not middle) was first described by Laplace in 1810 and represented a major advance over De Moivre's work a century earlier. Laplace proved the following theorem: Any sum or mean (not just the number of successes in N trials) will be approximately normally distributed if the number of terms is large. The larger N is, the better the approximation. In theory, as the limit approaches infinity, the approximation approaches perfection. But long before that, the approximation becomes good enough to be useful. The implications for statistical analysis are profound. If we draw a single score at random, unless we know exactly how those scores are distributed in the population, we have no basis for determining the probability of drawing that score. But if we draw a sample of scores and compute the mean for the sample, we now know how sample means (if not individual scores) should be distributed. According to the central limit theorem, no matter how the raw scores in a given population are distributed, sample means will be distributed normally. For example, no matter whether the distribution of raw scores is rectangular, or bimodal, or whatever, if we drew thousands of samples of size N, computed the mean for each, and graphed the distribution, the distribution of the sample means would approximate a normal distribution. Given what you learned about the normal distribution in the preceding section, this makes sense. Earlier we defined the mean as the sum of the raw scores divided by N (Equation 5.1). But this is simply an economical way of saying that the mean is a weighted summary score. Each raw score is multiplied (or weighted) by a weighting coefficient, which for the mean is 1 divided by N for all scores, and the resulting products are summed. Symbolically, the weighted summary score formulation for the mean is:
When drawing a sample of N scores, each X can be regarded as representing an independent, random influence. If this is so, then of course the summary score (or mean) for these multiple scores will be distributed normally. After all, the mean simply combines into one score each of the presumably independent, random separate scores.
INFERRING FROM A SAMPLE: THE NORMAL AND t
92
7.4
THE t DISTRIBUTION As noted earlier in this chapter, for only 4, 5, or even 10 trials, the normal is not a very good approximation for the binomial distribution. However, as the number of trials increases, the approximation becomes increasingly better. As a general rule of thumb, and assuming the value of P is not extremely small or extremely large, the normal approximation for the binomial is usually good enough to be practically useful if N is greater than 50, certainly if N is greater than 100. And in theory, as N approaches infinity, the binomial comes ever closer to being exactly like the normal. The same is true for the distribution of sample means. According to the central limit theorem, sample means are distributed approximately normally, and the approximation becomes better as the sample size becomes larger. The general rule of thumb is the same. The normal approximation for the distribution of sample means is good enough to be practically useful if the sample size (or N) is greater than 50. And as with the binomial, as N approaches infinity, the distribution of sample means comes ever closer to being exactly like the normal. For smaller numbers of trials, sampling distributions can be constructed for the binomial, which is exactly what you did for an exercise in chapter 3. The distribution of sample means for smaller sample sizes is likewise known. The distribution is called the t distribution and has been extremely important in the history of statistics. It can be applied to many problems and tests, and indeed many introductory texts in statistics spend considerable time developing the many applications of t tests. As noted at the end of chapter 4, we have elected to emphasize other, more general techniques that accomplish many of the same ends, but nonetheless students should be familiar with the t distribution. In any case, when dealing with small samples (i.e., samples for which N is less than 50, certainly less than 30), it is essential to use the t distribution for solving the two kinds of problems described in the next two sections: single-sample tests and 95% confidence intervals. The t distribution itself looks very much like the normal distribution, only flatter. Imagine the normal distribution is a tent, held up with a single wire. As the sample size decreases, the wire is lowered and the tent spreads out, assuming the shape of the t distribution for smaller and smaller N. The tails for the t distribution extend further than the tails for the normal distribution, so critical values for t (values that demarcate the most extreme 5% or 1% of the area under the curve) are larger than the corresponding values would be for the normal distribution; they become larger as N becomes smaller. For example, the alpha .05 nondirectional critical value for the normal distribution is 1.96. The corresponding values for a t distribution with a sample size of 30 is 2.05, for N = 20 is 2.09, and for N = 10 is 2.26. (See Table C, Critical Values for the t Distribution, in the statistical tables appendix. In these cases, degrees of freedom are N - 1 so for N = 30, df = 29; for N = 20, df = 19; and for N = 10, df = 9. What degrees of freedom are and how they are determined is explained in chapter 10.)
7.5 SINGLE-SAMPLE TESTS 7.5
93
SINGLE-SAMPLE TESTS An understanding of the central limit theorem and the normal and t distributions allows you to perform what are usually called single-sample tests, or tests that determine if a single sample was likely drawn from a particular population. Consider the lie detection study. The only data available to us are the number of lies detected for the 10 subjects in the lie detection study sample. From these sample data, we can compute the sample mean (M = 5.3) and standard deviation (SD = 2.00). Imagine our null hypothesis states that the sample is drawn from a population whose mean is 7, so H0: u = 7). Is the sample mean of 5.3 sufficiently different from the assumed population mean of 7 to render the null hypothesis that the sample was drawn from this particular population untenable? The statistic we have in hand is a single sample mean, M. What we want to know is whether or not M is especially deviant, compared to the distribution of sample means. In other words, if we drew thousands of samples of 10 scores each from a population whose mean score is 7, computed the mean for each sample, and examined the distribution of these sample means, would a value of 5.3 be near the center of this distribution of values for sample means or far out in one of the tails? We do know from the central limit theorem that, no matter how the raw scores were distributed in the population, the distribution of sample means will be approximately normal, or in this case distributed as t with N - 1 or 9 degrees of freedom. Hence, if we standardize the sample mean, we can evaluate the resulting standard score using the t distribution, which will tell us how likely a mean score of 5.3 is, if the population raw score mean is 7. The sample mean is standardized as follows:
It is important to remember that we are dealing with means, not with raw scores. Thus ZM indicates a standard score for means, not raw scores. And in order to compute it we need to know the mean for the distribution of sample means (symbolized UM), which conceptually is not the same as the mean for the scores in the population (symbolized u), and we need to know the standard deviation for the distribution of sample means (symbolized aM), which is not the same as the standard deviation for the scores in the population (which is symbolized a). If we know the mean and standard deviation for scores in the population, then the mean and standard deviation for the distribution of sample means can be determined. It seems reasonable to assume (and statisticians can prove) that the mean for the distribution of sample means is the same as the mean for the scores in the population. In other words,
In the present case, because u is assumed to be 7 (by our null hypothesis), we can assume that uM is also 7. In contrast, we would expect the standard deviation for sample means to be less than the standard deviation for the population scores. If we draw a sample of 10 scores, for example, some will likely be less than the mean, and some will be larger, in effect canceling each other out. As a result, the values for sample means will tend to cluster more tightly around the mean than the raw scores, often by quite a bit.
94
INFERRING FROM A SAMPLE: THE NORMAL AND t
The Standard Error of the Mean What is needed is a way of estimating the standard deviation for sample means (a M ). We know that the standard error of the mean depends on the sample size. If we draw hundreds of samples of size 20, for example, their means will cluster more tightly around the population mean than if we had drawn samples of size 10. We also know that the standard error of the mean will be less than the population standard deviation (a). Statisticians have proven to their satisfaction that an accurate estimate of the standard error of the mean is the population standard deviation divided by the square root of the sample size:
However, how do we determine the value of a in the first place? For single-sample tests, the value of u is assumed (determined by the null hypothesis) but the value of a is typically estimated from the sample. We already know how to compute the sample standard deviation (see Equation 5.6). It is the square root of the quotient of the sum of the deviation scores squared (symbolized SS for sum of squares) divided by N (the sample size):
The population standard deviation, as estimated from the sample, however, is not quite the same (see Equation 5.7). It is the square root of the sum of squares divided by N - 1:
Because N - 1 is smaller than N, the estimated value for the population standard deviation will be somewhat larger than the value computed for the sample standard deviation. This is explained in chapter 10. For now, note that although the sample standard deviation or SD is 2.00 (the square root of 40.1 divided by 10) for the 10 data points from the money-cure study, the estimated population standard deviation or SD' is 2.11 (the square root of 40.1 divided by 9). Now we can compute the standard deviation for the distribution of sample means. From Equation 7.10, the standard error of the mean (a M ) equals a (estimated as 2.11) divided by the square root of N. Thus, letting SD'M represent the estimated value for the standard error of the mean,
For the data from the money-cure study,
7.5 SINGLE-SAMPLE TESTS (computed retaining unshown significant digits for SD'). Equations 7.10 and 7.12 and simplifying algebraically:
95 Or, combining
Again, the result for our current example is .667 (the square root of the quotient of 40.1 divided by 90). This formula (Equation 7.14) for the standard error of the mean is convenient when the sum of the squared deviation scores is readily available. Finally we can answer the question posed at the beginning of this section: Is the sample whose mean is 5.3 likely drawn from a population whose mean is 7? The standard score representing the deviation of the sample mean from the assumed population mean (see Equation 7.8) is 5.3 minus 7 divided by 0.667, the standard error of the mean (recall that SD'M is the estimated value for a M ). Hence:
The sample size is small (N = 10); thus ZM will be distributed as t in the preceding equation, so we refer to Table C, not Table B, in the statistical tables appendix. According to this table, the critical value for t (alpha = .05, two-tailed) with nine degrees of freedom is 2.26. Therefore a sample mean as small as 5.3, whose standardized value is -2.55 (and is distributed as t with nine degrees of freedom), would occur less than 5% of the time if the population mean were in fact seven. We conclude that the sample is likely not drawn from the null hypothesis assumed population. As a second example of a single-sample test, let us return to our earlier problem of alien identification, assuming this time that instead of sampling just one individual, we sample groups of eight instead. In this case, we know that the population from which we are sampling is homogeneous, either all aliens or all terrestrials. As before, we know that the population mean and standard deviation for terrestrials are 100 and 11.8, respectively (in this case, the population standard deviation is given, not estimated from a sample). If the mean for the sample is 94, should we reject the null hypothesis (alpha - .05, two-tailed) that the sample is drawn from a population of terrestrials? The mean for the distribution of sample means is
and the standard deviation for the distribution of sample means is
Thus the standardized value for the deviation of the sample mean from the population mean is:
INFERRING FROM A SAMPLE: THE NORMAL AND t
96
Again, because the sample size is small, ZM is distributed as t (and we could have replaced ZM with t in the equation). According to Table C in the statistical tables appendix, the critical value for t (alpha = .05, two-tailed) with N - 1 or seven degrees of freedom is 2.37. Thus a value of — 1.44 is not sufficient to reject the null hypothesis. Probably the sample consists of terrestrials.
M (u
Note 7.2 The sample mean. Computed by summing the scores in the sample and dividing by N. The population mean. It can be estimated by the sample mean, or its value can be assumed based on theoretical considerations.
UM
The mean of the distribution of sample means. It can be estimated by the population mean.
SD
The sample standard deviation. Computed by taking the square root of the sum of the squared deviation scores divided by N.
a
The population standard deviation. It can be estimated by taking the square root of the quotient of the sum of the squared deviation scores divided by N -1. This estimate is symbolized SD'.
aM
The standard error of the mean, or the standard deviation for the distribution of sample means. It can be estimated by dividing a by the square root of N, or by dividing SD' by the square root of N, or by taking the square root of the quotient of the SS divided by N times N - 1. This estimate is symbolized SD'M . A test statistic similar to Z but for small samples. The shape of the t distribution is similar to the normal distribution but flatter. As a result, critical values for t are larger than corresponding critical values for Z.
t
Exercise 7.3 Single-sample tests This exercise provides practice in single-sample tests and in computing the standard error of the mean. 1.
If the alien identification sample size were 100 instead of 8, what would be the value for the standard error of the mean? What is the critical value for t (alpha = .01, two-tailed)? Would you conclude that the sample consisted of aliens or terrestrials? 2. Assume the sample size for the money-cure study is sufficiently large that you decide to use the standard normal (Table B) instead of the t table (Table C). Assume further that the standard error of the mean retains the value
7.5 SINGLE-SAMPLE TESTS
3. 4. 5.
6.
7.
8. 9.
97
.760, so that ZM retains the value -2.63 computed earlier. What percentage of the time would a sample mean as low as 5 occur if the population mean really were 7? A sample consists of six scores: 32, 25, 45, 38, 52, and 29. How likely is it that this sample was drawn from a population whose mean is 26? Assume a two-tailed test, alpha = .05. What would your decision be if alpha were .01? A second sample likewise consists of six scores: 32, 25, 97, 38, 52, and 29. How likely is it that this sample was drawn from a population whose mean is 26 (alpha = .05, two-tailed)? As in the last example given in the text, assume the population mean is 100 and the population standard deviation is 11.8. If the sample size were 16 (instead of 8), what values for the sample mean would allow you to reject the null hypothesis (alpha = .05, two-tailed)? If the sample size were 32? 64? As sample size gets larger, what happens to the value of the standard error of the mean? The standard error of the mean is often used in conjunction with bar graphs to indicate variability. Imagine you have selected two samples, one of five boys and one of seven girls, and have determined their weights. The mean weight is 72 pounds for the boys and 64 pounds for the girls. The population standard deviations, estimated from the sample data, are 9 for the boys and 8 for the girls. Compute standard errors of the mean for both the boys and the girls. Then create a bar graph. One bar will represent the boys, one the girls; the Y axis will indicate pounds. From the middle of the top of each bar draw two lines, one extending up and one extending down, each marking off exactly one standard error of the mean. Draw small horizontal lines to demarcate the top and bottom of these error bars. Adding error bars to a conventional bar graph is desirable because of the graphic way they indicate variability—and remind the viewer that the top of the bar is just the mean obtained from one sample, but other samples might locate the mean differently. (See Fig. 7.2 for an example.) (Optional) Using the functional definition for the standard normal distribution (Equation 6.6), set up a spreadsheet to compute Y values for values of X that range from -4.0 to +4.0 in steps of 0.2. Now graph this normal distribution (using your spreadsheet's graphing capability, if it has one). Your graph should look like Fig. 7.1. Note that probability is associated with the area under the curve; the exact height of the curve at various points is not especially meaningful in this context. (Optional) Starting with Equation 6.10, provide an algebraic derivation for Equation 6.14. (Optional) A third definition for the estimated standard error of the mean or SD'M (in addition to Equations 6.13 and 6.14) is the sample standard deviation divided by the square root of N - 1. Again starting with Equation 6.10, provide an algebraic derivation for this formula.
At this point, a confession is in order. In actual practice, single-sample tests are rarely used. They were introduced here, as they typically are in statistical texts, because of the particularly clear way they demonstrate principles of statistical inference and the use of the normal and t distributions. The standard error of the mean, however, is often used in conjunction with bar graphs to indicate variability, as is demonstrated in the next exercise.
98
INFERRING FROM A SAMPLE: THE NORMAL AND t
FIG. 7.2. A bar graph of the group means described in Exercise 7.3 showing error bars. The distance the error bars extend above and below a particular group mean represents an estimate of the standard error of the mean computed from data for that group.
7.6
NINETY-FIVE PERCENT CONFIDENCE INTERVALS Usually error bars indicate one standard error of the mean above and below the sample mean, but other intervals could be used. A particularly useful one is the 95% confidence interval. The interval is based on data from a single sample, but if we drew sample after sample repeatedly, 95% of the time the true population mean should fall within the indicated confidence intervals. Confidence intervals, whether or not displayed graphically, are valuable for the same reason that error bars are valuable: They serve to remind us that information gleaned from sample data provides probabilistic and not absolute information about the population. We may never know the value of the population mean with absolute certainty, but at least we can say that there is a 95% chance that the true population mean falls within the interval indicated. You know from the central limit theorem that sample means are distributed approximately normally and means for small samples follow the t distribution. You also know that the critical values for t, alpha = .05, two-tailed, given in Table C in the statistical tables appendix, demarcate the 5% of the area under the curve in the tails from the 95% of the area in the center under the hump, and that for large samples, this critical value is 1.96 (from Table B, the standard normal table). The t values (or Z values, for large samples) are standardized, which is why you divided by the standard error of the mean when computing them. In order to compute 95% confidence intervals, you need to reverse the process, multiplying (instead of dividing) by the standard error of the mean. Remember that t or Z scores are in standard units. By definition, the number of raw score units that correspond to one standard unit is the value of the standard error of the mean. Thus the upper 95% confidence interval, expressed in whatever units were used for the raw scores in the first place, is the mean plus the standard error of the mean multiplied by 1.96 for large samples, or multiplied by the appropriate
7.6 NINETY-FIVE PERCENT CONFIDENCE INTERVALS
99
value from Table C for small samples; the lower 95% confidence interval is the mean minus the standard error of the mean times the appropriate value. Symbolically (where CI stands for confidence interval),
Thus 95% error bar = SD'M x t(df)05,two-tailed Lower 95% confidence limit -M-
SD'M x t(df)05,two-tailed
Upper 95% confidence limit = M + SD'M x t(df)05,two-tailed As an example, consider the samples of boys and girls shown in Fig. 7.2. The standard error of the mean, estimated from sample data, was 4.02 for the boys and 3.02 for the girls, whereas the mean weights were 72 pounds for the five boys, 64 for the seven girls. The confidence intervals for the boys are: 95% error bar = SD'M x t(4) = 4.02 x 2.78 = 11.2 Lower 95% confidence limit = M - SD'M x t(4) = 72 - 11.2 = 60.8 Upper 95% confidence limit = M + SD'M x t(4) = 72 + 11.2 = 83.2 For the girls the confidence intervals are: 95% error bar = SD'M x t(6) = 3.02 x 2.45 = 7.4 Lower 95% confidence limit = M - SD'M x t(6) = 64 - 7.4 = 56.6 Upper 95% confidence limit = M + SD 'M x t(6) = 64 + 7.4 = 71.4 Often researchers want to know whether the means for two groups are significantly different. For example, is the average weight for the girls different from the average weight for the boys? Techniques designed to answer such questions are introduced in chapter 10. For now, it is worth noting that because the 95% confidence interval for the boys includes within its range the mean value for the girls, and vice versa, it is unlikely that the difference between their mean weights is statistically significant (at the .05 level). Exercise 7.4 Ninety-five Percent Confidence Intervals This exercise provides practice in computing 95% confidence intervals. Fig, 7.3 illustrates the results. 1. Compute the 95% confidence interval for the population mean using the sample of six scores from part 3 in the last exercise. 2. Compute the 95% confidence interval for the population mean using the sample of six scores from part 4 in the last exercise. 3. Do the results of the single-sample tests performed for the last exercise agree with the 95% confidence intervals computed for this exercise?
100
INFERRING FROM A SAMPLE: THE NORMAL AND t
FIG. 7.3. A bar graph of the group means described in Exercise 7.4, showing 95% confidence intervals.
Exercise 7.5 SPSS: Single-sample tests and confidence intervals This exercise provides practice in using SPSS to conduct single-sample tests. You will use the data from Exercise 7.3, parts 3 and 4, to determine whether the sample is drawn from a population with a mean of 20. 1. Open SPSS and create a variable called scores. Enter the data from Exercise 7.3, part 3. 2. Select Analyze->Compare Means->One-Sample T Test from the main menu. Move the scores variable to the Test Variable(s): window. Enter 20 in the Test Value window. Click on OK. 3. Examine the output. Are the values for the standard deviation and the standard error the same as the values you calculated in Exercise 7.3? What is the significance level of the t test? Do you conclude that the sample was drawn from a population with a mean of 20 at the alpha = .05 level? What would your decision be if alpha were .01? 4. Conduct a single-sample test using the data from Exercise 7.3, part 4. Why is this test not significant at the .05 level even though the mean is higher than the data from question number three? 5. To compute confidence intervals for the scores variable, select Analyze->Descriptive Statistics->Explore from the main menu. Move the score variable to the Dependent List window and click OK. The 95% confidence interval for the mean will appear in the Descriptives box. If you would like to calculate confidence intervals based on a value other than 95%, click the Statistics button in the Explore dialog box and change the 95 to another value in the Confidence Interval for Mean window.
7.6 NINETY-FIVE PERCENT CONFIDENCE INTERVALS
101
Some of the material presented in this chapter is more important conceptually than practically. Practically, knowing how to present figures with error bars representing the standard error of the mean, or 95% confidence intervals, allows you to present your data visually. It is often useful to know how to approximate a binomial with a normal distribution. It is also important, as a matter of basic statistical literacy, to have some familiarity with the t distribution. Single-sample tests, on the other hand, are probably more important conceptually than practically. Still, their principles nonetheless provide a particularly clear foundation for the material to follow. This chapter, in fact, represents something of a turning point. Having finished with basic topics like hypothesis testing and simple descriptive statistics, we now turn to the topic that will occupy us for the remainder of the book: accounting for and analyzing variance.
This page intentionally left blank
8
Accounting for Variance: A Single Predictor
In this chapter you will: 1. Learn how to determine empirically the best-fit line (the line that, when used as a basis for predicting one variable from another, minimizes the error sum of squares). 2. Learn what the phrase accounting for variance means. 3. Learn to distinguish measures that indicate the exact nature of the relation between two variables from measures that indicate the strength of their relation. In this chapter we begin laying the groundwork required for most of the analytic techniques described in the remainder of this book. Many attributes of interest to behavioral scientists are measured with interval, ratio, or at least ordinal scales. In such cases, almost all specific research questions can be reduced to a single general form. The general question is, if the values of some quantifiable attribute (a dependent or criterion variable measured on a quantitative scale) vary from subject to subject, can that variability be explained, or accounted for, by research factors identified by the investigator (independent variables measured on either qualitative or quantitative scales)? In other words, does the independent variable have an effect on the dependent variable? Is the number of lies the expert detects affected by the subject's mood, for example, or by whether or not the subject received a drug? The techniques described in this chapter explain how such questions can be answered. 8.1
SIMPLE REGRESSION AND CORRELATION Historically, both simple regression and correlation (one criterion and one predictor variable) and multiple regression and correlation (one criterion and multiple predictor variables; often abbreviated as MRC) have been taught as techniques for nonexperimental data exclusively. Indeed, the term correlational has come to signify a nonexperimental or observational study. The only reason for the segregation of MRC into the nonexperimental world, and the dominance of analysis of variance or ANOVA in the experimental world, however, is 103
104
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR
historical precedent and tradition. There is no compelling statistical basis for this bifurcation (Cohen, 1968). In fact, there is much to be gained by a more unitary approach. From the point of view of the general linear model, analysis of variance is simply a subset of multiple regression and correlation. Giving prominent play to this fact, as this book does, simplifies learning considerably. As noted in chapter 1, there are fewer topics to master initially and those topics possess considerable generality and power. The unified approach lets you see the forest as a whole without being tripped up by individual trees. Two numeric examples are presented in this chapter and the next. In both cases, the quantitative dependent variable accounted for is number of lies detected. In one case, the independent variable is quantitative (mood, used in this chapter), and in the other, qualitative (drug vs. no drug, used in chap. 9). These two examples are used first to demonstrate the basic concepts required to analyze such data, and second to demonstrate that no matter whether the predictor variable is quantitative (mood score) or qualitative (drug group), its effect can be analyzed using multiple-regression procedures. But remember, whether qualitative or quantitative, throughout this chapter we assume just a single predictor. Methods for dealing with two or more predictors are deferred until chapter 11.
Finding the Best-Fit Line In chapter 5, you learned how a set of scores (like the number of lies detected for the 10 subjects) could be described. In particular, you learned how, using the method of least squares, a single point (the mean) could be fit to the set, and how the variability of the scores around the mean could be described (using the variance and the standard deviation). Research questions rarely involve just one variable, however. Typically we want to know, at the very least, how much one variable (the independent or predictor variable) affects another (the dependent or criterion variable). For example, imagine that before determining the number of lies detected we had asked our subjects to fill out a mood scale (higher scores indicate a better mood). We do this because we want to find out whether subjects who are in a good mood (and who perhaps are less guarded) are more likely to have their lies detected. The question now is, what relation, if any, is there between mood and number of lies detected? In other words, if we know a subject's mood, can we predict the number of lies detected more accurately than if we did not know that individual's mood score, if we only knew the average number of lies detected irrespective of mood? In order to answer this question, we would first postulate a prediction equation that ignores mood, and a second one that takes mood into account, and then compare the two equations. If mood affects the number of lies detected, then the second equation should provide a better fit to the data (i.e., should make more accurate predictions) than the first. However, if mood has no effect, then there should be little difference between the predictions made by the two equations. In chapter 5 (Equation 5.3), we described a prediction or regression equation that consisted of a single constant:
This prediction equation contains only a constant and no variables, so the predicted scores will be the same for all subjects. Of particular interest is the equation for which a is set equal to the mean:
8.1 SIMPLE REGRESSION AND CORRELATION
105
When the mean for a group of subjects serves as the predicted score for each subject, then the sum of the squared deviation scores (SS, or the sum of differences between observed and predicted scores for each subject) will be the minimum value possible for that group's data. This is the total SS, which is defined as the sum of the squared differences between each raw score and the group's mean. Similarly, the variance for the sample (or the total variance) is defined as the total SS divided by the sample size or N. The preceding sentences are nothing more than a restatement of material first presented in chapter 5. Recall, for example, that for the lie detection study the mean number of lies was 5.3, the SS was 40.1, the number of subjects was 10, and so the sample variance was 4.01. It is this variance, the SS divided by N, to which accounting for variance refers, and it is this variance or at least portions of it that we hope to account for with our research factor (or factors). The first prediction equation (Equation 8.2) considered only the mean. A second prediction equation, one that takes the values of a single predictor variable into account and assumes a linear relation between the predictor and the criterion variable, is
Following the usual convention, Y is used for the DV or criterion variable and X for the IV or predictor variable. This regression equation generates an expected or predicted score for each subject equal to the sum of a (which is called the regression constant) and the product of b (which is called the regression coefficient) times the value of the variable X for that subject; thus, the value of the predictor variable is multiplied or "weighted" by b. Graphically, Equation 8.3 is an equation for the straight line defined by a and b. The regression constant, a, is the Y intercept of the line—that is, it is the value of Y at the point the line crosses the Y axis. The regression coefficient, b, is the slope of the line—that is, it is the ratio of rise (a vertical distance) to run (a horizontal distance) for the line. If values of Y tend to increase as values of X increase, the line will tilt up to the right and the value of b will be positive. If values of Y tend to decrease with increases in X, however, then the line will tilt down to the right and the value of b will be negative. The mood scores for the 10 subjects in the lie detection study are given in Fig. 8.1. An obvious question at this point is, how do we even know if an equation that attempts to fit a straight line to these data is reasonable? The best advice is, first graph the data. Prepare a simple scattergram like that shown in Fig. 8.2 and decide, on the basis of visual inspection, whether the data appear to reflect a roughly linear relationship—that is, as a general rule, are the increases (or decreases) along the Y axis about the same anywhere along the X axis? (The graph in Fig. 8.2 was prepared with Excel. If your spreadsheet has a similar graphing capability, your next exercise should be to prepare a similar graph.) If number of lies detected were perfectly predicted by mood, then these points would all fall on a straight line. In this case (and in most cases encountered in the behavioral sciences) they do not form a straight line. On the other hand, there appears to be some orderliness to these data, a general tendency for higher mood scores to be associated with more lies detected; thus, it makes some sense to assume linearity and proceed with Equation 8.3. We could even attempt to fit a line to these data points visually. As an exercise, you should draw a straight line on Fig. 8.2 that you think provides the best fit for these data.
106
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR
FIG. 8.1. Mood data for the lie detection study. Once you have drawn a straight line, you can figure out the values of a (the Y intercept) and b (the slope) directly from the graph and then use these values and Equation 8.3 to compute Y', the number of lies predicted for each subject based on that subject's mood score. Alternatively, you could read the predicted values directly from the graph. To do so, you would locate each subject's mood score on the X axis, draw a vertical line from it to your fitted line, and then extend a horizontal line from that point over to the Y axis. The predicted number of lies is the Y value at the point where the horizontal line crosses the Y axis. And now that you know both observed and predicted scores for each subject, you can compute each subject's error or residual score, which is the difference between the number of detected lies actually observed and the number predicted by the equation. For example, three lies were detected for the first subject. This individual's mood score was 5.5. The predicted Y value (number of lies) will be the height of your visually fitted line above the X axis at the point on the X axis where X (mood score) equals 5.5. If this height is 5, then the error of prediction (or the residual) for the first subject would be 3 (the observed score) minus 5 (the predicted score), which equals -2. Earlier you computed a total sum of squares (the sum of the squared differences between observed scores and the group mean). You can
FIG. 8.2. Scattergram for predicting number of lies detected from mood.
8.1 SIMPLE REGRESSION AND CORRELATION
107_
now compute an error sum of squares. As you would expect, it is the sum of the squared differences between observed scores and the scores predicted by the Equation 8.3 regression line. (And, as demonstrated in the next exercise, the model sum of squares is the sum of the squared differences between the scores predicted by the regression line and the group mean.) In your attempt to find the best-fit line, you might draw several,slightly different lines. Each would be uniquely specified by the values for its intercept and slope, but only one line would be best, in the sense that the sum of the squared deviations or errors (the error sum of squares) would be minimized. This best-fit line could be determined empirically, which is the purpose of the next spreadsheet exercise.
Note 8.1
a
The regression constant. When Y, the criterion variable, is regressed on X, a single predictor variable, a is the Y intercept of the best-fit line.
b
The simple regression coefficient. Given two variables, b is the slope of the best-fit line.
Exercise 8.1 The Error Sum of Squares The template developed for this exercise computes the error sum of squares (as well as the total and model sums of squares) for specified values of the regression constant a and the regression coefficient b. This allows you to determine the best values for these two regression statistics. You may find it convenient to begin with the spreadsheet shown later in Fig. 5.4 and modify it according to the following instructions. General Instructions 1. Establish columns (columns like these will be used repeatedly in the exercises to follow), labeled appropriately and containing the appropriate data or formulas, for: a. The subject number. b. The number of lies detected (Y). c. The mood score (X). d. The predicted score (Y' = a + bX). e. The deviation between Y and the Y mean (y = Y- MY). f. The deviation between Y'and the Y mean (m = Y'- MY). g. The deviation between Y- Y'(e = Y- V'). h. y* y or the square of y, their sum is SS total. i. m*m or the square of m; their sum is SS model. j. e*e or the square of e; their sum is SS error.
108
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR 2.
Establish rows that compute sums, counts, and means for the columns as appropriate. Enter a formula for the SDY. Reserve space for the regression statistics a and b. 3. Experiment with various values for a and b and note the effect on the various deviation scores and sums of squares. What exactly do the deviation scores represent? Finally, enter 2.5 for a and 0.05 for b. Do you get the values shown in Fig. 8.3? Detailed Instructions 1. In row 2, label the columns as follows: Labeel
Col
s Y X Y' Y-My
A B C D E
Meaning
Label
Column Meaning
SStot
H
The subject number or index; ranges from 1-10. The observed number of lies detected (the DV). The mood score (the IV). The predicted number of lies detected. The difference between the observed y score and the mean y score (represented here with a lower case y). Y'-My F The difference between the predicted Y score and the mean y score (represented with m for model). Y-Y' G The difference between the observed Y and the predicted Y score (represented with e for error or residual). y*y H The square of the difference between the observed raw score and the mean. m*m I The square of the difference between the score predicted by the model and the mean. e*e J The square of the difference between the raw score and the one predicted by the model. 2. To remind ourselves what the sums of the squares in columns H-J represent, in row 1 label columns H-J as follows:
3. 4.
5. 6.
Total sum of squares--the sum of the differences between the raw scores and the mean, squared and summed. (On some previous spreadsheets this same quantity was labeled SSY.) SSmod I Model sum of squares or SS explained by the regression equation (or model)--it is the differences between the predicted scores and the mean, squared and summed. SSerr J Error or residual sum of squares or SS unexplained by the regression equation--it is the differences between observed and predicted scores, squared and summed. In addition, enter the labels "Lies" in cell B1, "Mood" in cell C1, "y=" in cell E1, "m=" in cell F1, and "e=" in cell G1. In column A, label rows 13-16 as follows: Label Row Meaning Sum= 13 The sum of the scores in rows 3-12. N= 14 The number of scores in rows 3-12. Mean= 15 The mean of those scores. a,b= 16 The values for the regression statistics (a in column B; b in column C). Enter the observed lie and mood scores in columns B and C, rows 3-12. For the time being, enter a value of 2.5 for a (the Y intercept) in cell B16 and a value of 0.5 for b (the slope) in cell C16.
8.1 SIMPLE REGRESSION AND CORRELATION
109
7. Enter a formula for the predicted value in column D, cells 3-12. The predicted value is a (cell B16) plus b (cell C16) times this subject's mood score (X, in column C). 8. In cells E3-E12, enter a formula for subtracting the mean number of lies detected (MY, in cell B15) from the observed Y score (Y, in column B). 9. In cells F3-F12, enter a formula for subtracting MY from the predicted Y score (Y,' in column D). 10. In cells G3-G12, enter a formula for subtracting Y' from the raw score observed for Y (column B). 11. In cells H3-H12 enter a formula for the total sum of squares (Y-MY, in column E, multiplied by itself). 12. In cells 13-112, enter a formula for the model or regression sum of squares (Y'-MY, in column F, multiplied by itself). 13. In cells J3-J12, enter a formula for the error or residual sum of squares (Y-Y', in column G, multiplied by itself). 14. In cells B13-J13, enter a function that sums the entries in rows 3-12. 15. In cells B14-J14, enter a function that counts the entries in rows 3-12. 16. Enter the label "VAR=" in cell G15 and the label "SD=" in cell G16. 17. In cells B15-D15 and H15-J15 enter a formula that computes the mean for the column. In cell H16 enter the formula for the standard deviation for the Y scores (the square root of the Y variance). The means for the SStotal, SSmodel. and SSerror (columns H-J) are total Y score, model, and error variances, respectively. At this point, your spreadsheet should look like the one shown in Fig. 8.3. Of particular interest in this spreadsheet are the errors, which are given in column G. These are the deviations of the observed values from those predicted by the line whose intercept is 2.5 and whose slope is 0.5. This particular straight line, however, was only an initial guess. It may or may not be the best-fit line. One way to find out is by trial and error. Try other values of a and b until you find those that yield a lower error sum of squares (cell J13) than the others. These values then define the best-fit line. Exercise 8.2 Finding the Best Fit Line Empirically The template shown in Fig. 8.3, which was developed during the last exercise, is used again for this exercise. Only the values for the parameters a and b are changed. The purpose is to find the best fit line by trial and error. 1. By a process of trial and error, and guided by an inspection of the scattergram in Fig. 8.2, replace values for a and b in the previous spreadsheet until you find values that give the smallest value you can discover for the sum of the squared differences between observed and predicted values (SSerror or error sum of squares). For the time being, limit your search to numbers that have only two significant digits. 2. Now set a to 3.0 (if this was not the value you ended up with in number 1) and try to find a value for b that minimizes the error sum of squares. Is this sum smaller than the one you found before? 3. Now set a to 3.0 and b to 0.47 (if this was not the value you ended up with in number 2). Are you able to find other parameter values that yield a smaller sum of squares than these?
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR
110
At this point your spreadsheet should look like the one shown in Fig. 8.4. The values computed for the various sums of squares (total, model, and error) are based on values of 3.0 and 0.47 for a and b, and although these values are used for discussion purposes in the next several paragraphs, they are not quite correct. As becomes evident, more than two significant digits are required to express the exact values for these regression statistics. Still, several aspects of this last spreadsheet deserve comment. As you would expect, the sums and means of the raw Y scores and the predicted Y scores are almost the same. Also as you would expect, the sums of the deviation scores based on the predicted score (predicted minus mean, raw minus predicted) are close to zero. If we had found the best values for a and b, not just values that were close, the means for the raw and the predicted scores would be identical and the sums of these two deviation scores would be exactly zero. An error sum of squares of 28.862625 (Fig. 8.4) is better than 30.3125 30.31 (Fig. 8.3), but it is still not the minimum possible. These are more accurate values, given to considerably more precision than is needed, not the less accurate values of 28.86 and 30.31 displayed in the figures. Remember that the values displayed by your spreadsheet depend on the format you specify and the width of the column you select, but that values as accurate as your computer allows are retained and used by the spreadsheet program. Also note that, for each subject, the raw minus the predicted (the error deviation score or Y i - Y i ' ) and the predicted minus the mean (the model deviation score or Yi' - My) sum to the total deviation score (Yi - My), as they must and as you can prove algebraically. That is, for rows 3-12, the score in column E is the sum of the scores in columns F and G. This will be true no matter what the values of a and b are; it follows from the way the deviation scores are defined.
A 1 2 3 4 5 6 7 8
c
B s 1
D
Lies Mood X Y
3 2 4 6 6 4
9 10 11
2 3 4 5 6 7 8 9
12
10.
9
13 Sum= 14 N= 15 Mean= a,b= 16 17
53 10 5.3 2.5
5 7 7
5.5 2 4.5 3 1.5 6.5 3.5 7
6 9
F
E
y= m= Y' Y-My Y'-My 5 .25 3.5 4 .75 4 3 .25 5 .75 4 .25 6 5.5 7
48.5 49 .25 10 10 4.85 4.925 0.5
-2.3 -3.3 -1.3 0.7 0.7 -1.3 -0.3 1.7 1.7 3.7
G
H
I
J
e=
SStot
SSmod
SSerr
Y-Y'
y*y
m*m
e*e
-2.25 5.29 0.002 5.063 -1.5 10.89 3.24 2.25 -0.55 -0.75 1.69 0.303 0.563 2 0.49 4 -1.3 1.69 -2.05 2.75 0.49 4.203 7.563 0.45 -1.75 1 .69 0.203 3.063 -1.05 0.75 0.09 1.103 0.563 1 2.89 0.49 0.7 1 0.2 1.5 2.89 0.04 2.25 2 13.69 2.89 4 1.7 -0.05
-1.8
0 -3.75 10 10
40.1 14.16 30.31 10 10 10 10 VAR= 4.01 1.416 3.031 SD= 2.002 3.75
FIG. 8.3. Spreadsheet for finding the best-fit line by trial and error: results when a = 2.5 and b = 0.5.
111
8.1 SIMPLE REGRESSION AND CORRELATION
However, only for the best-fit line will the model sum of squares and the error sum of squares add up exactly to the total sum of squares. That is, for the best-fit line,
The total sum of squares for the present example is 40.1 (see Fig. 8.4). The model (or regression, or explained) sum of squares and the error (or residual, or unexplained) sum of squares would add up exactly to 40.1 if we had found the best fit values for a and b. In this case, they sum to 40.13825 (using nonrounded values), a value just slightly over 40.1. That means we are close but still not exactly on target with our guesses for a and b. Exercise 8.3 Deviation Scores The purpose of this exercise is to demonstrate graphically how the various deviation scores are formed and how they are related. 1. On a piece of graph paper, graph the mood/lies data, that is, reproduce the scattergram shown in Fig. 8.2. Be sure to label the X and Y axes appropriately. By convention, the dependent variable is usually graphed on the Y axis. 2. Draw the line whose Y intercept is 3.0 and whose slope is 0.47. Two points uniquely define a straight line, consequently an easy way to do this is to compute values of Y associated with two arbitrarily chosen X values using the formula Y'= 3.0 + 0.47X.
B
A 1 2
s
3 4 5 6 7 8 9 10 11 12
1 2 3 4 5 6 7 8 9 10
13 Sum= 14 N= 15 Mean= 16 a,b= 17
Lies Mood X Y
F
E
D
C
G
SSerr
m*m
e*e
5.29 -2.3 0.285 -2.59 -3.3 -1.36 -1.94 10.89 1.69 -1.3 -0.19 -1.12 0.49 1.59 0.7 -0.89 -1.6 2.295 0.49 0.7 1.69 -1.3 0.755 -2.06 -0.3 -0.66 0.355 0.09 0.71 2.89 1.7 0.99
0.081 1.85 0 .034 0 .792 2 .544 0.57
e= Y-Y1
4.5 5.115 3 4.41 1.5 3.705 6.5 6.055
5 7 7
3.5 4.645 7 6.29 6 5.82
1.7
0.52
9
9
7.23
3.7
1.93
53 10 5.3 3
48.5 10 4.85 0.47
52.8 10
0 10
5.28
J
S Smod
m=
5.5 5.585 2 3.94
I
sstot y*y
y=
Y' Y-My Y'-My
3 2 4 6 6 4
H
1.18 1.77
-0.2 0.205 10 10
2.89 13.69
6.682 3.764 1.243 2.528 5.267 4.223 0.429 0.126 0.98 0.504 0.27 1.392 3 .725 3.133
40.1 11.28 28.86 10 10 10 4.01 1.128 2.886
VAR= SD= 2.002
FIG. 8.4. Spreadsheet for finding the best-fit line by trial and error: results when a = 3 and b = 0.47.
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR
112
3. Draw arrows representing each deviation score. There are three kinds of deviation scores: total (Y- MY), model (Y'- MY), and error (Y- Y'). Arrows will begin at points where Y = MY for the total and model deviations and at points where Y= Y' 'for the error deviations. They will go up (+) or down (-) depending on the sign of the deviation scores. Their associated mood scores will determine how far away from the Y axis they are. For example, for subject 3 the mood score is 4.5 and the deviation between raw score and mean is 4 - 5.3, which equals -1.3. Therefore draw an arrow beginning at the point on the graph where X = 4.5 and Y= 5.3 and ending with an arrowhead at the point where X = 4.5 and Y= 4. Note that this arrow, which represents a deviation of -1.3, extends 1.3 units and points down. You will probably want to use three different colors (or some other stylistic device) to represent the three kinds of deviations. The three kinds of deviations (total, model, and error) and their associated sums of squares are basic to an understanding of much of the material presented in this book. It is worthwhile to ponder the figure you produced for the previous exercise (see Fig. 8.5) and think about what it represents. What do total deviations represent? Model deviations? Error deviations? Based on the figure, how strong is the relation between mood and number of lies detected? WhyT
Note 8.2 Y-My Y-My
The total deviation score, that is, the difference between each subject's observed score and the group mean. (Subscripts indicating subject are not shown.)
V - MY
The model (or regression) deviation score, that is, the difference between the score predicted by the regression equation (or model) for each subject and the group mean. (Subscripts indicating subject are not shown.)
Y-Y'
The error (or residual) deviation score, that is, the difference between each subject's observed score and each subject's predicted score. (Subscripts indicating subject are not shown.)
SStotal
The total sum of squares, or Y - My for each subject, squared and summed. This can also be symbolized SSy.
SSmodel
The model sum of squares, or Y - MY for each subject, squared and summed. This can also be symbolized 5Sreg, for regression sum of squares, or SSexp, for sum of squares explained by the model.
SSerror
The error sum of squares, or Y - Y' for each subject, squared and summed. This can also be symbolized SSres, for residual sum of squares, or SSunexp, for sum of squares left unexplained by the model.
8.2 WHAT ACCOUNTING FOR VARIANCE MEANS 8.2
113
WHAT ACCOUNTING FOR VARIANCE MEANS From a best-fit line (and remembering that we have not yet determined the absolute best-fit line for the lie detection data, although we will do so in the next chapter), two important facts can be determined. 1. The exact nature of the linear relation between predictor and criterion variable. 2. The strength of the linear relation between predictor and criterion variable. The nature of the linear relation is indicated by the regression statistics for the best-fit line, whereas the strength of the linear relation is indicated by the ratio of variability explained by the regression model to total variability. Exactly what this means is explained in the next several paragraphs.
The Exact Nature of the Relation Assume the values for a and b given in Fig. 8.4 are accurate. (We know they are accurate to two significant digits.) The value of a indicates that, for mood scores of zero, 3.0 lies would be detected (Y' = 3.0 + 0.47X = 3.0 when X= 0). This statement is meaningful only if mood scores of zero are meaningful. Although we can imagine any best-fit line stretching to infinity in both directions, any actual line we draw will be based on a particular sample of X values; values that will necessarily be confined to a particular range. In this case, mood scores ranged from 2 to 9, not minus to plus infinity, so predictions of number of lies based on mood scores outside this range (like zero) may not make much sense. More interesting is the information conveyed by the value for b. It indicates that, for each increase of one mood point, the number of lies detected increases 0.47. Numbers less than 1 often seem difficult for many people to grasp, thus in this case we might instead say that for each increase of 2.13 mood points, the number of lies detected increases by 1. (The algebra is as follows: b = rise:run = 0.47/1; 0.47/1 = 1/X; O.47X = 1; X = 1/0.47 = 2.13 rounded.) It is important to remember that the exact nature of the relation between a predictor and criterion variable—the change in value for the criterion expected for a given change in the predictor—is an idealization based on the model. It is an ideal statement about an ideal world, one in which error is ignored and the model is assumed to be true, when in fact it is better viewed only as a probabilistic guide. In the behavioral sciences, observed values often deviate strikingly from those predicted by a particular model, even though that model is based on the best possible fit to the data. Thus it becomes important to assess, not just what the best-fit model is (i.e., what are the best-fit values for a and b), but also how well that model fits a particular set of data. In other words, although we may have found a best-fit line, the question remains, how good is that fit?
The Strength of the Relation The strength of the relation (or association) between a particular predictor variable and its criterion, or the goodness of fit, can be assessed by the proportion of criterion to total variance that is accounted for when predictions take the predictor variable into account, that is, when predictions are based on the regression equation (Equation 8.3) instead of just on the group mean (Equation
114
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR 8.2). In effect, two variances are compared. First is total variance or the extent to which raw scores deviate from the mean (see Fig. 8.5, top). Second is error variance or the extent to which raw scores deviate from the best-fit line (see Fig. 8.5, bottom). Error variance is represented graphically by the dispersion of points around the best-fit line, whereas total variance is represented by the dispersion of points around the horizontal line representing the Y mean (recall Exercise 8.3). When the relation between X and Y is weak, the dispersion of data points around the best-fit line is almost as great as the dispersion around the mean line. When the relation between X and Y is strong, however, the data points are clustered far more tightly around the best-fit line than around the mean line; basing predictions on the linear model (Y'i = a + bXi) instead of the mean model (Yi' = My) results in a clear reduction in variability. For the present example, the total variance for the mean model (the SStotal divided by N) was 4.01, whereas the error variance (the SSerror divided by N) was 2.89 (see Fig. 8.4). We are now in a position to answer the question posed earlier in this chapter: How much do we gain by stepping up from the one-parameter mean model (with requires a value only for My) to the two-parameter linear model (which requires values for a and b)? Recall that for the best-fit linear model, the total sum of squares (SStotal) can be divided (or partitioned) into two pieces, the sum of squares associated with or explained by the regression model (SSmodel) and the error or residual sum of squares (SSerror), which is the sum of squares left unexplained by the model (Equation 8.4). The total variance, which is simply the total sum of squares divided by N, can also be partitioned in the same way:
The mean model generates the same value for all predicted scores. As a result, there is no variability in predicted scores and no variability to be associated with the model. In the case of the mean model, total variance is all error variance, portions of which we hope to account for by more complex models. Usually the error variance for the linear model will be less than the error (i.e., total) variance for the mean model. The amount by which it is less is the variance due to the model and represents the effect of the additional information (i.e., each subject's values for the predictor variable) used in the linear model as compared to the mean model. For the present example, the error variance for the mean model is 4.01 and the error variance for the linear model is 2.69 (assuming a = 3.0 and b = 0.47). This represents an improvement of approximately 4.01 minus 2.89, or an improvement of 1.12 (rounded). This improvement is due to the variance accounted for by the linear model (VAR model ), which according to Fig. 8.4 is 1.13 (rounded) and which would be exactly 1.13 (actually, 1.1275625) if we had used values for a and b that produced the absolute best-fit line. In any case, compared to the mean model, the linear model reduces error variance by approximately 28% (1.13 divided by 4.01 times 100). In other words, for the present example the linear model accounts for approximately 28% of the variance in criterion variable scores. The formula for proportion of variance accounted for can be expressed in terms of either variances or sums of squares. In terms of variances,
8.2 WHAT ACCOUNTING FOR VARIANCE MEANS
115
This formulation clearly expresses the notion that what is being computed is the proportion of total variance accounted for by the model. The second variant is given because sometimes total and error variances are more readily available than model variance, and clearly, if VARtotal = VARmodel + VARerror then VARmodel = VARtotal - VARerror. The various variances given in Equation 8.6 are all the appropriate sums of squares divided by N, the sample size, and therefore all Ns implicit in Equation 8.6 cancel, leaving
FIG. 8.5. Total (top), model (middle), and error (bottom) deviations; when squared and summed, the result is the sum of squares total, for the model, and for error.
116
ACCOUNTING FOR VARIANCE: A SINGLE PREDICTOR
This formulation bypasses the divisions by N required for computing the variances in Equation 8.6 and for that reason may occasionally be more convenient. But both formulas yield identical results. As an additional aid to understanding the concept of accounting for variance, it can be helpful to consider extreme, even degenerate circumstances. For example, imagine that prediction were perfect, that every data point fell on the best-fit line. As always, the total sum of the deviation scores squared (deviations of raw scores from the mean) would reflect variability for the criterion variable, the very thing we want to account for. But in this case, each error deviation would be zero (because every point is perfectly on the line), so the sum of the squared residuals would be zero. As a result, total and model sum of squares would be identical. With no error variance, all of the total variance is accounted for by the model. In other words, the SSmodel divided by the SStotal would equal exactly 1. When prediction is perfect, 100% of the variability in criterion scores is accounted for by the model; there is no error variability. Now imagine there is no association between predictor and criterion variables. In this case, the slope for the best-fit line would equal zero, indicating no relation, and the Y intercept would equal the mean of the Y scores. That is, the best-fit line would be horizontal (zero slope) and would intercept the Y axis at the Y mean. Given no relation, the best prediction for a Y score is simply the Y mean; there is no way to improve it. As a result, the sum of squares explained by the model is zero (because the model predicts the mean, the deviations between predicted and mean scores are zero). With no model variance, none of the total variance can be due to the model. In other words, the SSmodel divided by the SStotal would equal exactly zero. When prediction is nil, 0% of the variability in criterion scores is accounted for by the model. Normally, of course, the proportion of variance accounted for falls somewhere between 0 and 1. For the present example (Fig. 8.4), the sum of squares for the number of lies detected is 40.10. The sum of squares explained by the model is 11.28 and the error sum of squares is 28.86 (rounding to four significant digits). Thus the proportion of variability in number of lies detected that is accounted for by mood scores is .281 (11.3 divided by 40.1; but remember, this is based on values for a and b determined by trial and error and accurate only to two significant digits, which is why the present .281 is different from the more accurate .302 we compute in the next chapter. The purpose of the past several paragraphs has been to develop a clear understanding of what it means to say that a given proportion of criterion variable variance is accounted for by a particular predictor variable. Many readers will already have recognized that the proportion of variance accounted for is an important and widely used descriptive statistic, known as the coefficient of determination or r2 (read r-squared). It indicates the degree or strength of association between two variables, it can vary from 0 to 1, and it is symbolized and defined as follows:
Historically, another statistic, the correlation coefficient or r, was developed before r2. In traditional statistic texts, r is usually introduced first and receives
117
8.2 WHAT ACCOUNTING FOR VARIANCE MEANS 2
2
more attention than r . We prefer to emphasize r , partly because it has such a clear and concrete meaning (proportion of variance accounted for) and partly because accounting for variance is a general and powerful concept, one not limited to cases involving single predictor variables. In any case, and ignoring for a moment the matter of sign, you can always compute the value of r simply by taking the square root of r2. In this chapter you have been introduced to four important descriptive statistics: the regression constant or a, the regression coefficient or b, the correlation coefficient or r, and the coefficient of determination or r2. The regression coefficient indexes the exact nature of the relation between two variables, r indexes the strength of association between the predictor and the criterion variables, and r2 indicates the proportion of criterion variable variance accounted for by the model that includes the predictor variable. You have also learned how to determine the value of a and b by trial and error and how to use them to compute first the SStotal, SSmodel, and SSerror, and then r2. At this point, you should have a good understanding of accounting for variance. In chapter 9 you will learn how to compute the exact, best-fit values for a and b.
Note 8.3 VARtotal
The variance for the scores in a sample. It indicates the variability of raw scores relative to the group mean. To compute, divide SStotal by N. The subscript indicates the group of scores in question, e.g., VARx for the X scores, VARy for the Y scores, and so forth.
VARmodel
The model variance. It indicates the variability of scores predicted by a specified regression equation or model relative to the group mean. To compute, divide SSmodel by N.
VARerror
The error variance. It indicates the variability of raw scores relative to scores predicted by a specified regression equation or model. To compute, divide SSenor by N.
This page intentionally left blank
9
Bivariate Relations: The Regression and Correlation Coefficients
In this chapter you will: 1. Learn how to compute the slope and Y intercept for the best-fit line. 2. Learn how to compute and interpret the correlation coefficient. 3. Learn how to account for variance with a single predictor, either quantitative or binary. 4. Learn how to graph the regression line for both a quantitative and a binary predictor variable. In the preceding chapter, as a way of describing the relation between two variables, you determined the best-fit line by trial and error. You also learned that the slope of that line (the regression coefficient) indicates the exact nature of the linear relation between the independent and dependent variable. Topics introduced in the last chapter are discussed further in this chapter. The focus remains on descriptive statistics, and the major question remains: How can the strength of the relation between a quantitative dependent variable and an independent variable (either quantitative or qualitative) be assessed? Other ways to state this question are: How can we describe the effect of one variable on another, and how much variability in one variable can be accounted for by another? In the previous chapter, the numeric example used a quantitative independent or predictor variable (mood). In this chapter, after describing procedures for computing the regression and correlation coefficients, we demonstrate how the methods learned in the last chapter also apply when the predictor variable is binary. This is useful because researchers often compare two groups and in such cases the variable indicating group membership will be binary. 9.1
COMPUTING THE SLOPE AND THE Y INTERCEPT As you probably know, or certainly have guessed, in practice the slope and Y intercept are computed using formulas developed by statisticians, not discovered
119
120
BIVARIATE RELATIONS: REGRESSION, CORRELATION empirically by refined guesswork as demonstrated in the previous chapter. Still, it is instructive to learn the trial-and-error method first, partly because it is so simple and it does work, but mainly because it helps develop a clear and intuitive sense of a best-fit line. Statisticians can prove that the formulas for the slope and Y intercept given in this section do indeed result in best-fit lines. We do not reproduce their arguments here, preferring to leave such discussion for advanced and more mathematically oriented statistics texts. However, you can use the spreadsheets developed in the last chapter to demonstrate to yourself that the computed values for a and b really do describe a best-fit line. You will not find it possible to find any other values for a and b that yield a smaller value for the error sum of squares. Keep in mind, however, that the computations for the Y intercept and slope presented here apply only when a criterion variable is being accounted for by a single predictor variable. Procedures used when more than one predictor variable is under consideration are presented in subsequent sections. If we let a lower case y represent the total deviation score for a Y score, so that yi =Y0i- MY, then the sum of squares for Y is:
In other words, the sum of squares for Y is the sum of the squared differences between the raw Y scores and the Y mean. Likewise, if we let a lower case x represent a total deviation score for an X score, so that xi = Xi - MX, then the sum of squares for X is:
As you already know, the sum of squares for Y indicates the variability of the Y scores. Similarly, the sum of squares for X indicates how variable the X scores are. Now we can define a new sum of squares, the XY sum of squares, that indicates the extent to which the X and Y scores covary:
Note that SSx and SSy are always positive (because deviations are multiplied by themselves) but SS xy can be negative as well as positive. If X and Y covary in a positive direction—that is, if small values of Y tend to be paired with small values of X and large values of Y tend to be paired with large values of X— then most of the products of the paired x and y deviation scores will be positive (because most paired deviations will have the same sign), resulting in a large, positive value for SSxy. On the other hand, if X and Y covary in a negative direction—that is, if large values of Y tend to be paired with small values of X, and small value of Y tend to be paired with large values of X—then most of the products of the paired x and y deviation scores will be negative (because most paired deviations will have opposite signs) and thus the value of SSxy will be large and negative. There is a third possibility, of course: X and Y may not covary at all. In that case, about half of the xy products would be positive, half negative, and SS xy would be some value close to zero. You already know how to compute the variance for the X and Y scores in a sample. These variances are:
9.1 COMPUTING THE SLOPE AND THEY INTERCEPT
121
Similarly, the sample covariance is the XY sum of squares divided by the sample size:
We remind you about variances, and introduce the covariance, because the slope for the best-fit line can be defined in terms of these statistics. If Y is the criterion variable and X the predictor variable, then the slope for the best-fit line is the ratio of the covariance for X and Y to the variance for X:
(The Ns used to compute the covariance and variance cancel, so the slope could also be expressed as the ratio of SSxy to SSx.) If the covariance is positive, then the slope is positive (and tilts up to the right); if the covariance is negative, then the slope is negative (and tilts down to the right). In terms of units, Equation 9.7 makes sense. If Y is lies and X is mood, then the numerator, which is the product of lie and mood deviation scores, is measured in lie-mood units (or lies times mood). Similarly, the denominator is measured in mood-squared units (like area is measured in square feet). When lie-mood units are divided by mood-squared units, the mood units cancel, and the units for the resulting dividend are lies/mood (or lies per mood), which is exactly what the slope is: a ratio of rise to run or the increase in Y (number of lies) per unit increase in X (mood). If you remember this, you should have no trouble remembering that the slope is the covariance for X (run) and Y (rise) divided by the variance for X (run-squared). Given the slope, and the mean values for X and Y, the Y intercept can be determined by geometry. The formula for the Y intercept is:
By definition, the best-fit line must go through the point where X = MX and Y = My. Its slope, or the ratio of rise to run, is the increase in Y units for each X unit. At the point where the line intercepts the Y axis, X equals zero. In running from X = 0 to X = MX, a distance of MX X units, we rise b times Mx Y units. In order to get back to the Y value corresponding to X - 0 (the Y intercept), we in effect run backward, subtracting the total rise (b times MX) from the mean for Y, as indicated by Equation 9.8. If this seems unclear, try making a simple figure that depicts Equation 9.8 Exercise 9.1 Computing the Y intercept and Slope The template developed for this exercise adds the ability to compute the exact values for the regression constant (the Y intercept or a) and the regression coefficient (the slope or b) to the template shown in Fig. 8.4.
122
BIVARIATE RELATIONS: REGRESSION, CORRELATION General Instructions 1. Add three columns, labeled appropriately and containing the appropriate formulas, to your Exercise 8.4 spreadsheet. These columns are for: a. The deviation between X and the X mean (x= X- Mx). b. The square of x; their sum is SSX c. THE product of xy, their sum is SSXY. 2. Establish rows that compute sums, counts, and means for these columns as appropriate. Enter a formula for the SDX. 3. Now that your spreadsheet contains all the elements needed, enter formulas for a and b, the Y intercept and slope. Do your computed values agree with those shown in Fig. 9.1? How does the current value for the error sum of squares compare with the value computed for the last exercise? Detailed Instructions 1. Begin with the spreadsheet shown in Fig. 8.4. Insert a blank column between columns G and H, or, alternatively, move the block H1-J16 over one column, to I1-K16. This opens up column H, which will be used for labeling. 2. Move the VAR= and SD= labels in cells G15-G16 to H15-H16. 3. In row 2, add the following labels to columns L-N:
4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
Label
Column
X-Mx
L
Meaning
The difference between the observed mood score and its mean (symbolized x). x*x M The product of the mood deviation scores. The sum is SSx and indicates variance in X scores. x*y N The crossproduct of the mood and lie deviation scores. The sum is SSxy and indicates covariance in X and Y scores. In addition, enter the labels "x=" in L1, "SSx" in M1, and "SSxy" in N1. In column L (cells 3-12), enter a formula for subtracting the mean mood score (Mx, in cell C15) from the observed X score (in column C). In column M (cells 3-12), enter a formula for squaring the mood deviation score (in column L). In column N (cells 3-12), enter a formula for the cross product of the mood and lie deviation scores (in column E). In cells L13-N13, enter a function that sums the entries in rows 3-12. In cells L14-N14, enter a function that counts the entries in rows 3-12. In cells M15-N15, enter a formula that computes the mean for the column. The column M mean is the variance for X and the column N mean is the XY covariance. In cell M16, enter a formula for the standard deviation for the mood or X scores. In cell C16, enter the formula for the slope. This is the covariance (cell N15) divided by the X variance (cell M15). In cell B16, enter a formula for the Y intercept. This is the product of the slope (cell C16) and the X mean (cell C15) subtracted from the Y mean (cell B15).
At this point, your spreadsheet should look like the one shown in Fig. 9.1. Now we can see that the computed values for a and b (rounded to five significant digits) are 3.0235 and 0.46938 respectively. (Remember, the number of digits displayed depends on the format and column width you select; the number of digits stored internally is not affected by the display.)
123
9.1 COMPUTING THE SLOPE AND THE Y INTERCEPT
The guesses we used in the previous chapter for a and b, 3.0 and .47 (see Fig. 8.4) , were close but not the best possible. Now that our estimates for a and b are mor e accurate (Fig. 9.1), the error sum of squares (cell K13) is 28 .85840274,
A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
B
s 1 2 3 4 5 6 7 8 9 10 Sum= N= Mean= a,b=
H
Lies Y 3 2 4 6 6 4 5 7 7 9 53 10 5.3 3.024
C Mood X 5.5 2 4.5 3. 1.5 6.5 3.5 7 6 9 48.5 10 4.85 0.469
I
J
y= Y' 5.605 3.962 5-136 4.432 3.728 6.074 4.666 6.309 5.84 7.248 53 10 5.3
Y-My -2.3 -3.3 -1.3 0.7 0.7 -1.3 -0.3 1.7 1.7 3.7 0 10
m= Y'-My 0.305 -1.34 -0.16 -0.87 -1.57 0.774 -0.63 1.009 0.54 1.948 0 10
K
L
M
1
SStot
SSmod
SSerr
x=
2
y*y
m*m
e*e
X-Mx
5.29
0.49
0.093 1.79 0.027 0.754 2.472
1.69
0.6
0.09
0.402 1.018 0.291 3.794 11.24
6.787 3.851 1.29 2.46 5.164 4.303 0.111 0.477 1.346 3.07 28.86
0.65 -2.85 -0.35 -1 .85 -3.35 1.65 -1.35 2.15 1.15 4.15
3 4 5 6 7
10.89
1 .69 0.49
8 9 10 11 12 13 14 15 16 17
2.89 2.89 13.69 Sum=
40.1
G
F
E
D
N=
10
10
10
VAR=
4.01
1.124
2.886
SD=
2.002
0 10
e=
Y-Y' -2.61 -1.96 -1.14 1.568 2.272 -2.07 0.334 0.691 1.16 1.752 0 10
N SSX x*x
SSxY
0.423 8.123 0.123 3.423 11.22 2.723 1.823 4.623 1.323 17.22 51.03
-1.5 9.405 0.455 -1.3 -2.35 -2.15 0.405 3.655 1.955 15.36 23.95
x*y
10
10
5.103 2.259
2.395
FIG. 9.1. Spreadsheet for finding the Y intercept and slope of a best-fit line predicting number of lies detected from mood scores.
124
BIVARIATE RELATIONS: REGRESSION, CORRELATION which is very slightly less than the 28.862625 from Fig. 8.4 (when we adjust the column width to see more significant digits than the 28.86 displayed in the figures). This value from Fig. 9.1 should be the smallest value possible. Moreover, the model and error deviation scores (columns F-G) now sum to zero, as they should (or else a number like -2E-15, which in scientific notation is -2 divided by a ten followed by 15 zeros and which is essentially zero). And the SSmodel and the SSerror (cells J13-K13) sum exactly to SStotal (values, rounded to four significant digits, for SSmodel and SSerror are 11.24 and 28.86, which sum exactly to 40.10). Given a single predictor, then, the spreadsheet shown in Fig. 9.1 can be used to compute the regression constant (a or the Y intercept) and the regression coefficient (b or the slope for the best-fit line). In the next section, we show how to compute the correlation coefficient (r) as well. Note 9.1 SSxy The XY sum of squares, or E (Xi - MX) (Yi - My), where i = 1,N. Thus SSxy is the sum of the cross products of the X and Y deviation scores. COV The covariance. If for the X and Y scores, it is SSXY divided by N. Thus COVXY is the average cross product of the X and Y deviation scores. It indicates the direction (plus or minus) and the extent to which the X and Y scores covary, and is related to both the correlation and the regression coefficients.
9.2
COMPUTING THE CORRELATION COEFFICIENT The Pearson product-moment correlation coefficient, which is symbolized by r, is an historically important and widely used index of association usually applied to quantitative variables. It was first defined by Karl Pearson in 1895. Although Pearson approached the matter somewhat differently, we have already noted that r2 is the proportion of variance accounted for in a criterion variable when values for the predictor variable are taken into account. Thus, one way to compute r is simply to take the square root of r2:
This gives the absolute value of r; in order to determine its sign (whether the relation is positive or negative) you would need to look at the scattergram or the sign of the covariance. For the present example, relating number of lies detected to mood:
The values for VARmodel , VARtotal, and COVxy (which is positive) are taken from Fig. 9.1. The Ns in the variance formula cancel, so we could have defined r2 as the SSmodel divided by SStotal instead (11.24 divided by 40.1), which gives the same result. The correlation coefficient can vary from –1 (a perfect negative relation) to zero (no relation) to +1 (a perfect positive relation), whereas r2 can vary only
9.2 COMPUTING THE CORRELATION COEFFICIENT
125
from zero (no variance accounted for) to +1 (all variance accounted for). If you compute r as the square root of r2, you will need to supply the appropriate sign. As noted earlier, the covariance (cell N15 in Fig. 9.1) indicates the direction of the relation, so its sign can be used if you compute r as the square root of r2. In addition, there are several other ways to compute the standard Pearson correlation coefficient. One of these is demonstrated in the next exercise. Exercise 9.2 Computing the Correlation Coefficient Using Z Scores The purpose of this exercise is to add the ability to compute the correlation coefficient (r) to the template shown in Fig. 9.1. The method used for computing the correlation coefficient is based on Z or standardized scores. General Instructions 1. Add three columns, labeled appropriately and containing the appropriate formulas, to your Exercise 9.1 spreadsheet. These columns are for: a. Zy, the standardized Y score. b. Zx, the standardized X score. c. Zy Zx, the product of ZY and Zx. 2. Establish rows that compute sums, counts, and means for these columns as appropriate. 3. Provide labels and formulas for r and r2. Note that r is the average cross product of the Z scores. Compute r2 by dividing the variance accounted for by the model by the total variance. Is the square root of this value the same as the average cross product of the Z scores? Would it be if the average cross product were negative? Detailed Instructions 1. Begin with the spreadsheet shown in Fig. 9.1. In row 2, add the following labels to columns O-Q: Label
2. 3. 4. 5.
6. 7. 8.
Column
Meaning
ZY O Standard or Z scores for number of lies detected. Zx P Standard or Z scores for mood. ZY* Zx Q The cross product of ZY and Zx. In column O, enter the formula for a standardized lie score (Use the standard deviation for Y in cell 116.) If you do not remember the formula, reread the section on standard scores in Chapter 5. In column P, enter the formula for a standardized mood score. (Use the standard deviation for X in cell M16.) In column Q, enter a formula for the cross product of the ZY and Zx scores. In cells O13-Q13, enter a function for summing the data entries in the column. In cells O14-Q14, enter the function for counting the number of data entries in columns O to Q. In cells O15-Q15, enter a formula for the mean of the data entries in columns O to Q. The value in cell Q15 is the mean of the cross products of the Z scores. Enter the label "r=" (for correlation coefficient) in cell A17 and the label "R2=" (for r2 or r-squared) in cell H17. Point cell B17 to cell Q15 (the mean cross product of the standard scores). In cell 117, enter a formula for dividing model variance (cell J15) by total variance (cell 115). This is r2.
BIVARIATE RELATIONS: REGRESSION, CORRELATION
126
At this point, your spreadsheet should look like the one shown in Fig. 9.2. You already know from Equation 9.9 that the value of the correlation coefficient for our current running example is 0.529 (the square root of VARmodel divided by VARtotal). An examination of the exercise just completed should reveal a second way to define and compute the correlation coefficient. As you can see, the correlation coefficient is the average cross product of the standard or Z scores:
This formulation emphasizes the fact that the correlation coefficient is an average statistic that, like the mean, summarizes data for a group. For the present example (and rounding the correlation coefficient to two significant digits),
There is another way to define and compute the correlation coefficient. Again, the values needed are given in the spreadsheet you just prepared. This definition, like the one for the regression coefficient given in Equation 9.7, uses the variances and covariances for the two variables involved. It is:
H
B
A
I
Lies
1 2
s
Y
3 4
1 2
5 6 7 8
3 4 5 6
3 2 4
9 10 11 12
7 8 9 10
5 7 7 9
13 14
Sum=
Sum=
N=
53 10
N=
10
15
Mean=
5.3
16 17
a,b=
3.024 0.529
VAR= SD=
4.01 2.002
R2=
0.28
r=
P
0
Q
SSY
y*y 5.29 10.89 1 .69 0.49 0.49 1.69 0.09 2.89 2.89 13.69
6 6 4
40.1
Zy
Zx
Zy*Zx
-1.15 -1 .65 -0.65 0.35 0.35 -0.65 -0.15
0.288
-0.33
-1.26 -0.15 -0.82 -1.48 0.73 -0.6
2.079
0.849 0.849
0.952 0.509
1.848
1.837 2E-15
0.808 0.432 3.395 5.295
0 10 0
0.101 -0.29 -0.52 -0.47 0.09
10
10
2E-16
0.529
FIG. 9.2. Spreadsheet for computing the correlation between number of lies detected and mood scores. Columns that are the same as Fig. 9.1 are not shown.
9.2 COMPUTING THE CORRELATION COEFFICIENT
127_
For the present example,
(The values for these variances and covariances are given in cells I15, M15, and N15 in Fig. 9.1.) In other words, the correlation coefficient can be defined as the ratio of the covariation of X and Y, relative to the square root of the variation of X and Y considered separately. Both variance and covariance are group summary statistics, reflecting average variability about group means, which again serves to emphasize that r is an average, group-based statistic. It does not indicate the relation between two variables for an individual, but rather indicates the average relation between those variables for a group of individuals. However defined or computed, the correlation coefficient is by far the most common index of association for two quantitative variables. Moreover, the same identical computations can be applied when both variables are ranked (ordinal) data, in which case the statistic is called the Spearman rank-order correlation coefficient; when one of the variables is quantitative (i.e., interval-scaled) and the other is binary, in which case the statistic is called a point-biserial correlation; and when both variables are binary data, in which case the statistic is called the phi coefficient. Historically, special computational formulas have been developed for each of these cases, but they are merely computationally convenient derivatives of the formulas presented here. Unfortunately, the separate formulas have led generations of students to think these variants of the correlation coefficient were somehow different things. Because of their widespread use, it is important for you to know the different names given these correlation coefficients, but at the same time to understand that they are fundamentally the same. One final comment: We think it often makes more sense to report r2 instead of r. The units of the correlation coefficient have no particular meaning. An r of .4, for example, is not twice as much correlation as an r of .2, and the difference between an r of .8 and an r of .9 is not the same as the difference between an r of .4 and an r of .5. The units of r2, however do have a more intuitively grasped meaning: the proportion of variance accounted for.
The Correlation and Regression Coefficients Compared The correlation coefficient (r) and the regression coefficient (6, or the slope of the best-fit line) appear very similar. According to Equation 9.11, one definition for r is:
And according to Equation 9.7, the definition for b is:
The similarity between the two equations is made more evident if we substitute the square root of the X variance squared for VARx in the denominator:
128
BIVARIATE RELATIONS: REGRESSION, CORRELATION
This superficial similarity between r and 6, however, masks a very real difference. As noted earlier, the regression coefficient indicates the exact nature of the relation between X and Y. Its value indicates how much Y increases (or decreases, if b is negative) for each unit change in X. Its units are X units per Y units (e.g., lies per mood score). The correlation coefficient, on the other hand, is "unit free" (numerator and denominator units cancel). It is a pure index of the strength of the association between X and Y. Both correlation and regression coefficients convey important, but different, pieces of information. When reporting the results of the lie detection study, it would be traditional to say that mood and number of lies detected correlate 0.543. As noted earlier, our preference is to report r2, not r, because we think it is more informative to say that 29.5% of the variance in number of lies detected is accounted for when mood scores are taken into account. Either way, r or r2 provides information about the strength of the relation. In addition, in order to provide information about the exact nature of the relation, you would report that for each increase of 1 mood point, 0.0465 more lies were detected on average. Alternatively, because decimals less than 1 often seem difficult to grasp, you might report instead that for each increase of 21.5 mood points, one more lie is detected (recall the algebra used in Chap. 8).
Note 9.2 b The simple regression coefficient. Given a single predictor variable, b is the slope of the best-fit line. It indicates the exact nature of the relation between the predictor and criterion variables. Specifically, b is the change in the criterion variable for each one unit change in the predictor variable. To compute (assuming that Y is the criterion or dependent variable), divide the XY covariance by the X variance (b = COV XY /VAR x ). a The 'regression constant. When Y, the criterion variable, is regressed on X, a single predictor variable, a is the Y intercept of the best-fit line. To compute, subtract the product of b (the simple regression coefficient) and the mean of X from the mean of Y (a - My - b MX). r The correlation coefficient or, more formally, the Pearson product-moment correlation coefficient. It is an index of the strength of the relation between two variables, and its values can range from -1 to +1. To compute, find the average cross product of the Zx and Zy scores (r = E Zxi Zyi/N, where i = 1,N), or take the square root of r2 and assign it the same sign as the covariance. r2 The coefficient of determination, or r-squared. It indicates the proportion of criterion variable variance that can be accounted for given knowledge of the predictor variable: To compute, divide model variance by total variance (r2 = VARmodel/VARtotal).
9.3 DETECTING GROUP DIFFERENCES, BINARY PREDICTOR 9.3
129
DETECTING GROUP DIFFERENCES WITH A BINARY PREDICTOR Perhaps one of the most common research questions asked is, do two groups differ? The groups may differ naturally in some way or they may have been formed by random assignment and then treated differently, but in either the correlational or experimental case, the investigator wants to know whether the average of some measure of interest is different for the two groups. Imagine, for example, that 5 of the 10 subjects in the lie detection study had received a drug that made them less physiologically reactive than normal whereas the other 5 had received a placebo. Such a study design could be motivated by the hypothesis that a tranquilizing drug will make lies more difficult to detect. A question like this can be answered easily with only slight modification to the template shown in Fig. 9.2, as you will see in the next exercise. Then, in the next chapter, you will learn how the statistical significance of this information can be assessed. You can then use the template you are about to develop whenever subjects fall into two groups. But is the difference between the means for the two groups statistically significant? The procedure developed here gives results identical with those obtained using Student's t test, a test that was developed early in this century by Gossett (who used the pen name Student). However, as we mentioned in chapter 4 and again in the next chapter, we prefer to emphasize a more general approach appropriate not only for situations in which the historically much-used t test could be applied, but one that is also appropriate for many other situations as well. This general approach uses regression computations to perform what is called a one-way analysis of variance and requires that a subject's group membership (in the present case, whether the subject received a drug or placebo) be treated as a predictor variable. Exactly how a categorical variable like group membership can be coded so that its qualitative information is represented quantitatively is an important topic discussed in some detail in chapter 12. For present purposes, and without further justification, we use a single predictor variable to represent group membership. Arbitrarily, values for this variable are set to 1 for subjects who received the drug and to 0 for subjects who received the placebo. Exercise 9.3 Predicting Lies Detected From Drug Group The template developed during this exercise can be used whenever you want to describe the association between a predictor and a criterion variable. The predictor variable can be binary (indicating membership in one of two groups, for example) or quantitative (like mood score). 1. Begin with the spreadsheet shown in Fig. 9.2. Change the formula for r from the average cross products of the Z scores to the covariance divided by the square root of the product of the X and Y variances. This eliminates the need for the columns containing ZY, Zx, and their cross product. You can erase these columns. 2. Replace the mood scores with drug group scores (and change labels appropriately). Enter 1 for the first five subjects (indicating that they belong to the drug group) and 0 for the last five (indicating that they belong to the placebo group).
130
BIVARIATE RELATIONS: REGRESSION, CORRELATION 3. You are now done. All other formulas carried over from the previous template should be in place and correct. 4. As a final exercise, verify that Equation 9.9 gives the same value for r as Equation 9.11. The value for r-currently displayed is based on Equation 9.11 (r = the XY covariance divided by the square root of the product of the X and Y variances). In another cell, enter a formula based on Equation 9.9 (r= the square root of the quotient resulting from dividing the model variance by total variance). Except for one detail, the two values you just computed should be identical. In what way are they different? Why? At this point, your spreadsheet should look like the one shown in Fig. 9.3. From it you know that knowledge of drug group (drug or placebo) allows you to account for 30% of the variance in number of lies detected. This is a reasonable amount, and a bit higher than the 28% accounted for by mood scores (see Fig. 9.2). You also know that the value of the correlation coefficient describing the association between drug group and number of lies detected is -.55 (rounded to two significant digits). Drug group is a binary categorical variable. When the predictor variable is quantitative, the sign of the correlation coefficient indicates the direction of the relation (a positive sign indicates that increases in the IV tend to be associated with increases in the DV; a negative sign indicates that increases in the IV tend to be associated with decreases in the DV) and the Y intercept and slope have the meanings described in chapter 8 and in the previous section of this chapter. When the predictor variable is categorical, however, the sign of the correlation coefficient and the values for Y intercept and slope need to be interpreted in light of the arbitrary codes assigned to the two levels of the binary variable. In this case the drug group was coded 1 and the placebo group was coded 0. There are only two values for the predictor variables; thus there are only two values for the predicted scores. The predicted score for all subjects in the drug group was 4.2, which is the mean of the observed scores for the five drug group subjects, whereas the predicted score for all subjects in the placebo group was 6.4, which is the mean of the observed scores for the five placebo group subjects (see Fig. 9.3). You may want to ponder why, using a binary predictor variable, predicted values based on the best-fit line are the mean scores for the two groups. For now, however, note that because the lower code (zero for the placebo group) was assigned to the group with the higher mean (M - 6.4), the best-fit line (which you will graph during the next exercise) tilts upward to the left and as a result the correlation coefficient is negative (-0.55). If codes assigning a higher number to the placebo than to the drug group were chosen, the correlation coefficient would have been positive instead. Just as the sign of the regression coefficient depends on the codes chosen for the levels of the binary variable, so do the values computed for the Y intercept and slope. In this case, the Y intercept (or regression constant) is 6.4, which is the mean for the group coded zero (the placebo group). The slope (or regression coefficient) is -2.2, which means that for a unit increase in drug group, the number of lies detected decreases by 2.2. The drug group was coded 1, which is a unit increase from the code for the placebo group, so this means that the mean number of lies for the drug group is 2.2 less than the mean for the placebo group, which it is (6.4 - 4.2 = 2.2). However, because the values computed for a and b are affected by the codes used, the exact nature of the relation between a binary predictor and a quantitative criterion variable is best conveyed by simply reporting the group means in the first place, by saying that the mean number of
131
9.3 DETECTING GROUP DIFFERENCES, BINARY PREDICTOR
lies detected was 4.2 for subjects who had the drug and 6.4 for those who received the placebo. Whether the predictor variable is qualitative or quantitative, r and r2 are
A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
B s 1 2 3 4 5 6 7 8 9 10
Sum=
N= Mean=
a,b= r=
H
Lies Y 3 2 4 6 6 4 5 7 7 9 53 10 5.3 6.4 -0.55 I
y=
Drug X 1 1 1 1 1 0 0 0 0 0 5 10 0.5 -2.2
Y' 4.2 4.2 4.2 4.2 4.2 6.4 6.4 6.4 6.4 6.4 53 10 5.3
K
J
G
F
E
D
C
Y-My -2.3 -3.3 -1.3 0.7 0.7 -1.3 -0.3 1.7 1.7 3.7 0 10
m= Y'-My -1.1 -1.1 -1.1 -1.1 -1.1 1.1 1.1 1.1 1.1 1.1 0 10
L
M
N
1
SStot
SSmod
2
y*y
m*m
e*e
1.21 1.21
1.44 4.84
1.21
0.04
1.21 1.21
3.24
0.5
0.25
3.24
0.5
0.25
1.21
5.76
-0.5
0.25
3 4 5 6 7 8
5.29 10.89 1.69 0.49 0.49 1.69
9 10 11 12
2.89 2.89 13.69
1.21
Sum=
40.1
12.1
N= VAR=
10 4.01
1.21
SD=
2.002
13 14 15 16 17
0.09
2
R=
1.21 1.21 1.21
10
SSerr
e= Y-Y' -1.2 -2.2 -0.2 1.8 1.8 -2.4 -1.4 0.6 0.6 2.6 0 10
x=
SSX
X-Mx
x*x
x*y
0.5 0.5
0.25 0.25
0.5
0.25
-1.15 -1.65 -0.65 0.35 0.35 0.65
1.96
-0.5
0.25
0.36 0.36 6.76
-0.5 -0.5 -0.5
0.25 0.25 0.25
28 10 2.8
0 10
2.5 10
0.25
SSXY
0.15 -0.85 -0.85 -1.85 -5.5
10 -0.55
0.5
0.302
FIG. 9.3. Spreadsheet for predicting lies detected from drug used. Drug is treated as a binary categorical variable: 0 = no drug, 1 = drug.
132
BIVARIATE RELATIONS: REGRESSION, CORRELATION important statistics to report because they indicate the strength of the association. In addition, for a quantitative predictor, the slope of the best-fit line indicates the exact nature of the relation, whereas for a qualitative predictor, the same information is conveyed by group means. In either case, whenever you want to know if one variable affects another, the template developed for the last exercise provides the basic descriptive statistics. For now, we consider cases for which the single independent variable, if qualitative, has only two values or levels (drug vs. placebo group, for example), although later we show how to deal with more than two levels. How those two levels are coded is not especially critical, as the next exercise demonstrates. Exercise 9.4 Codes for a Two-Group Predictor Variable This exercise uses the template developed for the last exercise. Its purpose is to demonstrate some interesting properties of coded variables. 1. Begin with the spreadsheet shown in Fig. 9.3. Arbitrarily we had decided to code the drug group 1 and the placebo 0. Change the codes to -1 for the drug group and +1 for the placebo. How does this affect the values for a, b, and r2? 2. Imagine that the drug variable is quantitative, not qualitative, and that two levels of drug were selected for study. The first five subjects received 4 mg, the last five, 8 mg of the drug. Change the data in column C. What are the values of a, b, and r2 now? 3. Select one other pair of codes (in addition to 1/0 and -1/+1) for the two groups and recompute a, b, and r2. What do you conclude about the effect of the values selected to represent the two groups on the values computed for a, b, and r2? 4. Now turn your attention to Y, the predicted values. How are they affected by different ways of coding drug group? When a single binary predictor variable is used, what will the predicted scores always be?
9.4
GRAPHING THE REGRESSION LINE As mentioned earlier, if the independent variable is quantitative, the regression coefficient has a clear interpretation. It indicates the change in DV units expected for each one unit change in the IV. If the independent variable is qualitative, as in the last two exercises, the interpretation of the slope is less clear. This is because the numeric codes assigned the two categories of a binary nominal variable are arbitrary. Still, graphing the relation between a quantitative criterion variable (like number of lies detected) and a qualitative predictor variable (like drug group) can be meaningful, as the next exercise demonstrates.
9.4 GRAPHING THE REGRESSION LINE
133
Exercise 9.5 Graphing the Regression Line For this exercise, you graph two regression lines. One shows the relation between number of lies detected and mood score, the other shows the relation between number of lies detected and drug group. 1. Graph the lies/mood data given in Fig. 9.2. (This will look like the scattergram shown in Fig. 8.2.) Now, using the parameters given in Fig. 9.2, graph the regression line. If you are using your spreadsheet to draw this "xy" graph, graph the raw data using symbols (this gives the data points) and then graph the predicted scores using lines (this gives the regression line; Excel terms, x = mood scores, a = raw data, b = predicted data). 2. Note the mean value for the mood scores on the X axis. Draw a vertical line through it. Now note the mean value for number of lies detected and draw a horizontal line through it. If you have done this correctly, all three lines (the regression line and the two mean lines) cross at a common point. Why must this be so? 3. Using the graph, estimate about how many more lies are detected if mood increases from 4 to 6. (Draw a vertical line from mood = 4 to the regression line, then draw a horizontal line from that point to the Y axis. Do the same for mood = 6.) If you did not want to use this graphic method, which is only approximate, how could you have computed the value exactly? 4. Now graph the lies/drug data given in Fig. 9.3. (Use 1 to indicate the drug and 0 the placebo group.) In this case the slope is negative. How do you interpret this? 5. What is the predicted number of lies detected when drug = 0? When drug = 1? (Draw a line vertically from drug = 0 to the regression line and then horizontally to the Y axis. Do the same for drug = 1.) How else might you have determined these values? 6. In this case, what concrete meaning can you give the Y intercept and slope? As you can see, whether a predictor variable is quantitative (like mood score) or binary (like drug group), the regression line superimposed on the raw data provides a way to visualize the relation between the independent and dependent variables (see Figs. 9.4 and 9.5). The slope indicates the exact nature of the relation between criterion and predictor variables. It indicates the change in the criterion variable occasioned by a unit change in the predictor. If it is positive, increases in the predictor are associated with increases in the criterion variable, if negative, with decreases in the criterion variable. For a binary predictor, the slope passes through the means for the two groups and indicates the amount of difference between them. No matter whether the predictor is quantitative or qualitative, r2 indicates the strength of the relation between criterion and predictor variables. In other words, it indicates the proportion of variance accounted for when predictions for the dependent variable are made, based not just on the mean, but with knowledge of the value of the independent variable as well.
134
BIVARIATE RELATIONS: REGRESSION, CORRELATION
FIG. 9.4. Regression line for predicting number of lies detected from mood scores: a = 3.024, b = 0.469, r = .529.
FIG. 9.5. Regression line for predicting number of lies detected from drug group: a = 6.40, b = -0.220, r = –.549. On average, 4.2 lies were detected for drug-group subjects compared to 6.4 for placebo-group subjects.
Exercise 9.6 More Accounting for Variance . This exercise provides additional practice in variance accounting. You can easily modify the last spreadsheet in order to do these computations 1. Sixteen infants were recruited for a study. The number of different words each infant spoke at 18 months of age was 32, 27, 48, 34, 33, 30, 39, 23, 24, 25, 36, 31, 19, 28, 32, and 22 for the 1 st through the 16th infant, respectively. The first 7 infants all had no older siblings. The remaining 9 infants (the 8th through the 16th) had 2, 1, 4, 1, 1, 3, 2, 1, and 5 older siblings, respectively. What is the correlation between number of words spoken and number of older siblings? How do you interpret this correlation? How much variance in number of words spoken is accounted for by the number of older siblings?
9.4 GRAPHING THE REGRESSION LINE
135
2. What if you had made a data entry error and had entered 4 instead of 48 for the 3rd subject's number of words? What would r and r2 be then? What if you had entered 77 instead of 22 for the last subject's number of words? Or 222 instead of 22? Or 10 instead of 0 for the 7th subject's number of older siblings? What do you infer about the potential effect of a single incorrect datum? 3. What if you had divided the infants into two groups, those with no older siblings (coded 0) and those with one or more older siblings (coded 1)? What is the correlation between the number of words spoken and the binary code for none versus one or more older siblings? How do you interpret this correlation? How much variance in number of words spoken is accounted for by knowing whether infants had at least one as opposed to no older siblings? 4. How many words are predicted for the 7 infants who had no older siblings? For the 9 who did? Do these values surprise you?
Exercise 9.7 Regression in SPSS In this exercise you will use SPSS to conduct a regression analysis and create a graph of the lies and mood data. 1. Invoke SPSS. Create variables and enter the Lies and Mood data, or copy the data from the spreadsheet you last used in Exercise 9.2. 2. Select Analyze->Regression->Linear from the main menu. Move Lies to the Dependent window and Mood to the Independent(s) window. Click on Save and check the Unstandardized boxes under Predicted Values and Residuals. Click on Continue and then OK. 3. Examine the Model Summary in the output. Do the values for R and R2 agree with your spreadsheet? Now look at the ANOVA box. The Sums of Squares Regression, Residual, and Total should agree with the values from your spreadsheet for SSmod, SSerr, and SStot, respectively. 4. Examine the coefficients in the output. Can you find the values for a and b? 5. Finally return to the Data Editor. In step two you instructed SPSS to create two new variables. The variable pre__1 contains the Y' predicted values and res_1 contains the residuals (Y-Y'). Do these values agree with you spreadsheet? 6. Select Graphs->Scatter from the main menu. Click on Simple and then Define. Move Lies to the Y axis window and Mood to the X axis window. Click on OK. 7. To create a scatter plot and regression line for the lies and mood data, double click on the scatter plot to open the chart editor. Select Chart->Options from the main menu and check Total under Fit Line. 8. Save the lies and mood data. 9. For additional practice you should try running the SPSS Regression procedure and creating scatter plots for the lies and drug data. Remember to save the data in a separate file.
BIVARIATE RELATIONS: REGRESSION, CORRELATION Both the exact nature and the strength of a relation or association are important descriptive topics. In the next chapter we consider a third and different topic: the statistical significance of an observed association. There we ask the key inferential question: Is the association observed between two variables in a sample large enough to merit attention? In other words, is the observed association statistically significant: Is it likely a real effect and not just a chance result?
10
Inferring From a Sample: The FDistribution
In this chapter you wil: 1. Learn how to estimate the population variance from the value computed for the sample. 2. Be introduced to the concept of degrees of freedom. 3. Learn about the F distribution. 4. Learn how to determine whether the proportion of variance accounted for in a sample by a single predictor is statistically significant. 5. Learn how to do a one-way analysis of variance with two independent groups. In chapter 5 you learned how to compute the variance for a sample of scores, and in the last chapter you learned how a portion of that sample variance could be accounted for, using the best prediction equation (in the least-squares sense) and values for a single independent variable. These topics belong to the realm of descriptive statistics. In this chapter, on the other hand, we discuss ways of moving beyond the sample, inferring facts and relations that probably characterize the population from which the sample was drawn. Thus the material presented here, like that in chapter 7, concerns inferential statistics. 10.1
ESTIMATING POPULATION VARIANCE Equation 5.4 in chapter 5 defined the sample variance as the sum of squares divided by the sample size. That is:
Subscripts such as "X" or "Y" or "total" can be added to VAR and SS to identify the group of scores or the sample in question, but in general terms, the variance for a sample of scores is the average squared deviation from the mean for scores in that sample. It is computed by subtracting the sample mean from each score, squaring each difference, summing the squares, and dividing the sum by the
137
138
INFERRING FROM A SAMPLE: THE F DISTRIBUTION number of scores in the sample. This is a perfectly reasonable way to describe the extent to which scores in a sample are dispersed about the mean but as an estimate of the value of the population variance (O2), the formula for the sample variance has one undesirable characteristic. It is biased.
Biased and Unbiased Estimates Not all estimates are biased. For example, imagine that we drew 100 samples and computed the mean for each. Each sample mean would be an unbiased estimate of the population mean. Rarely would a particular sample mean exactly equal the value for the population mean. Some values might be too high, some too low, but overall there should be no pattern to the direction of the errors—for example, no tendency for the sample means to be smaller than the population mean. A statistic that is biased, on the other hand, will tend to give values that are either too low, on average, or else too high: The values will be consistently off in a particular direction. For example, if we computed the variance for each of the same 100 samples, many more of the values computed for the sample variances would underestimate the population variance than would overestimate it. Thus the sample variance is a biased estimate of the population variance because it consistently underestimates the true population value. This makes intuitive sense. When drawing samples from a population, we are likely to miss some of the rarer extreme scores, so it is likely that the variability in the sample will be less than the variability in the population. Statisticians have proved to their satisfaction that the following formula provides an unbiased estimate of the population variance (recall from chapter 5 that the apostrophe or prime after a symbol indicates an estimate):
This formula was first introduced in chapter 5 (Equation 5.5), but without explaining that the purpose of dividing by N –1 was to provide an unbiased estimate. When first encountered, this formula may seem too neat. Why should dividing the sum of the squared deviation scores by N–1i instead of N produce an unbiased estimate? Accept for now that proof is possible (e.g., see Hays, 1981), that the sample variance consistently underestimates the population value, and that in order to provide an unbiased estimate a correction factor is needed, some number that will remove the bias from the sample statistic. This correction factor must be greater than 1 (because the population variance is larger than the sample variance) and must become smaller as the sample size becomes larger (because bias decreases with sample size). Statisticians agree that this correction factor is exactly N divided by N – 1. Hence:
This equation reduces to Equation 10.2 because VAR equals SS divided by N , and because the Ns cancel. In other words:
10.1 ESTIMATING POPULATION VARIANCE
139
Similarly, the sample standard deviation (SD) is a biased estimate of the true population standard deviation (a). Like the sample variance, the sample standard deviation underestimates the true population value. A better estimate of the population standard deviation is the square root of the estimated population variance or the square root of the quotient of the sum of squares divided by N – 1:
Technically, a second correction factor is necessary to make this a truly unbiased estimate. However, for sample sizes larger than 10 the amount of bias is small and thus in the interests of simplicity—and remembering that the analysis of variance is based on variances, not standard deviations—the correction is usually ignored (again, see Hays, 1981). Indeed, many texts ignore this nicety and state without qualification that Equation 10.4 provides an unbiased estimate of a. Exercise 10.1 The Estimated Population Variance The purpose of this exercise is to examine the correction factor, N divided by N– 1, and to provide practice computing the estimated population variance. 1. What is the correction factor for a sample size of 5? 10? 20? 40? 100? 2. Graph sample size (on the X axis) against the correction factor (on the Y axis). Compute the correction factor for sufficient sample sizes in the range 1-200 so that the shape of the curve is accurately portrayed. As a practical matter, beginning with what sample size does it makes sense to ignore the correction? 3. What is the estimated population variance for the number of lies detected (see Exercise 9.5)? 4. What is the estimated population variance for the number of words spoken (see Exercise 9.6)?
Note 10.1 VAR'
SD'
The estimated population variance. It is an unbiased estimate of O2, the true population value. It can be estimated by multiplying the sample variance, VAR, by the quotient of N divided by N–-1, or by dividing the sum of squares by N– 1. Often it is symbolized with a circumflex (a ^ or hat) above O2 or S2. The estimated standard deviation. It is almost an unbiased estimate (especially if N > 10) of a, the true population value. It can be estimated by taking the square root of VAR' , or by taking the square root of the quotient of the sum of squares divided by N– 1. Often it is symbolized with a circumflex (a ^ or hat) above a or S.
140
I
NFERRING FROM A SAMPLE: THE F DISTRIBUTION
Mean Squares and Degrees of Freedom There is another more general way to view the equation defining the estimated population variance:
Here N – 1 is replaced with df, which stands for degrees of freedom. In the analysis of variance tradition, an estimate of variance is usually called a mean square because it is a sum of squares divided by its appropriate degrees of freedom. Hence Equation 10.5 is often written as:
Subscripts such as or Y or total (and, as you will see, model and error) can be added to MS (and SS) to identify a particular variance estimate. But in general, a mean square is a particular sum of squares divided by its appropriate degrees of freedom. Mean squares, and the degrees of freedom used to compute them, are basic to inferential statistics. To begin with, and as described in the next section, the ratio of two mean squares, called the F ratio, plays an important role in the analysis of variance. 10.2
THE F DISTRIBUTION Judging from articles in professional journals, perhaps the sampling distribution behavioral scientists use most often in their day-to-day work is the F distribution. It was developed primarily by R. A. Fisher during the first few decades of the 2Oth century and was named in his honor by another important 20th-century statistician, George Snedecor. The F statistic, or F ratio, is the ratio of two independent variance estimates or mean squares. Both the numerator and the denominator of the F ratio are mean squares, that is, sums of squares divided by their appropriate degrees of freedom. Thus, in general:
Because sums of squares cannot be negative, neither can the F statistic. Its possible values range from zero to infinity. The F distribution is in fact a family of distributions. Each member of the family is characterized by its numerator and denominator degrees of freedom. Thus df might equal 2,8 for one distribution (meaning 2 df associated with the numerator SS and 8 df with the denominator SS) and 6,24 for another. The actual shape of the F distribution depends on the values for the numerator and denominator degrees of freedom (see Fig. 10.1). With two degrees of freedom in the numerator, the distribution looks like a diagonal line that sagged; values are high near zero and steadily decline as F increases. With higher degrees of freedom in the numerator, the distribution rises sharply from F = 0, rapidly reaches a peak, and then falls back more slowly as F increases. And, as you might guess from chapter 7, as the numerator and denominator degrees of freedom
10.2
THE F DlSTRIBUTION
141
FIG. 10.1. F distributions for 2 and 8 (lighter line) and for 6 and 24 (heavier line) degrees of freedom. Critical values (marked with arrows) are 2.51 for F(2,8) and 4.46 for F(6,24).
become extremely large, the shape of the F distribution approaches that of the normal. In practice, null hypotheses are usually formulated so that if the null hypothesis is true then the two independent variance estimates under consideration will be equal. And if two variances are equal, their ratio will be 1, which is the null-hypothesis expected value for the F ratio. Moreover, the mean squares selected for the numerator and the denominator of the F ratio usually require large values (certainly greater than 1) of F to disconfirm the null hypothesis. In other words, in practice the F test is always and inherently onetailed. Although values for the F ratio less than 1 can and do occur, they usually indicate deviant data that likely violate assumptions required for an appropriate use of the F test in the first place. We could overstate the matter as follows: There are two kinds of Fs, big Fs (that allow us to reject a particular null hypothesis) and small Fs (that do not). Of primary interest to the researcher is the region of rejection, that 5% (if alpha = .05) or 1% (if alpha = .01) of the area under the sampling distribution curve that falls in the right-hand tail of the distribution and is demarcated by the critical value of F. If the computed F ratio falls in this area, that is, if the computed F ratio exceeds the critical value, then the null hypothesis is rejected. Thus the result usually desired by researchers is an F ratio big enough to reject the null hypothesis. Critical values for various members of the F distribution family, as defined by various pairs of values for the numerator and denominator degrees of freedom, are given in Table D in the statistical tables appendix. The next exercise gives you practice in using this table. At this point you know enough to answer a simple and common preliminary question. Imagine that you have two sets of scores (e.g., one from a group of subjects who received a drug and one from another group of subjects who received a placebo) and you want to know if the variability of the scores in one
142
INFERRING FROM A SAMPLE: THE F DISTRIBUTION group is significantly different from the variability of the other group. (This is useful to know before we ask whether the groups means differ significantly.) First you would compute estimated variances (mean squares) for each group separately; then you would divide the larger by the smaller, computing an F ratio. According to the null hypothesis, the variances for the two groups are equal (presumably because the groups were sampled from the same and not different populations). If in fact the variances are roughly equal, the F ratio will be small, not much larger than 1. But if it exceeds the critical value, as determined by the numerator and denominator degrees of freedom, then you would reject the null hypothesis. For example, if the estimated variance or mean square (VAR' or MS) is 9.62 for a group containing 16 subjects and 5.17 for a second group containing 19 subjects, the F ratio (with 15 degrees of freedom in the numerator and 18 in the denominator) is 1.86. The critical value for F(15,18) is 2.27 (alpha = .05); thus we would not reject the null hypothesis. (By convention, numbers in parentheses after an F ratio indicate degrees of freedom for the numerator and denominator, respectively.)
Exercise 10.2 Statistical Significance of the F statistic The purpose of this exercise is to provide practice in using Table D in the statistical tables appendix to determine critical values for the F statistic or F ratio. 1. What is the critical value for the F ratio if alpha = .05 and degrees of freedom = 3,24? lf df =6,24? 2. Would you reject the null hypothesis if F(1,8) = 5.24 and alpha = .05? Why or why not? Note that F(1, 8) indicates an F statistic with 1 df in the numerator, 8 in the denominator. 3. If degrees of freedom = 2,30, what is the critical value for alpha = .05? For alpha = .01 ? Why must the latter be a larger number than the former? 4. If your computed F were 4.42 and the degrees of freedom were 1 for the numerator and N - 2 for the denominator, what is the smallest sample size (N) that would allow you to reject the null hypothesis, alpha = .05? 5. Can an F ratio of 3.8 with one degree of freedom in the numerator ever be significant at the .05 level? Why or why not? 6. Reaction time scores (in milliseconds) for subjects who received an experimental drug are 348, 237, 681, 532, 218, 823, 798, 644, 734, 583 and for subjects who received a placebo are 186, 463, 248, 461, 379, 436, 304, 212. Are the variances for the two groups significantly different?
10.3
THE F TEST In the preceding chapter we found that 28% of the sample variance in number of lies detected could be accounted for by knowing subjects' mood scores and 30% could be accounted for by knowing whether subjects were in the drug or placebo group (see Figs. 9.2 and 9.3). That was straightforward description. But how can we decide whether or not the proportion of variance accounted for is statistically significant? For example, if drug has no effect on the number of lies detected in the population from which a sample is drawn (as the null hypothesis claims), how often, just by the luck of the draw, would we select a sample in which we were able to account for 28% of the variance? Is the probability low enough so that we
10.3 THE F TEST
143
can reject the null hypothesis? The statistic used to answer these and similar questions is the F ratio, which was introduced in the previous section (Equation 10.7), and a test that uses the F ratio is called an F test. Recall that in chapter 8 we noted that the total sum of squares could be divided, or partitioned, into two pieces, the sum of squares associated with the model and the error sum of squares (Equation 8.3). Again, this is simply a descriptive statement. But now, given our present inferential concern (Were the 10 subjects in the lie detection study drawn from a population in which drug has no effect on number of lies detected?), it is useful to examine the mean squares, or variance estimates, associated with the model and error sums of squares. As you might guess, the ratio of these two mean squares will be distributed as F if the null hypothesis is true. Thus if the F we compute is large, we have a basis for rejecting the null hypothesis and can conclude that in general, when people who have taken this drug lie, their lies are less easily detected by experts.
Degrees of Freedom for the Total Sum of Squares As you already know, mean squares are sums of squares divided by their appropriate degrees of freedom (Equation 10.6). Determining the correct degrees of freedom for different sums of squares is relatively straightforward and a basic statistical skill you must master. There are two common approaches that can be understood on a relatively nontechnical level. The first emphasizes how many scores are free to vary, whereas the second, which is used somewhat more in this book, emphasizes the number of parameter estimates used in computing the particular sum of squares. For simplicity of presentation and ease of understanding, these two approaches are demonstrated first for total degrees of freedom and then for model and error degrees of freedom, even though it is the model and error mean squares that are needed to compute the F ratio. Assume a sample size of 10. If taking the first approach, we accept the sample size and the sample mean as fixed. If we did not know the actual scores, we could specify any number we like for the first score— in that sense it is free to vary; likewise for the second score, the third, and so forth. However only nine scores are free to vary in this manner. Once the first nine scores are in place, the loth is determined. There is one value, and only one value, that will insure that the mean of the 10 numbers is the value specified initially. Thus, because nine scores are free to vary, there are nine degrees of freedom. If taking the second approach, we note that an error sum of squares is computed by summing the squares of the deviations of the raw scores from the predicted scores. For the total sum of squares, the predicted scores are always the group's mean. Symbolically:
The first element of the deviation (Yi) represents a raw score and because there are 10 of them, we begin with 10 degrees of freedom. From them we subtract the predicted scores. The equation for a predicted score (which first appeared in chap. 5) is
144
___
INFERRING FROM A SAMPLE: THE F DISTRIBUTION
There is just one parameter estimate, a, which for the total sum of squares is the sample mean (a = MY). Thus when computing SStotal, we begin with 10 degrees of freedom and use 1 estimating a parameter, which leaves 9 degrees of freedom. In general,
This formula applies whenever the sum of squares represents deviations from the sample mean. Moreover, the total degrees of freedom, like the total sum of squares, can be partitioned into two pieces. In other words,
But do not overextend this notion. Although the constituent sums of squares add up to the total sum of squares, and although the constituent degrees of freedom add up to the total degrees of freedom, the constituent mean squares do not sum to the total mean square.
Degrees of Freedom for the Error Sum of Squares The error sum of squares is computed by summing the squares of the deviations of the raw scores from the predicted scores. Symbolically:
Again, the first element of the deviation (Yi) represents a raw score and because there are 10 of them, we begin with 10 degrees of freedom. From them we again subtract the predicted scores. In this case, the prediction equation for Yi' is
There are two parameter estimates, a and b, which are computed using regression procedures (Equations 8.7 and 8.8). Thus when computing SSerror, we begin with 10 degrees of freedom and use 2 estimating parameters, which leaves 8 degrees of freedom. In general,
This formula applies whenever a single predictor variable is used. One degree of freedom is "lost" to the regression constant (a), the other to the regression coefficient (b) for the single predictor variable. More generally, the degrees of freedom for error will always be N minus 1 (for the constant) minus the number of predictor variables. Alternatively, and more traditionally, we would accept that there are five scores in each of two groups and the means for the two groups are fixed. If we did not know the actual scores, we could specify any number we like for the first score in the first group— in that sense, it is free to vary— and likewise for the second, third, and fourth, but not the fifth score in the first group. The fifth score is constrained by the mean for the first group. The same logic applies to the second group. Thus, because eight scores are free to vary (four in each group), there are eight degrees of freedom.
THE F TEST
145
Degrees of Freedom for the Model Sum of Squares The model sum of squares is computed by summing the squares of the deviations of the predicted scores from the mean. Symbolically:
In this case, the first element of the deviation (Yi') represents a predicted score, which as we already know (from Equation 10.11) requires two parameter estimates; thus we begin with two degrees of freedom. From the predicted scores we subtract the mean, which requires one estimate (see Equation 10.8). Thus when computing SSmodel, we begin with two degrees of freedom and lose one, which leaves one degree of freedom. In general,
This formula applies whenever a single predictor variable is used. More generally, the degrees of freedom for the model will always be the number of predictor variables. Alternatively, we would accept that there are two groups whose means are fixed, and an overall or grand mean, whose mean is also fixed. If we did not know the actual group means, we could specify any number we like for the first group's mean—in that sense, it is free to vary. The mean for the second group, however, is constrained (or determined) by the grand mean. Thus, because only one score is free to vary, there is only one degree of freedom. For a sample size of 10 with a single predictor variable, we have shown that the total degrees of freedom is 9, the degrees of freedom for error is 8, and the degrees of freedom for the model is 1. Thus for the present example, degrees of freedom for error and for the model add up to the total degrees of freedom, as they should (Equation 10.10).
Model and Error Mean Squares In general terms, a mean square is a sum of squares divided by its appropriate degrees of freedom. For a single predictor variable and a total sample size of N, the formulas for the model and error mean squares are:
Do not confuse the SSmodel and SSerror, which are descriptive statistics and sum to SStotal, with MSmodel and MS error , which are estimates of population parameters and do not sum to MStotal. But how are these estimates of variance used? What, exactly, do they estimate? As it turns out, their use in inferential statistics depends on what they mean when the null hypothesis is true. Let us now consider each in turn. Mean squares are population estimates based on sample data. We know that such estimates are not perfect reflections of a population but reflect some degree
146
INFERRING FROM A SAMPLE: THE F DISTRIBUTION of sampling error. If the null hypothesis is true, MSmodel and MSerror provide two different and independent ways to estimate that sampling error, that is, the population error variance. If the null hypothesis is not true, then the mean square for the model reflects variability between groups in addition to sampling error: MSmodel = effect of predictor variable + sampling error However, if the means for the two groups are equal as claimed by the null hypothesis (i.e., if the effect of the predictor variable is nil), then the model mean square is a pure estimate of sampling error. Again, do not confuse SStotal, which is a descriptive statistic and can be decomposed into sums of squares based on between group and within group deviations, with MSmodel, which is a population estimate and is affected by sampling error as well as by variability associated with the model if any. No matter whether or not the null hypothesis is true, the mean square for error is a pure estimate of sampling error: MSerror = sampling error It is not influenced by the effect of the predictor variable. Subtracting predicted scores from raw scores removes any such effect from the within group deviations used to compute the error mean square. Both MSmodel and MSerror provide independent estimates of sampling variance when the null hypothesis is true, so the ratio of MSmodel to MSerror should be distributed according to the sampling distribution for F, which provides us with a way to evaluate the null hypothesis, as described in the next section. (For a formal development of this argument, see Winer, 1971.)
The F Ratio The F ratio that tests the null hypothesis that claims no effect for the predictor variable (or no difference between group means) is
Theoretically MSmodel should be equal to or larger than MSerror because MSmodel estimates variability associated with the effect of the predictor on the criterion variable, if any, plus sampling error, whereas MSerror estimates only sampling error. Therefore if the null hypothesis is true, F should equal 1. But if it is not true, F should be larger than 1 (although as noted later it sometimes may be less than 1). Values of F that exceed the critical value associated with dfmodel and dferror are grounds for rejecting the null hypothesis. For the lie detection study, one null hypothesis suggests that treatment group (drug versus placebo, represented with a single predictor variable) has no effect on number of lies detected. If this null hypotheses were true, and if the sample data reflected it exactly, then the proportion of criterion variance accounted for by the predictor variable would be zero (and the regression coefficients, correlation coefficients, and r2s would all be zero). Moreover, again if the null hypothesis were true, the F ratio—the ratio of MSmodel to MSerror—would be distributed as F with one and eight degrees of freedom (because N = 10). According to the null hypothesis, MSmodel and the MSerror are equal (they both estimate the same population parameter); thus the expected value for F is 1.
147
10.3 THE F TEST Note 10.2 MSmodel
Mean square for the model. Computed by dividing the sum of squares associated with the model by the degrees of freedom for the model. It is also called mean square between groups or mean square due to regression.
dfmodel
Degrees of freedom for the model. It is equal to the number of predictor variables.
MSerror
Mean square for error. Computed by dividing the error sum of squares by the error degrees of freedom. It is also called the mean square within groups or the residual mean square.
dferror
Degrees of freedom for error. It is equal to N minus 1 (for the regression constant) minus the number of predictor variables.
F ratio
Usually the ratio of MSmodel to MSerror; more generally, the ratio of two variances. If the null hypothesis is true, it will be distributed as F with the degrees of freedom associated with the numerator and denominator sums of squares.
Needless to say, the ratio of MSmodel to MSerror is rarely exactly 1. A computed F may occasionally even be less than 1, which suggests that the F test was probably not appropriate for those data in the first place. However, because the ratio of two theoretically equal variances is distributed as F with the appropriate degrees of freedom, the statistical significance of a computed F ratio can be determined—and if the F ratio is big enough, the null hypothesis can be rejected. In the next two exercises you are asked to use the F test to evaluate, first, the null hypothesis that drug does not affect number of lies detected and, second, the null hypothesis that mood does not affect number of lies detected. Exercise 10.3 The Significance of a Single Binary Predictor The template developed for this exercise allows you to evaluate the statistical significance of a single predictor variable. The predictor variable examined is drug group and the null hypothesis states that drug has no effect on number of lies detected. As you will see, only a few modifications to earlier spreadsheets are required. General Instructions 1. A spreadsheet you developed earlier computed the proportion of variance in number of lies detected that is accounted for by knowing the subject's drug group (see Fig. 9.3). Modify this spreadsheet so that it computes MStotal, MSmodel, MSerror, and the F statistic that evaluates whether the proportion of variance is statistically significant. You may use Fig. 10.2 as a guide. 2. What is the value of the F ratio you computed? What is the critical value for this F (alpha = .05)? Would you reject the null hypothesis? Why or why not? Is the effect of drug treatment (drug or placebo) on the number of lies detected significant at the .05 level or better?
148
INFERRING FROM A SAMPLE: THE F DISTRIBUTION Detailed Instructions 1. Begin with the spreadsheet shown in Fig. 9.3. In order to make room for the inferential statistics, move the summary descriptive statistics from the sumof-squares columns (I-K) to the deviation-score columns (E-G). Specifically, move (use the cut and paste functions, rather than the copy function) the labels and the formulas for N, the variance, the standard deviation, and R2 from block H14–K17 to block D14–G17. This will erase any formulas currently in cells D14-G15. 2. Check to make sure that all function references and formulas are correct. Due to the way spreadsheets execute a move (as opposed to a copy), the count functions in cells E14-G14 will refer to columns I-K, the variance formulas in cells E15-G15 will divide the sums of squares in cells I13-K13 by the Ns now in cells E14–G14, and the formulas for the standard deviation (cell E16), R2 (cell E17), and r (cell B17) will likewise point to the correct cells. Other formulas (predicted values, deviation scores, sums of squares, parameter values) should still be correct from before. 3. Provide labels for the inferential statistics. In column H, label rows 13–16 as indicated:
4. 5. 6. 7. 8.
Label
Row
ss=
13 14 15 16
Meaning
The sum of squares for this column. The degrees of freedom for this SS. df= The mean square (estimated population variance). MS= SD'= The estimated population standard deviation. Enter the correct degrees of freedom for the SStotal, SSmodel, and SSerror in cells I14–K14, respectively. Enter formulas for the mean squares (the sums of squares divided by the appropriate degrees of freedom) in cells I15–K15. Enter the formula for the estimated population standard deviation in cell 116. Enter the label "F=" in cell J17 and enter the formula for the F ratio (the MSmodel divided by the MSerror) in cell K17. What is the value of the F ratio you computed? What is the critical value for this F (alpha = .05)? Would you reject the null hypothesis? Why or why not? Is the effect of drug treatment (drug or placebo) on the number of lies detected significant at the .05 level or better?
At this point your spreadsheet should look like the one shown in Fig. 10.2. You have now produced a template that allows you to determine the significance of a single predictor variable and can easily be modified for other data, for example, the mood score data, as we demonstrate in the next exercise. Finally, in the next section, we discuss how to interpret the results of both the drug group and mood score analyses. Exercise 10.4 The Significance of a Single Quantitative Predictor The template you developed for the last exercise is used for this exercise too. Only the data are different. The predictor variable examined is mood and the null hypothesis suggests that mood has no effect on number of lies detected. 1. Change the data entered in the previous spreadsheet (see Fig. 10.2) from a binary code for drug group to mood scores (the mood scores were last
149
10.3 THE F TEST
shown in Fig. 9.1). All other formulas from the previous spreadsheet should be in place correct. 2. What is the value of the F ratio you computed? What is the critical value for this F (alpha = .05)? Would you reject the null hypothesis? Why or why not? Is the effect of mood on the number of lies detected significant at the .05 level or better?
10.4
THE ANALYSIS OF VARIANCE: TWO INDEPENDENT GROUPS At this point your spreadsheet should look like the one shown in Fig. 10.3. It is worth reflecting for a moment on what these last two exercises have accomplished. The analysis performed for Exercise 10.3 (and shown in Fig. 10.2) is an analysis of variance with two independent groups. This analysis allows you to determine whether the means for two different groups of subjects are sufficiently different so that, in all likelihood, subjects in the two groups were sampled from two different populations, not one. In practical terms, this analysis allows you to determine whether there is a significant difference between the means computed for the two groups. The analysis performed in Exercise 10.4 (and shown in Fig. 10.3) evaluates the significance of a regression or correlation coefficient. This analysis allows you to determine whether the observed association is sufficiently great so that, in all likelihood, subjects were not sampled from a population in which there is no association. In practical terms, this analysis allows you to determine whether the two variables are significantly
D
1 2
s
13 14 15 16 17
Sum= N= Mean= a,b= r=
Lies Y
Drug X
53 10
5 10 0.5 -2.2
5.3 6.4
-0.55
Y'
y=
m-
Y-My
Y'-My
e= Y-Y'
4E–15 10 1.21
0 10 2.8
53
0
N= VAR= SD= R2=
10 4.01 2.002 0.302
K
L
M
N
I
J
1
SStot
SSmod
SSerr
x=
y*y
m*m
e*e
X-Mx
SSX x*x
SSxY
2
40.1 9
12.1 1 12.1
28 8 3.5
0 10
2.5 10
-5.5
0.25
-0.55
F=
3.457
H
13 14 15 16 17
SS= df= MS=
SD'=
4.456 2.111
x*y
10
0.5
FIG. 10.2. Spreadsheet for computing the F ratio that evaluates the effect of drug treatment on number of lies detected. Rows 3-12 are the same as Fig. 9.3 so are not shown.
INFERRING FROM A SAMPLE: THE F DISTRIBUTIONS
150
related. It should now be evident to you that both kinds of questions can be answered with the same analysis. The overarching question concerns the proportion of variance accounted for by a single predictor variable. Is it sufficiently greater than zero so that, in all likelihood, subjects were sampled from a population in which that predictor has an effect, and not sampled from a population in which the predictor has no effect as the null hypothesis claims? In other words, does the predictor variable matter? Can we predict individual outcome significantly better if we know how subjects scored on the predictor variable?
Statistical Significance and Effect Size You computed two F ratios for the last two exercises, one evaluating the contribution of drug group and one evaluating mood. Based on the sample data provided, neither mood nor drug group significantly affected the number of lies experts could detect. Although not finding effects significant may seem disappointing from a research point of view, remember that our example included data for only 10 subjects, which is not very many. Still, the magnitude of the effect was not trivial: In this sample, drug group accounted for 30% and mood scores accounted for 28% of the variance in number of lies detected. Thus the data from the lie detection study rather dramatically demonstrate the difference between real-world significance, or the magnitude of the effect, and statistical significance. In fact, with five additional subjects, both effects would have been statistically significant at the .05 level (assuming that the additional
D
A
1
y=
s
Lies Y
Mood
2
X
Y'
13 14
Sum= N= Mean= a,b=
53 10 5.3 3.024
48.5
53 N= VAR=
r=
0.529
15 16 17
H
I
10 4.85 0.469
SD= R2= K
J
1
SStot
SSmod
SSerr
y*y
m*m
e*e
40.1 9
11.24 1
4.456 2.111
11.24
28.86 8 3.607
F=
3.116
15 16 17
SS= df= MS=
SD'=
e=
Y'-My
Y-Y'
0 10 4.01 2.002
0 10 1.124
0 10 2.886
M
N
0.28
L
2 13 14
m=
Y-M