ISSUES IN THE IDENTIFICATION OF LEARNING DISABILITIES Thomas E. Scruggs and Margo A. Mastropieri ABSTRACT This chapter reviews problems in the identification of learning disabilities, with particular reference to issues involving discrepancy between IQ and achievement as a criterion for definition. Alternatives to present procedures for identification of learning disabilities are described. It is concluded that no presently proposed alternative meets all necessary criteria for identification of learning disabilities, and that radically altering or eliminating present conceptualizations of learning disabilities may be problematic. The major problems of identification of learning disabilities – including overidentification, variability, and specificity – can be addressed, it is suggested, by increasing specificity and consistency of state criteria and strict adherence to identification criteria on the local implementation level. However, further research in alternative methods for identifying learning disabilities is warranted.
ADDRESSING THE PROBLEMS OF IDENTIFICATION OF LEARNING DISABILITIES The phenomenon of learning disabilities (LD) has intrigued and challenged professionals for over a century, since its early descriptions by clinicians of individuals of apparently normal ability, who nevertheless revealed pronounced
Identification and Assessment Advances in Learning and Behavioral Disabilities, Volume 16, 1–36 © 2003 Published by Elsevier Science Ltd. ISSN: 0735-004x/doi:10.1016/S0735-004X(03)16001-6
1
2
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
difficulties with typical academic tasks. Nineteenth-century writers, including Broadbent, Kussmaul and Morgan, identified some of the first cases (Hallahan, 2002). Hinshelwood (1917) described a case of a 10-year-old boy who exhibited severe reading difficulties. Demonstrating his severe problems with reading, Hinshelwood nevertheless recorded that the boy “was apparently bright, and in every respect an intelligent boy . . . his progress in arithmetic has been regarded as quite satisfactory . . . his visual acuity is good” (pp. 46–47; see Hallahan, 2002; Torgesen, 1998, for historical overviews). More recently, Mastropieri and Scruggs (2002) described the case of “Andrew,” a third grader in a middle-class suburban school with severe reading problems. Presented with an age-appropriate text, he read only six words per minute, on a much easier text he read only eight words per minute. A free writing sample revealed pre-school level skills. Nevertheless, on an IQ test, Andrew received a Full Scale score of 104. His perceptual-motor functioning was average; his vocabulary was above average; test scores of listening comprehension and verbal expression test scores were average, as were behavior rating scales. Nevertheless, his reading scores were considerably below average (standard scores of 82 or 74, depending on the test used. Math and spelling scores were similarly low. Andrew and the Hinshelwood case are representative of students with learning disabilities, those students who, although apparently normal otherwise, exhibit unexpected underachievement in specific academic areas, usually reading, writing, and spelling. Since the inclusion of learning disabilities as a category in the Education for all Handicapped Children Act (PL 94-142, 1975), concerns have been expressed over appropriate identification procedures. LD is far from alone in searching for better definitions; in fact, most disability categories face issues in identification. For example, the American Association on Mental Retardation (formerly the American Association for the Study of the Feebleminded) has revised its 1921 definition of mental retardation nine times, in 1933, 1941, 1957, 1959, 1961, 1973, 1977, 1983, and 1992 (Beirne-Smith, Ittenbach & Patton, 2002). Nevertheless, the field of learning disabilities has experienced some very specific problems in identification (Bradley, Danielson & Hallahan, 2002; Kavale & Forness, 1995; Scruggs & Mastropieri, 1994–1995). Problems in identification are not always thought to be subject to simple remediation. As a consequence of these problems, it has sometimes been argued that the category of learning disabilities should be eliminated or significantly altered (Aaron, 1997; Algozzine, 1985; Lyon et al., 2001). As argued by Aaron (1997), . . . the concept of learning disability belongs in the history of science, not at the forefront of contemporary educational practice and research. When the [IQ-achievement] discrepancy
Issues in the Identification of Learning Disabilities
3
formula disappears from the educational scene, so will the concept of LD. After 40 years of wandering in the wilderness of learning disabilities, we are beginning to get a glimpse of the promised land (Aaron, 1997, p. 489; see also Swerling & Sternberg, 1996).
In this chapter, we describe the common characterizations of learning disabilities, and review problems in identification. We will then discuss proposed alternatives to the present procedures for identifying of learning disabilities. Although a number of alternatives have been proposed, we suggest that none of these has been adequately tested; neither does any alternative meet all the necessary criteria for identification of learning disabilities that meets with current conceptualizations. Finally, we will suggest that the problems commonly identified can be addressed by other means, particularly, by including more specific state criteria and carefully and faithfully applying these criteria to identification on the school level. This chapter is based largely on a previous work on issues in identification of learning disabilities (Scruggs & Mastropieri, 2002), in addition to information gained from presentation at and participation in the LD Summit recently held in Washington, DC (Mastropieri, 2001), and recently published summaries of papers and responses presented at that meeting (Bradley, Danielson & Hallahan, 2002).
WHAT ARE THE PROBLEMS IN IDENTIFICATION OF LEARNING DISABILITIES? Since the development of the concept of learning disabilities and its inclusion in federal special education legislation, concerns have been raised about the legitimacy of the category. Although numerous and varied concerns have been raised, they can be generally organized as concerns with: (a) (b) (c) (d) (e) (f) (g)
overidentification of learning disabilities; variability in rates of identification; specificity of populations of students; conceptual considerations; issues involving IQ-achievement discrepancy formulae; early identification of learning disabilities; and problems in local implementation of state and federal criteria.
Too Many Students are Identified as LD Over the past quarter century, the population of individuals identified with learning disabilities has increased about 150% to a level that represents over half
4
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
of all students with disabilities and over 5% of all students in school, a percentage far higher than that found in, for example, mental retardation or serious emotional disturbance (U.S. Department of Education, 2000). The rapid increase in identification rates is frequently the first fact noted in arguments that current definitions are problematic (e.g. Algozzine, 1985; Fuchs, Fuchs & Speece, 2002; Lyon et al., 2001). On one hand, the high identification rates may simply reflect the fact that more students have LD than have other types of disabilities. And, in fact, more individuals are identified with milder forms of disabilities. For example, the number of students identified with mild mental retardation is much higher than the number of students identified with severe mental retardation (Beirne-Smith, Ittenbach & Patton, 2002). As the “mildest” category of disability, it might be expected that it might contain the largest number of students. However, there may be other reasons for the high identification rate. These high identification rates may in fact result from imprecision in federal and state definitions of learning disabilities. However, other researchers have suggested that a confounding of different high-incidence disabilities has resulted in classifying individuals as having learning disabilities, who previously may have been classified as having mental retardation (MacMillan, Siperstein & Gresham, 1996). Wong (1996) suggested that if the concept of learning disabilities has been overgeneralized, perhaps it reflects “teachers’ noble goal of teaching as many problem learners as possible, and not restricting instructional help only to students with learning disabilities” (p. 8; see also Torgesen, 1999, p. 108; Zigmond, 1993). Clearly, learning disabilities is not the only reason for school failure, and overidentification may occur when programs for other underachievers are not available. Regardless of the reason, many consider that prevalence rates for learning disabilities have been “alarmingly high” (Algozzine & Ysseldyke, 1987). Wagner and Garon (1999) shared the view of many when they suggested, “the prevalence of this disability is likely to be closer to 1–3% of school-age children as opposed to recent estimates of 20–30%” (p. 100).
There is Too Much Variability in LD Identification Proportions of individuals identified with learning disabilities have varied considerably across various agencies, including state departments of education and local educational authorities. Finlan (1992) reported considerable variation in state identification rates, with the lowest rate in Georgia (2.10%) and the highest in Rhode Island (8.66%). Coutinho (1995) examined more recent data and reported similar variability, from a low of about 2% (Wisconsin) to a rate of over 7% (Massachusetts) (see also Frankenberger & Fronzaglio, 1991; Lester
Issues in the Identification of Learning Disabilities
5
& Kelman, 1997). Although there has been some debate whether the variability in identification exceeds that of other disability areas (Algozzine & Ysseldyke, 1987; Hallahan, Keller & Ball, 1986), there is little doubt that the variability is considerable. This variability may reflect lack of consistency or precision in identification procedures. Finlan (1992), for example, observed a systematic relation between identification rate and whether or not states employed a specific discrepancy requirement, which relation suggested that identification procedures may be largely dependent on identification policies. Finlan reported that seven of the lowest 10 states in rates of identification employed a specific method of assessing a discrepancy, while only two of the highest 10 states in rates of identification employed a specific discrepancy requirement. This finding reflects the position of Dawes, Faust and Meehl (1989) that more objective procedures in identification may lead to more consistent outcomes. More recently, Lester and Kelman (1997) evaluated state identification rates and concluded that demographic and sociopolitical factors were related to some aspects of state prevalence of learning disabilities, although the same was not true of prevalence of physical disabilities. It does not necessarily follow, however, that there is no reason for actual prevalence rates to vary by geographical region, and that future procedures must lower variability in order for identification procedures to be valid. Characteristics of residents can vary by jurisdiction, and these characteristics can lead to lower vs. higher prevalence of any specific condition. For example, the mean percentage of low-birthweight children is 7.03% nationally, but individual percentages by state range from 4.7 to 15.4%. This observation does not in itself suggest that procedures for identification of low birthweight should be modified so that percentages are the same in each state (see Lester & Kelman, 1997). Nevertheless, variability in identification must be understood far better than it is today, so that judgments can be made regarding whether such variability is the result of variability in the characteristics of state populations, or whether it is the result of inconsistently applied identification procedures. A true and complete understanding of the observed variability in identification can lead directly to proposals for policies to reduce such variability, or to a better understanding of existing variability in learning disabilities.
Learning Disabilities Lack Specificity One of the perennial arguments with the learning disabilities category is that it has been argued that individuals with learning disabilities cannot be reliability distinguished from individuals with generally low achievement (LA) (e.g. Algozzine,
6
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
1985; Ysseldyke, Algozzine, Shinn & McGue, 1982). More specifically, it is argued that students with reading disabilities cannot be distinguished from “garden variety” poor readers (Fletcher et al., 1994; Spear-Swerling, 1999; Wagner & Garon, 1999). Shaywitz, Escobar, Shaywitz, Fletcher and Makuch (1992) suggested that “dyslexia” simply reflects the lower tail of a normal distribution of reading ability, in contrast to the findings of earlier research (Rutter & Yule, 1975). Ysseldyke et al. (1982) compared students who had been identified as having learning disabilities with students who had not been identified but had scored lower than the 25th percentile on achievement tests, and concluded that “there were no psychometric differences in the performances of the two groups of students” (p. 83). Citing this and the results of related investigations, Algozzine (1985) concluded that “the learning disabilities category has outlived its usefulness” (p. 73). In response, some researchers, re-analyzing research in which learning disabilities – low achievement comparisons were made – have concluded that differences are reliable and substantial (e.g. Fuchs, Fuchs, Mathes & Lipsey, 2000; Kavale, Fuchs & Scruggs, 1994). Algozzine, Ysseldyke and McGue (1995) maintained that, even if some differences can be identified, the two groups can benefit from qualitatively similar instructional approaches, and therefore specific identification of learning disabilities is unnecessary (see also Lyon, 2001; Spear-Swerling, 1999). Fletcher et al. (2002) suggested, “the question is not so much whether children defined as IQ discrepant and low achieving are different, but how much they differ and whether the differences are meaningful for research and practice” (p. 205).
Learning Disabilities Lacks Conceptual Clarity It has been argued that problems in identification arise in part from problems in conceptualizing the definition of learning disabilities. Kavale and Forness (2000) evaluated previous definitions and suggested, . . . the failure to produce a unified definition has meant that LD lacks two critical elements: understanding – a clear and unobscured sense of LD – and explanation – a rational exposition of the reasons why a particular student is LD (p. 240).
Problems in identification will continue if there is not general agreement on the concept of learning disabilities (Kavale & Forness, 1995; Kavale, Forness & Lorsbach, 1991). Part of the difficulty arises from the lack of commonly agreed-upon positive measures of learning disability; that is, there is no test for learning disabilities, as there are IQ tests to be used as an important measure for identification of mental retardation. Another source of difficulty is the partial reliance on exclusionary criteria – that is, things that learning disabilities can not be, including mental
Issues in the Identification of Learning Disabilities
7
retardation, emotional disturbance, or cultural disadvantage (Kavale & Forness, 2000).
IQ-Achievement Discrepancies are Problematic IQ-achievement discrepancy criteria in the identification of learning disabilities have been widely employed (Schrag, 2000); however, discrepancy criteria have been criticized by many, for a variety of conceptual and statistical reasons (Aaron, 1997; Mastropieri, 1987; Scruggs, 1987; Spear-Swerling, 1999; Stanovich, 1991). Technical problems include the degree of measurement error associated with discrepancy methods, in that difference scores involve the additive measurement error of both measures. Another problem is the problem of statistical regression (Cone & Wilson, 1981; Shepard, 1980), where individuals who have been identified for low performance on one measure (e.g. academic achievement) can be expected to score higher on a second measure correlated to the first (e.g. IQ). Fletcher et al. (2002), described previously, referencing generally research that has suggested there is considerable overlap between IQ-discrepant samples and low achieving samples, concluded, “consistent with the call of many researchers, the viability of the IQ-discrepancy classification hypothesis must be questioned” (p. 205). Implicit within arguments against use of discrepancy criteria are criticisms of IQ tests. The use of IQ tests has been criticized, and it has been argued that students exhibiting discrepancies do not differ from poor readers with lower IQs on many reading, spelling, language, or memory tests (Siegel, 1999). It has also been suggested that IQ does not predict reading achievement (Fletcher et al., 1998; Vellutino, Scanlon & Lyon, 2000), although this position has been challenged (Naglieri, 2001). Stanovich (1991) argued that discrepancy definitions as applied to reading disability are threatened by findings that literacy development also develops the cognitive skills revealed on aptitude measures such as IQ tests. Some research has suggested that IQ-achievement discrepancies are not predictive of academic growth rates (Lyon, 2001; Vellutino, Scanlon & Lyon, 2000, but see Speece & Case, 2001, for a different perspective). Embedded in this argument is a concern for the value of IQ testing in special education. Fletcher et al. (2002) maintained, “The concept of IQ as it is applied to LD is outmoded and reflects an obsolete practice . . .. IQ tests do not measure aptitude for learning or provide an index of response to intervention” (p. 234). Fletcher et al. (2002) also maintained that IQ score does not appear to be an indicator of the slow learner, and that “there is not natural subdivision that demarcates mental deficiency from LD” (p. 234). Speece (2002) reported on the
8
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
results of a recent survey of members of editorial boards in reading and learning disabilities. Fewer than half of the respondents agreed that IQ should be used as a component of a definition of LD, and none of the respondents felt that IQ was the most important consideration. However, respondents who considered that exclusion clauses were important most often chose mental retardation as a criterion, a category of exceptionality generally identified as having low IQ. According to Kavale and Forness (1995), the major problem with discrepancy is how it has been applied – that is, that “the prominence of discrepancy in conceptualizations of LD has resulted in its reification and deification” (p. 162). In many instances, discrepancy (underachievement) has come to be seen as the same thing as learning disabilities, rather than one possible component of a conceptual understanding of learning disabilities. In fact, there are many reasons why a student may exhibit IQ-achievement discrepancies without having learning disabilities. One of the present authors recalls a new student who was suspected of having a learning disability, based on a substantial discrepancy between IQ and achievement. However, this student was observed to make remarkable progress in reading during the first few weeks of the school year. After an examination of the student’s record, it became clear that the observed IQ-achievement discrepancy was probably the result of chronic school inattendance in previous years. In other instances, discrepancies have been observed in students of presumed socio-cultural disadvantage (e.g. Scruggs & Cohn, 1983). In instances, identification procedures rely on exclusionary considerations to determine that adequate educational opportunity has not been provided, and the characterization of learning disabilities cannot be applied. Criticisms of discrepancy criteria are commonly made Aaron (1997) argued passionately against the use of discrepancy criteria in the identification of learning disabilities. Lyon et al. (2001) reviewed literature relevant to discrepancy procedures and concluded, “The IQ-achievement discrepancy, when employed as the primary criterion for the identification of LD, may well harm more children than it helps” (p. 266; see also Fletcher et al., 2002). Such concerns suggest that other procedures for identification may be preferable.
Students with Learning Disabilities are Not Identified Early Enough Keogh and Becker (1973) many years ago alerted the field to the importance of early identification and treatment and the dangers of misidentification (see also Keogh, 1986). Haring et al. (1992) argued that, if the condition of learning disabilities is defined as a lack of academic progress, how could such a determination can not be made on the preschool level? Because of this, different characterizations should be
Issues in the Identification of Learning Disabilities
9
applied. Others have suggested that it may be better not to wait for failures to occur prior to implementing interventions. Fletcher and Foorman (1994) suggested that learning problems are more easily remediated in the earlier grade levels, and concluded that “the focus should be on prevention and early intervention for children at risk for developing learning difficulties” (p. 187). However, models of identification that emphasize observed academic failure may lose valuable time needed for treatment (Hallahan, Kauffman & Lloyd, 1999). Lyon et al. (2001) argued that the present IQ-achievement discrepancy criterion requires a “wait to fail” model: many children can not be reliability identified as having learning disabilities until they have conclusively “failed” in academic learning. This determination might not be made before third grade because of psychometric limitations in discrepancy criteria (p. 269), and, in fact, a large number of students are identified at about third grade. It is true that grade level discrepancies, which require, e.g. academic functioning two years below grade level may be problematic. For a two-year discrepancy to be observed, for example, it is necessary that students be enrolled in school at least two years; this may in some cases eliminate referral before 3rd grade. However, grade level discrepancies to identify learning disabilities are rarely the dominant consideration in learning disabilities identification today. IQ-achievement discrepancies based on standard scores, which are more generally employed, do not necessarily require long periods of academic failure, as students can exhibit severe discrepancies between ability and standardized academic achievement scores early in the 1st grade. For example, Horn and O’Donnell (1985) found that discrepancy scores, even in the first few weeks of 1st grade, were better predictors of future learning disabilities classification than was a more general low-achievement criterion. One reason this “wait to fail” argument against discrepancy criteria has been so generally used may be that many students in fact may not be identified until the third grade, and valuable instruction time is consequently lost. However, the reluctance to identify students with learning disabilities may not be due to the discrepancy criteria, as much as simply a general reluctance to refer students to special education at very early ages. This is unfortunate, however, since early treatment can be effective (Mastropieri, 1988).
Identification Procedures are Not Implemented Appropriately on the Local Level A critical component of the identification process, often lost in purely conceptual or technical analyses of the issues, is the individual or group judgment of professionals who are part of the referral and eligibility process. These judgments can be critical in identifying students who might otherwise be missed, and can help avoid the false
10
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
negatives or false positives that may result from blindly applying the quantitative criteria. MacMillan, Gresham and Bocian (1998) maintained, “the assignment of responsibility to a committee clearly conveys the desire to receive input from varied perspectives and not rely exclusively on test scores” (p. 322). On the other hand, however, more systematic “actuarial” procedures can result in much more consistent and systematic identification methods, and can be key factors in reducing variability in identification (Dawes, Faust & Meehl, 1989; Ysseldyke, Algozzine, Richey & Graden, 1982). In response to such issues, Meehl (1973) asked, “When should we use our heads instead of the formula?” (p. 81). Clearly there is a need for both judgment and objective measures, but care must be applied. At present, it appears that the identification process is employed less systematically than should be desirable, and that perhaps the process itself contributes to overidentification and inappropriate variability. Gresham (2001) described the identification process as involving inconsistent criteria, including local criteria for school failure and national norms for assessment. Decisions are frequently based upon the degree to which the student could be expected to profit from special education, rather than on established criteria. Investigations of school practices have revealed that precise identification criteria are often not employed. Gottlieb, Alter, Gottlieb and Wishner (1994) reported that a sample of urban students identified with learning disabilities had an average IQ of 81.4, and concluded, “these children today are classified as learning disabled, when in fact most are not.” MacMillan et al. (1998), using a sample from California, concluded that fewer than half of the students identified as having learning disabilities met state requirements, and concluded that “public school practices for diagnosing children with learning disabilities bear little resemblance to what is prescribed in federal and state regulations” (p. 123). In an earlier investigation, MacMillan et al. (1996) had identified a number of students who had been classified as having learning disabilities, with IQ scores lower than 75, and as low as 58. McLeskey and Waldron (1990) examined the records of 1,742 Indiana students referred for evaluation for learning disabilities eligibility, and reported that students identified as having learning disabilities (52%) differed markedly from students referred but not identified (48%). However, over one third of students identified as having learning disabilities did not meet existing state criteria. MacMillan and Siperstein (2002) reviewed relevant literature and concluded: It is evident that the “concept” of LD used by the schools deviates markedly from the original concept of LD articulated in authoritative defnitions. We have no doubt the SI [school identified] LD population reflects a group of children who do, in fact, need assistance; however, among the children identified as LD by the schools are subsets never considered in previous descriptions of LD that acknowledged the heterogeneity present in the original conception of LD (problems
Issues in the Identification of Learning Disabilities
11
in reading, writing, mathematics, verbal expression, etc.). Today we find children classified as LD who would more appropriately be classified as MR or ED if diagnostic criteria were applied rigorously. As long as the LD category absorbs children with IQ scores in the 70–85 range, as well as those with scores below 70, we will never clean up the LD category (p. 319).
Although identification procedures may be more carefully applied in different regions (see Wilson, 1985, for a more positive example), evidence to date suggests that a very substantial source of overidentification of learning disabilities is the application (or misapplication) of state-established criteria at the local level. These data suggest that numbers of students identified could be reduced by as much as one-third or more simply by consistently and systematically applying state criteria at the local level.
HOW CAN WE CHARACTERIZE LEARNING DISABILITIES? Problems in identification have led some to argue that learning disabilities does not exist as a viable condition. Some have argued that learning disabilities do not really exist, but rather have been socially or organizationally constructed, to serve the interests of certain aspects of society rather than children (Christensen, 1999; Coles, 1987; Skrtic, 1999; Sleeter, 1995; see Kavale & Forness, 1998, for a review). Others have suggested the concept is not useful, in that students characterized as having learning disabilities are not distinguishable in any meaningful way from other poor readers or low achievers not characterized as having learning disabilities (Aaron, 1997; Fletcher et al., 1994; Shaywitz, Fletcher et al., 1992; Ysseldyke et al., 1982). These issues have led researchers in two different directions over the years (Lester & Kelman, 1997). Some, on the one hand, have suggested that improvements in precision of identification procedures will lead to resolution of these issues, and support treatment of those who “truly” have learning disabilities. For example, Reynolds (1984) argued: The tremendous disparities in measurement models adopted in the various states in their respective learning disability guidelines and the varying levels of expertise are obvious, major factors contributing to the difference in the proportion of children served as LD in the various states. Lack of a specific definition, improper or lack of application of the severe discrepancy criterion, and the failure to develop appropriate mathematical models . . . are the primary, certainly interrelated causes of these disparities (p. 455; see also Keogh, 1994).
On the other hand, others have suggested that the concept of learning disabilities has little or no practical utility and should be abandoned:
12
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI It is time to quit viewing eligibility decision making as a technical problem. It means putting an end to efforts to try to find better ways of defining concepts and conditions that cannot be defined and may not exist (Algozzine & Korinek, 1985, pp. 392–393).
Regardless, any accounting of the existence of learning disabilities, positive or negative, must in some way consider the existence of descriptive reports dating back to the 19th century or earlier (see Anderson & Meier-Hedde, 2001; Berlin, 1887; Dejerine, 1892; Hinshelwood, 1896; Kussmaul, 1877), the proliferation of professional and advocacy organizations, such as the Council for Learning Disabilities, the Division for Learning Disabilities of the Council for Exceptional Children, and the Learning Disabilities Association of America, and the thousands of service providers who not only testify to the existence of learning disabilities from their own experience, but who also advocate for improved services (see Scruggs & Mastropieri, 1988). Although issues in identification are far from resolved, a consistent set of observations characterizing learning disabilities has been reported over the years. Wong (1996) suggested, . . . those very characteristics observed by parents, educators, psychologists, and medical professionals about children with learning disabilities in 1963 are the very same characteristics that we see today in children, adolescents, and adults with learning disabilities! (Wong, 1996, p. 22).
As an example, Mastropieri and Scruggs (2002) described the case of “Andrew,” a middle third-grader in a middle-class, suburban school, referred to previously. His reading rate was painfully slow. For a “free writing” activity, he drew a picture which seems to be a comet, and wrote below the picture the single sentence, “It vush the oue wun” [it was the only one]. Psycho-educational testing revealed considerable variability: a Full-Scale IQ of 104, but a reading standard score of 72, with similar deficits in math and spelling. His vocabulary is above average; his listening comprehension and verbal expression scores are almost exactly average. His teachers described him as a good-natured student with good verbal ability, but who exhibited significant problems in reading and math concepts. As such, Andrew provides a classic example of a student who exhibits many of the characteristics considered “typical” of learning disabilities: In the case of Andrew, failure of this alert, cooperative third grader to perform adequately in academic areas was painfully obvious, as was his need for special help. Analysis of his test scores revealed that he exhibited discrepancies in reading of 22 or 30 points, depending on which measure is used, 15 points in spelling, and 31 points in math. His mental ability was normal, as were his vision, hearing, and motor abilities. He was not deprived of necessary cultural exposure, and had received normal educational experiences. Are such considerations inadequate or misguided in identifying Andrew as having a learning disability? We think not . . . (Mastropieri & Scruggs, 2002, p. 453).
Issues in the Identification of Learning Disabilities
13
Kavale and Reese (1991) surveyed 547 teachers of students with learning disabilities in the State of Iowa, and found substantial agreement on their conceptualizations of the nature of learning disabilities. Over 80% of the teachers agreed that the condition of learning disabilities was associated with the following: (a) an ability-achievement discrepancy; (b) intra-individual differences resulting in learning strengths as well as learning weaknesses; (c) academic strengths as well as academic weaknesses; (d) a processing deficit that interferes with learning ability; (e) average or above intelligence; (f) a need for special materials and instructional techniques; and (g) the ability to learn at a different rate than students with mental retardation (pp. 146–147). These statements correspond closely with the three components of learning disabilities consistently mentioned over the past 30 years: “(1) unexpected low achievement relative to aptitude or ability; (2) deficits and uneven profiles in specific perceptual or cognitive processes; and (3) evidence of within-child, presumably causal, neurological condition(s)” (Keogh, 1994, p. 16). However, Keogh acknowledged, “although they may have validity on a construct level, these definitional criteria present serious problems of measurement when making diagnostic decisions or assigning individuals to classes” (p. 16). Wong (1996) provided an overview of many commonly reported features of learning disabilities, including the following: Learning disabilities are associated with unexpected underachievement. This unexpected nature of learning disabilities is a commonly cited characteristic (Keogh, 1994). It is not completely separate from other considerations, because the unexpected or unanticipated nature of learning disabilities has much to do with the concept of deficits in cognitive processing in spite of normal intelligence. Learning problems associated with, e.g. low intelligence, sensory impairments, or cultural disadvantage, in contrast, are “expected,” and provide a direct conceptual contrast with learning disabilities (Kavale & Forness, 1995). The idea of underachievement in spite of normal general ability has been an essential component of the concept of learning disabilities since the earliest reports of over a century ago. Morgan (1896, p. 1378) described a case of “word-blindness” in a 14-year-old boy with severe reading problems as being “bright and of average intelligence in conversation.” Hinshelwood’s (1917) 10-year-old boy with severe reading problems was similarly described as intelligent. More recently, Mastropieri
14
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
and Scruggs’ (2002) case of a student with average abilities in intelligence and general language skills, who nevertheless exhibited very considerable difficulty in math and literacy tasks, provides an example of this unexpected underachievement. That this low level of achievement was not expected from the student’s known intellectual or linguistic functioning is integrally linked to a general understanding of learning disabilities. Learning disabilities are multifaceted in nature, in that they may be manifest in a variety of areas of academic and/or psychological functioning. Although most students identified as having learning disabilities exhibit problems in the literacy areas of reading, spelling, and written language, and for which considerable comorbidity has been identified (Beitchman, Cantwell, Forness, Kavale & Kauffman, 1998), specific problems in mathematics are often identified (American Psychiatric Association, 2000; Baroody & Ginsburg, 1991; Fleischner, 1994; Kosc, 1981; McLeskey & Waldron, 1990). In some cases these problems are may be associated with problems in reading or spelling (Scruggs & Mastropieri, 1986); in other cases, there may be no such association (Rourke, 1989; Strang & Rourke, 1983). In addition, problems in more than one academic area may be associated with deficits in specific cognitive processes, such as memory (Cooney & Swanson, 1987; Siegel & Ryan, 1988; Swanson, 1993) or attention (Hallahan & Cottone, 1997). Even among students who exhibit reading disabilities, it is unclear whether observed deficits in phonological awareness (Torgesen, 1999) underlie all reading disorders, and are also responsible for the deficits in reading comprehension that are so common in this population (Mastropieri & Scruggs, 1997; Sternberg, 1999). A multivariate analysis of a large data set (Kavale & Nye, 1985–1986, 1991), revealed that learning disabilities is most consistently related to deficits in reading and math, as related to deficits in linguistic (e.g. semantic, syntactic, phonological), neuropsychological (e.g. selective attention, memory, cognitive style), and social/behavioral functioning (e.g. interpersonal perception, intrapersonal perception). These results suggest that learning disabilities are multifaceted, rather than unitary in nature. In response to this multifaceted nature of learning disabilities, attempts have been made to identify “subtypes” of learning disabilities (e.g. Lyon, 1985; McKinney, Short & Feagans, 1985). However, general agreement has not been reached on the nature of learning disabilities subtypes, or how they should be identified (Kavale & Forness, 1987). Learning disabilities involve intraindividual differences or deficits, and result in uneven profiles in specific perceptual or cognitive processes. Individuals with learning disabilities are frequently observed to have relative patterns of strengths and weaknesses (Keogh, 1994). Kirk (1971) described these patterns
Issues in the Identification of Learning Disabilities
15
as intraindividual differences, while Gallagher (1966) referred to them as “imbalances.” These intraindividual differences are not necessarily revealed in large group summaries of subtype scores (e.g. WISC profiles, Kavale & Forness, 1984), since group averages reduce overall intraindividual differences. However, researchers and practitioners have frequently noted relative strengths and weaknesses within the patterns of functioning of individuals with learning disabilities (Wong, 1996). Learning disabilities are associated with within-child, presumably neurological conditions. As such, learning disabilities are not thought to be primarily due to sensory, motor, or intellectual deficits, or cultural disadvantage. This characteristic is included in many definitions of learning disabilities (Kavale & Forness, 2000), and has contributed both to conceptual models of learning disabilities, and to the most severe criticisms of the concept of learning disabilities. Cognitive deficits of known neurological origin have been difficult to measure directly, and problems of reliability and validity of psychological “process” measures called into question the issue of neurological functioning (e.g. Larsen & Hammill, 1975). Mercer, Forgnone and Wolking (1976) suggested, “due to the vague nature of the concept of a process disability, a description of it for the purpose of analyzing definitions of LD is subject to criticism” (p. 378). However, a consideration of problems in cognitive processing was helpful in redirecting previous assumptions that learning disabilities generally reflect visual perceptual processing difficulties, and require perceptual motor training (Kavale & Mattson, 1983). Modern techniques, including functional magnetic imaging (fMRI) techniques, have provided support for a neuropsychological basis for learning disabilities (Hynd, Clinton & Hiemenz, 1999), and that “both anatomical and physiological signatures of dyslexia exist,” for example, in decreased activation of the left temporoparietal and superior temporal cortex during phonological processing (Rumsey, 1996, p. 72). In addition, research from autopsy studies (Galaburda, 1991), positon emission tomography (PET) scans (Wood, Flowers, Buchsbaum & Tallal, 1991) and genetic studies (Olson, 1999) have tended to support these findings. Hynd et al. (1999) suggested, “learning disabilities are most appropriately viewed from a neuropsychological perspective” (p. 60). Whether or not such a characteristic is a useful component of a definition of learning disabilities (Spear-Swerling, 1999), the idea that learning disabilities involves neuropsychological dysfunction – and which may be manifest as disorders of cognitive processes – has been prominent in definitions of learning disabilities. Further, processing deficits in students with learning disabilities have been observed in measures of cognitive processing, including attention, memory, and linguistic processes (Hallahan, Kauffman & Lloyd, 1999).
16
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
WHAT ALTERNATIVE PROCEDURES FOR IDENTIFICATION HAVE BEEN PROPOSED? Proposed Alternatives Issues in the identification of learning disabilities have tended toward two very different perspectives. On the one hand are those who have argued that concept of learning disabilities is problematic, and should be abandoned entirely. Others, on the other hand, acknowledge the problems in identification, but nevertheless support the overall concept of learning disabilities. Thirty years ago, McCarthy (1971) wrote: The most important decision you will make is that of definition – because your definition will dictate for you the terminology to be used in your program, the prevalence figure, your selection criteria, the characteristics of your population, and the appropriate remedial procedures (p. 14).
Issues of definition, identification, and assessment are of critical importance to the field of learning disabilities. Several alternatives to the present identification procedures have been proposed, many of which overlap in some of their considerations. These alternative identification procedures include the following: The Double Deficit Wolf and Bowers (1999) proposed criteria for identification of learning disabilities that were based on observed deficits on phonological analysis tasks and rapid continuous naming of digits and letters. These two criteria, referred to as a double deficit, have also been seen to discriminate between students with reading disabilities and normally achieving readers in the early grades. Allor (2002) reviewed 16 studies of phonemic awareness and rapid naming conducted between 1990 and 1997, and concluded that performance on each of these two tasks contributes uniquely to development of word reading. Continued efforts to evaluate the double-deficit hypothesis were recommended. However, Ackerman, Holloway, Youngdahl and Dykman (2001) investigated a sample of 101 elementary school children with and without learning disabilities, and found that these children differed on a number of tasks in addition to the double deficit criteria, including orthographic tasks, attention, arithmetic achievement, and most WISC-III factors. They reported that students who met double deficit criteria were no more limited in reading and spelling than those with a single deficit in phonological analysis. Phonological Process Core Difference Model Stanovich (1988) suggested that differences in phonological processes could discriminate between “dyslexic” and “garden-variety poor readers,” particularly
Issues in the Identification of Learning Disabilities
17
during the early grade levels. Torgesen and Wagner (1998), similarly, and have proposed tests of phonological awareness, rapid automatic naming, and verbal short-term memory in the identification of reading disabilities at the early grade levels (Wagner, Torgesen & Rashotte, 1999). Fletcher, Francis, Shaywitz, Lyon, Foorman, Stuebing and Shaywitz (1998), suggested that learning disabilities classification could be considered for any student failing to reach the 25th percentile for specific reading skills, including core reading process measures. They suggested that this could result in higher numbers identified, but may have greater relevance than present policy-based decisions which, in their view, merely fulfill a “gatekeeping function” (p. 199), that is, to reduce number of referrals. Torgesen (2002) recommended a procedure that involved process-marker variables for early identification, and response to treatment (discussed subsequently) variables for later diagnosis. According to this model, all students in early grades exhibiting problems with phonological awareness and rapid automatic naming would receive special instruction and periodic assessment of progress to determine the need for continued specialized interventions. If problems persisted through 2nd or 3rd grade (or higher), students could be identified as having learning disabilities. Chronological Age Definitions Siegel (1989) suggested that IQ is not relevant to the concept of learning disabilities. She maintained that the influence of IQ and achievement is bidirectional, in that lack of achievement may in time exert a negative influence on IQ. She suggested that students should be identified as having learning disabilities if they score below age expectancy in achievement (particularly, in reading of pseudowords) and are not found to be mentally retarded. Lyon and Fletcher (2001) also suggested that discrepancy criteria be abandoned, and should be replaced by comparing achievement in academic areas with age and grade levels. Bayesian Procedures A number of years ago, Alley, Deshler and Warner (1979) proposed a method for identification based upon earlier models proposed by Wissink, Kass and Ferrell (1975). Bayes’ formula combines prior information with current information to determine the probability of learning disabilities. General education teachers complete a checklist for students who are demonstrating learning problems. A numerical weight is assigned to each factor on the checklist, the weight depending on each factor’s odds of defining learning disability. A formula then combines the probability data. For example, although reading decoding considered independently had a probability of only 0.21 of prediction of learning disabilities, decoding in conjunction with reading recognition in conjunction with detecting
18
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
spelling errors in conjunction with problems with math algorithms resulted in a 0.96 probability of prediction of learning disabilities. It was suggested that additional testing be done to verify these conclusions. Component Model of Reading Disability Aaron (1997) proposed a model of identification that was “not intended to classify, categorize, or label poor readers” (p. 476). Aaron suggested that students be given standardized tests of listening comprehension and reading comprehension. Students who score in the average or high range on listening comprehension but fail to achieve at this level in reading comprehension “are usually hampered by poor decoding skills” (p. 477), which could be confirmed on a measure or reading decoding. Similarly, students can be evaluated for weak comprehension skills, or weakness in both word attack and comprehension. Programs, then, can be implemented that emphasize decoding, comprehension, or both. Neuropsychological Assessment Rourke (1993) described a battery of neuropsychological assessments for the identification of learning disabilities, including tactile-perceptual tests, visualperceptual tests, auditory-perceptual and language related tests. He suggested that these measures could also discriminate among subtypes of learning disabilities, such as nonverbal learning disabilities. Assessment of Cognitive Processing Several researchers proposed using tests of cognitive processing for providing direct evidence for identification of learning disabilities. Naglieri (2001) described the Cognitive Assessment System (Naglieri & Das, 1997), and Swanson (1993) described use of the Swanson Cognitive Processing Test as a dynamic assessment measure. Each of these measures is intended to identify individual differences in information processing, and could be potentially useful in the identification of learning disabilities. Operational Interpretation Kavale and Forness (1995, 2001) proposed a five level process of identification, that included: (1) underachievement or discrepancy to be used as a necessary but not sufficient first criterion; (2) significant deficits in basic skills, focusing specifically on the major academic areas of reading, writing, language, and math; (3) deficits in the efficiency of learning, including strategy use and learning rate;
Issues in the Identification of Learning Disabilities
19
(4) psychological process deficits, including attention, memory, linguistic processing, and metacognition; (5) an exclusionary clause, ruling out alternative causes of learning failure, such as mental retardation, sensory impairment, emotional disturbance, or inadequate instruction. Only when all five operational criteria are met should students be identified as having learning disabilities. Failure to respond to validated treatment protocols, or dual discrepancy criteria. One of the major problems in identification of learning disabilities is in discrimination between constitutional and experiential deficits. Berninger and Abbott (1994) argued that sufficient “opportunities to learn” are often assumed rather than demonstrated, and proposed development of validated protocols of which treatment approaches work best for which learning characteristics (based upon static assessment of learning in 11 domains). When this information is available, learning disabilities can be diagnosed on the basis of treatment nonresponding; that is, when a student fails to make expected gains in spite of educational treatments that have demonstrated positive results for similar learners. Fuchs and Fuchs (1998; see also Speece & Case, 2001) suggested that lower academic functioning could be evaluated dynamically, through identification of deficits in level and slope of learning, as evaluated by curriculum-based measurement (CBM) procedures. Resistance to instruction would be demonstrated by slopes on probes of academic skills, including reading and math. Speece and Case (2001) implemented these procedures on students in the area of reading, grades K-2, and identified samples that differed in some respects (e.g. they were lower in age and mean IQ) from students identified according to traditional discrepancy formulas. Al-Otaiba (2000) suggested, given widespread and converging evidence that phonological processing deficits often lead to learning disabilities, unresponsiveness to effective treatment protocols can provide an alternative to discrepancy-based formulae currently used to identify students with learning disabilities (p. 12; see also Al-Otaiba, this volume; Berninger & Abbott, 1994; Fuchs & Fuchs, 1998; Vellutino et al., 1996).
Fuchs, Fuchs and Speece (2002) referred to unsatisfactory level of performance in addition to inadequate rate of growth as a “dual discrepancy” (p. 34). They described in detail the “treatment validity model” and how it could be used in the identification process. In Phase 1, the regular classroom environment is assessed to determine whether it is sufficiently supportive to warrant identification of individual students as having learning disabilities. If this is not the case, the intervention is conducted at the classroom level to increase the student academic
20
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
growth rate to a level comparable with the school, district, or nation. If classroom instruction is considered appropriately supportive, Phase II assessment is used to identify students with dual discrepancies, that is, students who are functioning dramatically below peers in level and slope of performance on, e.g. reading rate. In Phase III assessment, a determination is made whether adaptations in the general education classroom can produce acceptable learning for individual students. If these adaptations are unsuccessful, CBM is used, in Phase IV, to determine whether learning disability classification and special education placement is effective for a given student. If not, labeling and placement in special education is not considered justifiable for that student. Recently, representatives of the National Center for Learning Disabilities (NCLD), the National Association of School Psychologists (NASP), and the International Reading Association (IRA) proposed a similar approach, applying CBM to identify students with learning disabilities (Horowitz, Lichtenstein & Roller, 2002). They recommended that students at risk for academic failure should be provided with a “research-based, general education intervention” (p. 2) for 8–10 weeks, monitored by CBM. Students who do not display appropriate gains in slope and level of learning progress on CBM measures are considered potential candidates for special education. This intervention process is expected to eliminate the necessity of IQ-achievement discrepancies. It was also suggested that “at risk” status could be determined by failure to meet a 20th percentile cutoff on standardized, norm-referenced academic measures, and that failure to make “meaningful gains” could be determined by failure to achieve growth at the 20th percentile or higher on CBM measures. Horowitz et al. suggested that there is “sufficient evidence” (p. 3) to suggest that these procedures could be generally implemented in school settings, and that they would have substantial impact. It was argued, rather implausibly, that these procedures would not change the numbers of children served as LD, although referrals to special education would be reduced by almost one-half (p. 3). However, direct empirical tests of this specific model appear to be lacking at present. Published research applications would be useful in determining whether this model should replace current procedures. Al-Otaiba and Fuchs (2002) recently reviewed 23 research reports on the characteristics of students who are unresponsive to early literacy intervention, and found that most unresponsive students were characterized by deficits in phonological awareness. Other characteristics identified less consistently included deficits in phonological retrieval or encoding, verbal ability, behavior problems, or developmental delays. Al-Otaiba and Fuchs suggested that future research efforts should attempt to address a common definition of “treatment unresponsiveness.” For example, Good, Simmons and Smith (1998) suggested reading fluency
Issues in the Identification of Learning Disabilities
21
below 40 words per minute at the end of first grade, while Torgesen (2000) suggested the 30th percentile on standardized reading tests. Al-Otaiba and Fuchs also suggested that more study be given to characteristics such as phonological memory and low IQ, and to the training and fidelity of treatment implementation of trainers. Finally, they highlighted the difficulty in describing a “typical” treatment-non-responder, due to the considerable variation both between and within treatment-non-responders: Our review of the non-responder literature suggests that it may be difficult to characterize a “typical” non-responder because: (a) she or he is likely to have a relatively complex profile of strengths and weaknesses; and (b) her or his learner profile will probably be different from that of other non-responders . . . In short, as research on treatment non-responsiveness continues, we anticipate that unresponsive students will be characterized by considerable intra- and interindividual variation, which would seem to defy precise applications of treatments to learner strengths and weaknesses (Al-Otaiba & Fuchs, 2002, p. 313, see also Al-Otaiba, this volume).
At present, the Bush administration is promoting this type of approach to identification to learning disabilities, in place of identification criteria which include a discrepancy component (Al-Otaiba & Fuchs, 2002). Recently, in a report sponsored by the U.S. Department of Education, Office of Special Education Programs (2002), it was recommended that discrepancy criteria be eliminated from the identification of learning disabilities, that renewed attention be placed on early identification of learning disabilities, and that alternative procedures for identification be considered. One highly recommended alternative was the response-to-intervention approach, where general education teachers implement scientifically-based practices and use curriculum based measurement to document student progress on a regular basis. Students who prove to be “treatment resisters” may be eligible for more intensive interventions, or referral to special education. It seems possible that criteria such as these may be employed in the near future. How Should Alternative Approaches be Evaluated? No alternative procedure for identification of learning disabilities to date has achieved general acceptance. It is possible, however, to specify criteria to be met in any valid, generally accepted identification procedure. These could include the following: Does the alternative identification procedure address the multifaceted nature of learning disabilities? Is it appropriate for identifying disabilities in reading comprehension as well as decoding skill areas, and other possible areas, including for example math, writing, and spelling; or memory, attention, and study/organizational skills? If learning disabilities is a multifaceted condition,
22
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
measures of a single area of functioning (e.g. phonological core processes) will be of limited use for identification of all learning disabilities. Is the alternative appropriate across the age spectrum of students with learning disabilities? That is, although early identification is of great importance, can the procedures also be used (or modified) to identify students at other age levels? Alternatives that focus entirely on early identification of learning disabilities must assume either that all students can be identified early and remediated, or else that no cases of learning disabilities will emerge after the primary grades. Does the alternative identification procedure possess technical adequacy? That is, have reliability, validity, or administration issues been addressed adequately? No procedure, however conceptually adequate, is of value if it cannot be shown to be both reliable and valid. Further, conceptually adequate procedures are of little value if they cannot be implemented faithfully on a large-scale basis. Will the alternative procedure reduce overidentification of learning disabilities from present procedures? Since overidentification is commonly cited as a major problem of learning disabilities, any alternative that does not reduce the proportion of students identified will not be an improvement, according to this commonly expressed concern. Will the alternative procedure reduce inappropriate variability in rates of identification across both state and local educational authorities? If the alternative procedure does not meet this criterion, it will not be an improvement in this regard. Will the alternative procedure be faithful to present and traditional conceptualizations of learning disabilities? If an alternative procedure identifies populations that are different from those commonly regarded as having learning disabilities, it can not be considered an improvement over present procedures for identifying learning disabilities. For example, many students with mental retardation do not perform well on tests of phonemic awareness, and many do not show response to instruction on a level with normally-achieving peers. If an alternative procedure identifies students with mental retardation as having learning disabilities, then the present conceptualization of learning disabilities will have changed appreciably. Will the alternative procedure produce fewer false positives (identifying students who do not “really” have learning disabilities) or false negatives (failing to identify students who “really” do have learning disabilities)? If not, it will not be an improvement over present methods of identification.
Although many concerns have been expressed regarding existing practice in identification of learning disabilities, there is to date no alternative to existing practice that has general acceptance. Clearly, any modification in present identification
Issues in the Identification of Learning Disabilities
23
procedures must address the criticisms that have been previously been raised. For example, any substantial study of identification procedures must address the issue of across-state and within-state variability in identification rates. With particular reference to the “resistance to validated treatments” approach, several questions remain unanswered. It is unquestionably true that empirically validated treatments ought to be implemented generally (Mastropieri & Scruggs, 2000), and that curriculum-based measurement (CBM) is a most effective method of monitoring progress and identifying those who exhibit learning problems (Fuchs, Fuchs, Hamlett, Phillips & Bentz, 1994). And certainly, documentation of a student’s learning failure in spite of valid and systematically-applied treatments should be a necessary component of LD identification (and in fact, many states do attempt to address this issue with their use of pre-referral intervention teams). Nevertheless, there are reasons for questioning the validity of this “treatment resister” model as a psychometric instrument for identification of learning disabilities. Given the reluctance of general educators to use CBM, it seems unlikely that it will be universally applied, and even less likely that it can be applied in a standardized manner, so that all students receive the same instruction and measurement under the same conditions. Since virtually all of the attention to date has been paid to early reading problems, how this model could be applied to different academic areas and different grade levels remains to be seen. All of these problems speak to a problem of reliability and validity of “treatment resistance” as a psychometric measure. How these procedures will discriminate those with learning disabilities from those with, e.g. mental retardation, autism, or hearing impairments remains unclear. Finally, how such procedures to identify learning disabilities are to be used in conjunction with procedures for identification of other categories of exceptionality remains to be seen.
WHAT ARE OTHER WAYS OF ADDRESSING PROBLEMS IN IDENTIFICATION? Over the years, and most particularly in recent years, there have been criticisms of procedures for the identification of learning disabilities. Several alternative identification procedures have been proposed. However, none of these to date have sufficient research to document the likely consequences of their implementation. Additional research could help determine which, if any, of these proposed alternatives could result in improved identification of learning disabilities. But before alternative procedures are widely implemented, they must be shown to be generally superior to present methods.
24
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
Any modification in identification, however, should address effectively the aspects of identification that are presently most widely criticized. A review of the literature critical of learning disabilities identification reveals that these problems include: (a) (b) (c) (d) (e) (f) (g)
overidentification; variability; specificity; conceptual considerations; discrepancy issues; early identification; and local implementation.
Following is a set of suggestions that can be employed to address all of these concerns. (1) Reduce overidentification at the school or district level. Overidentification is often assumed because the proportions of students identified as having learning disabilities have been consistently growing, and because the present percentage of identified students (over 5%) seems higher to many than reasonable for a “disability” status. Research in local and state identification procedures reveal that many students – perhaps as many as one-third to one-half – are identified as having learning disabilities without meeting state criteria. One reason this is done is that school personnel realize that many students require special attention which in many cases can only be provided by referral to special education. Another commonly-described reason for overidentification is that other categories of exceptionality – particularly mental retardation and emotional disturbance – are seen to be more stigmatizing than the category of learning disabilities. Therefore, students are identified as having learning disabilities so that they can be provided with special services without use of a more stigmatizing label (MacMillan & Siperstein, 2001, 2002). Overidentification, then, can be significantly reduced not by provision of a new definition on the federal level, but by requiring local educational authorities to employ strict adherence to state definitional criteria. Such a requirement could potentially reduce the proportion of students identified as having learning disabilities. If this is done, however, some provision must be made for other students who need assistance but do not meet learning disabilities criteria. For students who are simply low achievers, or those who were formerly referred to as “slow learners” (e.g. IQs from 70 to 85), schools should provide, and states should support, some form of additional assistance. Since these students could include
Issues in the Identification of Learning Disabilities
25
over 15% of the school population, however, scarce special education funds should not be used to support these learners. Better education for general education teachers, emphasizing educational methods for low achieving students, could also do much to reduce referrals for learning disabilities. For students who are identified as having learning disabilities because more appropriate categories are thought to be stigmatizing, perhaps these other categories could be described differently, using terminology with more neutral connotations. (2) Reduce variability. With more consistently applied local implementation procedures, variability will also likely decrease. States can reduce variability by employing more consistent and specific criteria (perhaps with the support of the U.S. Department of Education). All states could employ a specific discrepancy formula (or other objective criteria for identifying unexpected underachievement), and discrepancy formulae could be made more consistent. Since regression formulae are generally considered technically superior (Kavale & Forness, 2000), states could specify that regression formulae be employed, and provide specific input on the types of assessment measures that are acceptable for identification of learning disabilities. (3) Establish specificity. It appears that earlier concerns that learning disabilities could not be distinguished reliably from general low achievement were overstated. Nevertheless, it is important to demonstrate that students identified as having learning disabilities are reliably different from general low achievers, and that there is reason to infer a deficit in psychological processing that results in a relative inability to learn. This can be accomplished by strict adherence to explicitly stated state criteria, and the careful and consistent exclusion of alternative explanations of discrepant functioning, including the lack of adequate learning opportunities, cultural disadvantage, emotional disturbance, mental retardation, or sensory impairments (see Wilson, 1985). (4) Address conceptual considerations. Conceptual considerations can be addressed by careful adherence to operational state criteria, and general application of the understanding that IQ-achievement discrepancy is not equivalent to learning disabilities, but is only one of the considerations used as evidence of the existence of learning disabilities. Determination that students have received adequate opportunities to learn should be carefully made. Systematic prereferral interventions should be implemented in the general education classroom, and shown to be inadequate, before students are identified. The nature of pre-referral interventions should be made systematic, and reflect validated practices. If, as a result, all students identified as having learning disabilities demonstrate average general abilities, but demonstrate significant problems in areas relevant to academic functioning in spite of adequate and even intensified general education, many of the conceptual concerns will have been met.
26
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
(5) Address discrepancy issues. Discrepancy remains a significant issue. According to Lyon et al. (2001), “no definitional element of LD has generated as much controversy as the use of IQ-achievement discrepancy in the identification of students with LD” (p. 265). In fact, some individuals clearly will never accept the idea of discrepancy as a criterion for identification (e.g. Aaron, 1997). Nevertheless, at present discrepancy is a most objective indicator of learning disabilities, and its elimination would likely result in overidentification and variability at even higher levels than those at present. In addition, it is difficult to understand current characterizations of learning disabilities without some notion of intraindividual differences between ability and achievement (Mastropieri & Scruggs, 2002). More careful application of discrepancy criteria, in conjunction with carefully documented exclusion criteria, can greatly improve present identification practices. For example, the process suggested by Kavale and Forness (2001) includes specific discrepancy criteria that is supported by documentation of pervasive basic skills deficits, deficits in learning efficiency, deficits in psychological processes, and the exclusion of alternative causes of learning failure. Carefully implemented, such a procedure could address many concerns with present identification practices. Data from the state of Iowa provide evidence that large scale implementation of state criteria can be accomplished to a reasonable degree of fidelity, and that identified students can be demonstrated to be substantially different than students who are not identified (Wilson, 1985). With additional safeguards, a highly appropriate and systematic procedure for identification of learning disabilities can be implemented. (6) Early identification. It is clear that students with learning disabilities who are identified and treated early have a brighter future than students with learning disabilities who are not identified early. It is likely that in many cases lower identification rates during the primary years is the result not of inferior assessment measures and identification, but rather the reluctance of school personnel to classify at early ages. Early identification can be greatly improved by encouragement of early identification practices by the state and federal governments. Nevertheless, it does not logically follow that “the idea that special education funds can be used for early identification and prevention is critical” (Lyon et al., 2001, p. 281). Special education funds were never fully provided by the federal government, and are too limited to be employed in general education arbitrarily, identifying students in need of assistance, but who do not consistently meet specific criteria for learning disabilities. Because of the number of concerns raised, a number of alternative identification methods have been proposed. These include observed deficits in level and slope of academic performance as measured by CBM, deficits
Issues in the Identification of Learning Disabilities
27
in phonological core processes, and failure to respond to validated treatment protocols. Each of these alternative procedures has addressed some important aspects of learning disabilities, and although some seem promising, none have been demonstrated to provide results superior to present procedures. For any alternative approach to be effective, ultimately it must address conceptual, technical, and implementation aspects of the identification process, across grade levels and skill areas, and it must identify individuals who share characteristics commonly associated with learning disabilities, including intraindividual differences and unexpected underachievement. For example, for a “failure to respond to validated treatment” approach to ultimately be of general utility, several criteria must be met. It must be demonstrated that it can be implemented systematically in a standardized way across classrooms, that it can identify “treatment non-responders” in areas other than reading fluency, that it can be implemented from K-12 grade levels, and that the students identified by these methods will meet the characteristics commonly identified with learning disabilities. If this alternative method does not address any of these criteria, additional appropriate procedures must be specified. We suggest that radically altering or eliminating the concept of learning disabilities – without thorough evaluation – could be counterproductive, creating at least as many problems as it addresses. Many of the commonly heard criticisms of prevention efforts – efforts to address the academic needs of the lower 25% in academic achievement can do much to limit referral to special education, but these efforts should be provided through general education funds. The more limited special education funds should be reserved specifically for support of intensive, high-quality interventions for students meeting strict disability standards. (7) Local implementation. Improvements in local implementation of state and federal criteria in the identification of learning disabilities may be the key to addressing many other problem areas in identification. If state criteria for learning disabilities were carefully implemented by all local school districts, many if not most of the criticisms of identification of learning disabilities would be addressed. However, as long as local practices remain subjective and inconsistent, any change in definition or federal policy will be unsuccessful.
SUMMARY AND CONCLUSION: WHAT DO WE KNOW TO DATE ABOUT IMPROVING IDENTIFICATION? In this chapter, we have reviewed criticisms of present practices in identification of learning disabilities. Much of the criticism found in the literature involves
28
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
concerns with overidentification, variability in identification, and specificity in reliably discriminating learning disabilities from general low achievement. The definition of learning disabilities has also been challenged on conceptual grounds. Particularly common are concerns about discrepancy criteria, which, it has been suggested, suffer from problems in measurement, conceptualization, reliance on IQ testing, and exert an inhibiting influence on early identification. Finally, identification procedures as implemented by local school authorities have been characterized as subjective and identification of learning disabilities can be addressed by careful attention to the local implementation aspect of identification. Improving local identification practices could reduce numbers of students identified by one-third or more, and could increase the homogeneity of identified populations. Improving local identification practices would result in identification procedures that are more within the spirit of IDEA legislation. It is also important that general education programs be implemented for the students who would not be served as having learning disabilities, but who nonetheless may need some additional assistance in acquiring important academic skills. Such programming would be very beneficial to general education students, and would likely reduce referrals to special education. However, scarce special education funds not be used for this purpose. Federal and state general education funds could provide for low achieving students in need of academic assistance, while special education funds can continue to be reserved for students identified as having disabilities. As improved procedures are implemented and numbers of students identified as having learning disabilities diminish, federal support, never fully provided, can begin to pay a greater share of the costs of special education. It is clear that further research is needed in alternative identification procedures. Such research should examine all reasonable alternatives, in addition to examining how present procedures could be improved. Through additional research efforts, and careful consideration of possible alternatives that maintain the conceptual underpinnings of learning disabilities, much needed progress in identification can be made.
REFERENCES Aaron, P. G. (1997). The impending demise of the discrepancy formula. Review of Educational Research, 67, 461–502. Ackerman, P. T., Holloway, C. A., Youngdahl, P. L., & Dykman, R. A. (2001). The double-deficit of reading disability does not fit all. Learning Disabilities Research & Practice, 16, 152–160. Algozzine, R. (1985). Low achiever differentiation: Where’s the beef? Exceptional Children, 52, 72–75. Algozzine, R., & Korinek, L. (1985). Where is special education for students with high prevalence handicaps going? Exceptional Children, 51, 388–394.
Issues in the Identification of Learning Disabilities
29
Algozzine, R., & Ysseldyke, J. E. (1987). In defense of different numbers. Remedial and Special Education, 8, 53–56. Algozzine, R., Ysseldyke, J. E., & McGue, M. (1995). Differentiating low-achieving students: Thoughts on setting the record straight. Learning Disabilities Research & Practice, 10, 140–144. Alley, G. R., Deshler, D. D., & Warner, M. M. (1979). Identification of learning disabled adolescents: A Bayesian approach. Learning Disability Quarterly, 2, 83–86. Allor, J. H. (2002). The relationships of phonemic awareness and rapid naming to reading development. Learning Disability Quarterly, 25, 47–57. Al-Otaiba, S. D. (2000). Children who do not respond to early literacy intervention: A longitudinal study across kindergarten and first grade. Unpublished doctoral dissertation, Vanderbilt University, Nashville. Al-Otaiba, S. D., & Fuchs, D. (2002). Characteristics of children who are unresponsive to early literacy intervention: A review of the literature. Remedial and Special Education, 23, 300–316. American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Text Revision. Washington, DC: Author. Anderson, P. L., & Meier-Hedde, R. (2001). Early case reports of dyslexia in the United States and Europe. Journal of Learning Disabilities, 34, 9–21. Baroody, A. J., & Ginsburg, H. P. (1991). A cognitive approach to assessing the mathematical difficulties of children labeled “learning disabled.” In: H. L. Swanson (Ed.), Handbook on the Assessment of Learning Disabilities: Theory, Research and Practice (pp. 177–228). Austin, TX: PRO-ED. Beirne-Smith, M., Ittenbach, R. F., & Patton, J. R. (2002). Mental retardation (6th ed.). Columbus, OH: Merrill. Beitchman, J. H., Cantwell, D. P., Forness, S. R., Kavale, K. A., & Kauffman, J. M. (1998). Practice parameters for the assessment and treatment of children and adolescents with language and learning disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 37(Supplement), 46S–62S. Berlin, R. (1887). Fine besondre art der worstblindheit: Dyslexia [A special type of wordblindness: Dyslexia]. Wiesbaden: Bergmann. Berninger, V. W., & Abbott, R. D. (1994). Redefining learning disabilities: Moving beyond aptitude – achievement discrepancies to failure to respond to validated treatment protocols. In: G. R. Lyon (Ed.), Frames of Reference for the Assessment of Learning Disabilities: New Views on Measurement Issues (pp. 163–183). Baltimore: Brookes. Bradley, R., Danielson, L., & Hallahan, D. P. (Eds) (2002). Identification of learning disabilities: Research to practice. Mahwah, NJ: Lawrence Erlbaum. Christensen, C. A. (1999). Learning disability: Issues of representation, power, and the medicalization of school failure. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 227–249). Boulder, CO: Westview Press. Coles, G. S. (1987). The learning mystique: A critical look at learning disabilities. New York: Pantheon. Cone, T. E., & Wilson, L. R. (1981). Quantifying a severe discrepancy: A critical analysis. Learning Disability Quarterly, 4, 359–371. Cooney, J. B., & Swanson, H. L. (1987). Memory and learning disabilities: An overview. In: H. L. Swanson (Ed.), Memory and Learning Disabilities: Advances in Learning and Behavioral Disabilities (Supplement 2, pp. 1–40). Stamford, CT: JAI Press. Coutinho, M. (1995). Who will be learning disabled after the reauthorization of IDEA? Two very distinct perspectives. Journal of Learning Disabilities, 28, 664–668. Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.
30
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
Dejerine, J. (1892). Contribution a l’etude anatomo-pathologique et clinique des differentes varietes de cecite verbale. Memoriale Societe Biologique, Fev. 27, pp. 61–63. Finlan, T. G. (1992). Do state methods of quantifying a severe discrepancy result in fewer students with learning disabilities? Learning Disability Quarterly, 15, 129–135. Fleischner, J. E. (1994). Diagnosis and assessment of mathematics learning disabilities. In: G. R. Lyon (Ed.), Frames of Reference for the Assessment of Learning Disabilities: New Views on Measurement Issues (pp. 441–458). Baltimore: Brookes. Fletcher, J. M., & Foorman, B. R. (1994). Issues in definition and measurement of learning disabilities: The need for early intervention. In: G. R. Lyon (Ed.), Frames of Reference for the Assessment of Learning Disabilities: New Views on Measurement Issues (pp. 185–200). Baltimore: Brookes. Fletcher, J. M., Francis, D. J., Shaywitz, S. E., Lyon, G. R., Foorman, B. R., Stuebing, K. K., & Shaywitz, B. A. (1998). Intelligent testing and the discrepancy model for children with learning disabilities. Learning Disabilities Research & Practice, 13, 186–203. Fletcher, J. M., Lyon, G. R., Barnes, M., Stuebing, K. K., Francis, D. J., Olson, R. K., Shaywitz, S. E., & Shaywitz, B. A. (2002). Classification of learning disabilities: An evidence-based evaluation. In: R. Bradley, L. Danielson & D. P. Hallahan (Eds), Identification of Learning Disabilities: Research to Practice (pp. 185–250). Mahwah, NJ: Erlbaum. Fletcher, J. M., Shaywitz, S. E., Shankweiler, D. P., Katz, L., Liberman, I. Y., Stuebing, K. K., Francis, D. J., Fowler, A. E., & Shaywitz, B. A. (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement definitions. Journal of Educational Psychology, 86, 6–23. Frankenberger, W., & Fronzaglio, K. (1991). A review of states’ criteria and procedures for identifying children with learning disabilities. Journal of Learning Disabilities, 24, 495–500. Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13, 204– 220. Fuchs, L. S., Fuchs, D., Hamlett, C. L., Phillips, N. B., & Bentz, J. (1994). Classwide curriculum-based measurement: Helping general educators meet the challenge of student diversity. Exceptional Children, 60, 518–527. Fuchs, D., Fuchs, L. S., Mathes, P. G., & Lipsey, M. W. (2000). Reading differences between low-achieving students with and without learning disabilities: A meta-analysis. In: R. Gersten, E. P. Schiller & S. Vaughn (Eds), Contemporary Special Education Research: Syntheses of the Knowledge Base on Critical Instructional Issues (pp. 81–105). Mahwah, NJ: Erlbaum. Fuchs, L. S., Fuchs, D., & Speece, D. L. (2002). Treatment validity as a unifying construct for identifying learning disabilities. Learning Disability Quarterly, 25, 33–45. Galaburda, A. (1991). Anatomy of dyslexia: Argument against phrenology. In: D. D. Duane & D. B. Gray (Eds), The Reading Brain (pp. 119–131). Pankton, MD: York Press. Gallagher, J. J. (1966). Children with developmental imbalances: A psychoeducational definition. In: W. Cruickshank (Ed.), The Teacher of Brain-Injured Children (pp. 20–34). Syracuse, NY: Syracuse University Press. Good, R. H., Simmons, D. C., & Smith, S. B. (1998). Effective academic interventions in the United States: Evaluating and enhancing the acquisition of early reading skills. School Psychology Review, 27, 56–70. Gottlieb, J., Alter, M., Botlieb, B. W., & Wishner, J. (1994). Special education in urban America: It’s not justifiable for many. Journal of Special Education, 27, 453–465.
Issues in the Identification of Learning Disabilities
31
Gresham, F. (August, 2001). Responsiveness to intervention: An alternative approach to the identification of learning disabilities. Paper presented at the U.S. Department of Education LD Summit, Washington, DC. Hallahan, D. P. (2002). Learning disabilities: Historical perspectives. In: R. Bradley, L. Danielson & D. P. Hallahan (Eds), Identification of Learning Disabilities: Research to Practice (pp. 1–67). Mahwah, NJ: Lawrence Erlbaum. Hallahan, D. P., & Cottone, E. A. (1997). Attention deficit hyperactivity disorder. In: T. E. Scruggs & M. A. Mastropieri (Eds), Advances in Learning and Behavioral Disabilities (Vol. 11, pp. 27–68). Stamford, CT: JAI Press. Hallahan, D. P., Kauffman, J. M., & Lloyd, J. W. (1999). Introduction to learning disabilities (2nd ed.). Boston: Allyn & Bacon. Hallahan, D. P., Keller, C. E., & Ball, D. W. (1986). A comparison of prevalence rate variability from state to state for each of the categories of special education. Remedial and Special Education, 7, 8–14. Haring, K. A., Lovett, D. L., Haney, K. F., Algozzine, B., Smith, D. D., & Clarke, J. (1992). Labeling pre-schoolers as learning disabled: A cautionary position. Topics in Early Childhood Special Education, 12, 151–173. Hinshelwood, J. (1896). A case of dyslexia: A peculiar form of word-blindness. Lancet, 2, 1451–1454. Hinshelwood, J. (1917). Congenital word-blindness. London: H.K. Lewis. Horn, W. F., & O’Donnell, J. P. (1985). Early identification of learning disabilities: A comparison of two methods. Journal of Educational Psychology, 76, 1106–1108. Horowitz, S. H., Lichtenstein, B., & Roller, C. (February, 2002). An intervention-oriented, multi-tiered approach for identifying and serving students with learning disabilities. Straw man prepared for the Learning Disabilities Roundtable “Finding Common Ground” Initiative, Washington, DC. Hynd, G. W., Clinton, A. B., & Hiemenz, J. R. (1999). The neoropsychological basis of learning disabilities. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 60–82). Boulder, CO: Westview Press. Kavale, K. A., & Forness, S. R. (1984). A meta-analysis assessing the validity of Wechsler Scale profiles and recategorizations: Patterns or parodies? Learning Disability Quarterly, 7, 136–156. Kavale, K. A., & Forness, S. R. (1987). The far side of heterogeneity: A critical analysis of empirical subtyping research in learning disabilities. Journal of Learning Disabilities, 20, 374–382. Kavale, K. A., & Forness, S. R. (1995). The nature of learning disabilities: Critical elements of diagnosis and classification. Mahwah, NJ: Erlbaum. Kavale, K. A., & Forness, S. R. (1998). The politics of learning disabilities. Learning Disability Quarterly, 21, 245–273. Kavale, K. A., & Forness, S. R. (2000). What definitions of learning disability say and don’t say: A critical analysis. Journal of Learning Disabilities, 33, 239–256. Kavale, K. A., Forness, S. R., & Lorsbach, T. C. (1991). Definition for definitions of learning disabilities. Learning Disability Quarterly, 14, 257–266. Kavale, K. A., Fuchs, D., & Scruggs, T. E. (1994). Setting the record straight on learning disabilities and low achievement. Learning Disabilities Research & Practice, 9, 70–77. Kavale, K. A., & Mattson, P. D. (1983). “One jumped off the balance beam”: Meta-analysis of the efficacy of perceptual motor training. Journal of Learning Disabilities, 16, 165–173. Kavale, K. A., & Nye, C. (1985–1986). Parameters of learning disabilities in achievement, linguistic, neuropsychological, and social/behavioral domains. Journal of Special Education, 19, 443–458.
32
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
Kavale, K. A., & Nye, C. (1991). The structure of learning disabilities. Exceptionality, 2(3), 141–156. Kavale, K. A., & Reese, J. H. (1991). Teacher beliefs and perceptions about learning disabilities: A survey of Iowa’s practitioners. Learning Disability Quarterly, 14, 141–160. Keogh, B. K. (1986). Future of the LD field: Research and practice. Journal of Learning Disabilities, 19, 455–460. Keogh, B. K. (1994). A matrix of decision points in the measurement of learning disabilities. In: G. R. Lyon (Ed.), Frames of Reference for the Assessment of Learning Disabilities: New Views on Measurement Issues (pp. 15–26). Baltimore: Brookes. Keogh, B., & Becker, L. D. (1973). Early detection of learning problems: Questions, cautions, and guidelines. Exceptional Children, 40, 5–11. Kirk, S. A. (1971). Psycholinguistic learning disabilities: Diagnosis and remediation. Urbana, IL: University of Illinois Press. Kosc, L. (1981). Neuropsychological implications of diagnosis and treatment of mathematical learning disabilities. Topics in Learning and Learning Disabilities, 1, 19–30. Kussmaul, A. (1877). Disturbances of speech. In: H. von Ziemssen (Ed.), Cyclopedia of the Practice of Medicine. J. McCreery (Trans.). New York: William Wood. Larsen, S. C., & Hammill, D. D. (1975). The relationship of selected visual perceptual abilities to school learning. Journal of Special Education, 9, 281–291. Lester, G., & Kelman, M. (1997). State disparities in the diagnosis and placement of pupils with learning disabilities. Journal of Learning Disabilities, 30, 599–607. Lyon, G. R. (1985). Identification and remediation of learning disability subtypes: Preliminary findings. Learning Disabilities Focus, 1, 21–35. Lyon, G. R. (March, 2001). Measuring success: Using assessments and accountability to raise student achievement. Statement to Subcommittee on Education Reform, Committee on Education and the Workforce, U.S. House of Representatives, Washington, DC http://edworkforce. house.gov/hearings/107th/edr/account3801/lyon.htm Lyon, G. R., & Fletcher, J. M. (2001). Early warning system. Education Next (Summer), 23–29. http://www.educationnext.org/20012/22.html Lyon, G. R., Fletcher, J. M., Shaywitz, S. E., Shaywitz, B. A., Torgesen, J. K., Wood, F. B., Schulte, A., & Olson, R. (2001). Rethinking learning disabilities. In: C. E. Finn, Jr., A. J. Rotherham & C. R. Hokanson, Jr. (Eds), Rethinking Special Education for a New Century (pp. 259–287). Washington, DC: Thomas B. Fordham Foundation. MacMillan, D. L., Gresham, F. M., & Bocian, K. M. (1998). Discrepancy between definitions of learning disabilities and school practices: An empirical investigation. Journal of Learning Disabilities, 31, 314–326. MacMillan, D. L., Gresham, F. M., Siperstein, G. N., & Bocian, K. M. (1996). The labyrinth of I.D.E.A.: School decisions on referred students with subaverage general intelligence. American Journal on Mental Retardation, 101, 161–174. MacMillan, D. L., & Siperstein, G. N. (August, 2001). Learning disabilities as operationally defined in schools. Paper presented at the U.S. Department of Education LD Summit, Washington, DC. MacMillan, D. L., & Siperstein, G. N. (2002). Learning disabilities as operationally defined by schools. In: R. Bradley, L. Danielson & D. P. Hallahan (Eds), Identification of Learning Disabilities: Research to Practice (pp. 287–333). Mahwah, NJ: Lawrence Erlbaum Associates. MacMillan, D. L., Siperstein, G. N., & Gresham, F. M. (1996). A challenge to the viability of mild mental retardation as a diagnostic category. Exceptional Children, 62, 356–371. Mastropieri, M. A. (1987). Statistical and psychometric issues surrounding severe discrepancy: A discussion. Learning Disabilities Research, 30, 29–31.
Issues in the Identification of Learning Disabilities
33
Mastropieri, M. A. (1988). Learning disabilities in early childhood. In: K. A. Kavale (Ed.), Learning Disabilities: State of the Art and Practice (pp. 161–179). Boston: College-Hill/Little Brown. Mastropieri, M. A. (August, 2001). Discrepancy models in the identification of learning disabilities. Paper presented at the U.S. Department of Education LD Summit, Washington, DC. Mastropieri, M. A., & Scruggs, T. E. (1997). Best practices in promoting reading comprehension in students with learning disabilities. Remedial and Special Education, 18, 197–213. Mastropieri, M. A., & Scruggs, T. E. (2000). The inclusive classroom: Strategies for effective instruction. Upper Saddle River, NJ: Prentice Hall. Mastropieri, M. A., & Scruggs, T. E. (2002). Discrepancy models in the identification of learning disabilities: A response to Kavale. In: R. Bradley, L. Danielson & D. P. Hallahan (Eds), Identification of Learning Disabilities: Research to Practice (pp. 449–456). Mahwah, NJ: Erlbaum. McCarthy, J. M. (1971). Learning disabilities: Where have we been: Where are we going? In: D. Hammill & N. Bartel (Eds), Educational Perspectives in Learning Disabilities (pp. 10–19). New York: Wiley. McKinney, J. D., Short, E. J., & Feagans, L. (1985). Academic consequences of perceptual-linguistic subtypes of learning disabled children. Learning Disabilities Research, 1, 6–17. McLeskey, J., & Waldron, N. L. (1990). The identification and characteristics of students with learning disabilities in Indiana. Learning Disabilities Research, 5, 72–78. Meehl, P. (1973). Psychodiagnosis: Selected papers. Minneapolis: University of Minnesota Press. Mercer, C. D., Forgnone, C., & Wolking, W. D. (1976). Definitions of learning disabilities used in the United States. Journal of Learning Disabilities, 9, 376–386. Morgan, W. P. (1896). A case of congential word blindness. The Brititsh Medical Journal, 2, 1378. Naglieri, J. (2001). Do ability and reading achievement correlate? Journal of Learning Disabilities, 34, 304–306. Naglieri, J., & Das, J. P. (1997). Cognitive assessment system. Itasca, IL: Riverside. Olson, R. K. (1999). Genes, environment, and reading disabilities. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 3–21). Boulder, CO: Westview Press. Reynolds, C. R. (1984). Critical measurement issues in learning disabilities. The Journal of Special Education, 18, 451–476. Rourke, B. (1989). Non-verbal learning disabilities: The syndrome and the model. New York: Guilford Press. Rourke, B. (1993). Arithmetic disabilities, specific and otherwise: A neuropsychological perspective. Journal of Learning Disabilities, 26, 214–226. Rumsey, J. M. (1996). Neuroimaging in developmental dyslexia: A review and conceptualization. In: G. R. Lyon & J. M. Rumsey (Eds), Neuroimaging: A Window to the Neurological Foundations of Learning and Behavior in Children (pp. 57–78). Baltimore: Brookes. Rutter, M., & Yule, W. (1975). The concept of specific reading retardation. Journal of Child Psychology and Psychiatry, 16, 181–197. Schrag, J. (2000). Discrepancy approaches for identifying learning disabilities. Alexandria, VA: National Association of State Directors of Special Education. Scruggs, T. E. (1987). Theoretical issues surrounding severe discrepancy: A discussion. Learning Disabilities Research, 3, 21–23. Scruggs, T. E., & Cohn, S. J. (1983). A university-based summer program for a highly able but poorly achieving Indian child. Gifted Child Quarterly, 27, 90–93.
34
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
Scruggs, T. E., & Mastropieri, M. A. (1986). Academic characteristics of behaviorally disordered and learning disabled children. Behavioral Disorders, 11, 184–190. Scruggs, T. E., & Mastropieri, M. A. (1988). Legitimizing the field of learning disabilities: Does research orientation matter? Journal of Learning Disabilities, 21, 219–222. Scruggs, T. E., & Mastropieri, M. A. (1994–1995). Assessing students with learning disabilities: Current issues and future directions. Diagnostique, 20, 17–31. Scruggs, T. E., & Mastropieri, M. A. (2002). On babies and bathwater: Addressing the problems of identification of learning disabilities. Learning Disability Quarterly, 25, 155–168. Shaywitz, S. E., Escobar, M. D., Shaywitz, B. A., Fletcher, J. M., & Makach, R. (1992). Evidence that dyslexia may represent the lower tail of a normal distribution of reading ability. The New England Journal of Medicine, 326, 145–150. Shaywitz, B. A., Fletcher, J. M., Holahan, J. M., & Shaywitz, S. E. (1992). Discrepancy compared to low achievement definitions of reading disability: Results from the Connecticut Longitudinal Study. Journal of Learning Disabilities, 25, 639–648. Shepard, L. A. (1980). An evaluation of the regression discrepancy method for identifying children with learning disabilities. Journal of Special Education, 14, 79–91. Siegel, L. S. (1989). I.Q. is irrelevant to the definition of learning disabilities. Journal of Learning Disabilities, 25, 618–629. Siegel, L. (1999). Learning disabilities: The roads we have traveled and the path to the future. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 159–175). Boulder, CO: Westview Press. Siegel, L. S., & Ryan, E. B. (1988). Development of working memory in normally achieving and subtypes of learning disabled children. Child Development, 60, 973–980. Skrtic, T. M. (1999). Learning disabilities as organizational pathologies. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 193–226). Boulder, CO: Westview Press. Sleeter, C. E. (1995). Radical structuralist perspectives on the creation and use of learning disabilities. In: T. Skrtic (Ed.), Disability and Democracy: Reconstructing (Special) Education for Postmodernity (pp. 153–165). New York: Teachers College Press. Spear-Swerling, L. (1999). Can we get there from here: Learning disabilities and future education policy. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 250–276). Boulder, CO: Westview Press. Speece, D. (2002). Classification of learning disabilities: Convergence, expansion, and caution. In: R. Bradley, L. Danielson & D. P. Hallahan (Eds), Identification of Learning Disabilities: Research to Practice (pp. 279–285). Mahwah, NJ: Erlbaum. Speece, D. L., & Case, L. P. (2001). Classification in context: An alternative approach to identifying early reading disability. Journal of Educational Psychology, 93, 735–749. Stanovich, K. E. (1988). Explaining the differences between the dyslexic and the garden-variety poor reader: The phonological core-variable difference model. Journal of Learning Disabilities, 21, 590–604. Stanovich, K. E. (1991). Conceptual and empirical problems with discrepancy definitions of reading disability. Learning Disability Quarterly, 14, 269–283. Sternberg, R. J. (1999). Epilogue: Toward and emerging consensus about learning disabilities. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 277–282). Boulder, CO: Westview Press.
Issues in the Identification of Learning Disabilities
35
Strang, J. D., & Rourke, B. P. (1983). Concept-formation/non-verbal reasoning abilities of children who exhibit specific academic problems with arithmetic. Journal of Clinical Child Psychology, 24, 28–37. Swanson, H. L. (1993). Theoretical, technical, and practical aspects of the S-cognitive processing test: The development of a dynamic assessment measure. In: T. E. Scruggs & M. A. Mastropieri (Eds), Advances in Learning and Behavioral Disabilities (Vol. 10, Part A, pp. 135–208). Stamford, CT: JAI Press. Swerling, S. L., & Sternberg, R. J. (1996). Off track: When poor readers become learning disabled. Boulder, CO: Westview Press. Torgesen, J. K. (1998). Learning disabilities: An historical and conceptual overview. In: B. Y. L. Wong (Ed.), Learning about Learning Disabilities (2nd ed., pp. 3–34). New York: Academic Press. Torgesen, J. K. (1999). Phonologically-based reading disabilities: Toward a coherent theory of one kind of learning disability. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 106–135). Boulder, CO: Westview Press. Torgesen, J. K. (2000). Individual differences in response to early interventions in reading: The lingering problem of treatment resisters. Learning Disabilities Research and Practice, 15, 55–64. Torgesen, J. K. (2002). Empirical and theoretical support for diagnosis of learning disabilities by assessment of intrinsic processing weaknesses. In: R. Bradley, L. Danielson & D. P. Hallahan (Eds), Identification of Learning Disabilities: Research to Practice (pp. 565–652). Mahwah, NJ: Erlbaum. Torgesen, J. K., & Wagner, R. K. (1998). Alternative diagnostic approaches for specific developmental reading disabilities. Learning Disabilities Research and Practice, 13, 220–232. U.S. Department of Education (2000). Twenty-second annual report to Congress on the implementation of the Individuals with Disabilities Education Act. Washington, DC: Author. U.S. Department of Education, Office of Special Education Programs (2002). Specific learning disabilities: Finding common ground. Washington, DC: Author. Vellutino, F. R., Scanlon, D. M., & Lyon, G. R. (2000). Differentiating between difficult-to-remediate and readily remediated poor readers: More evidence against the IQ-discrepancy definition of reading disability. Journal of Learning Disabilities, 33, 223–238. Wagner, R. K., & Garon, T. (1999). Learning disabilities in perspective. In: R. J. Sternberg & L. Spear-Swerling (Eds), Perspectives on Learning Disabilities: Biological, Cognitive, Contextual (pp. 83–105). Boulder, CO: Westview Press. Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (1999). Comprehensive test of phonological processes. Austin, TX: PRO-ED. Wilson, L. R. (1985). Large-scale learning disability identification: The reprieve of a concept. Exceptional Children, 52, 44–51. Wissink, J. F., Kass, C. E., & Ferrell, W. R. (1975). A Bayesian approach to the identificaiton of children with learning disabilities. Journal of Learning Disabilities, 8, 58–66. Wolf, M., & Bowers, P. G. (1999). The double-deficit hypothesis for the developmental dyslexias. Journal of Educational Psychology, 91, 415–438. Wong, B. Y. L. (1996). The ABCs of learning disabilities. New York: Academic Press. Wood, F., Flowers, L., Buchsbaum, M., & Tallal, P. (1991). Investigation of abnormal left temporal functioning in dyslexia through CBF, auditory evoked potentials, and positron emission tomography. Reading and Writing: An Interdisciplinary Journal, 3, 379–393. Ysseldyke, J. E., Algozzine, B., Richey, L., & Graden, J. P. (1982). Declaring students eligible for learning disability services. Why bother with the data? Learning Disability Quarterly, 5, 37–44.
36
THOMAS E. SCRUGGS AND MARGO A. MASTROPIERI
Ysseldyke, J. E., Algozzine, B., Shinn, M. R., & McGue, M. (1982). Similarities and differences between low achievers and students classified learning disabled. Journal of Special Education, 16, 73–85. Zigmond, N. (1993). Learning disabilities from an educational perspective. In: G. R. Lyon, D. B. Gray, J. R. Kavanagh & N. A. Krasnegor (Eds), Better Understanding Learning Disabilities: New Views from Research and their Implications for Education and Public Policies (pp. 251–272). Baltimore, MD: Brookes.
STARTING AT THE BEGINNING FOR LEARNING DISABILITIES IDENTIFICATION: RESPONSE TO INSTRUCTION IN GENERAL EDUCATION Deborah L. Speece, Dawn Eddy Molloy and Lisa Pericola Case ABSTRACT Most definitions of learning disabilities (LD) include a qualification that adequate general education instruction was received and the child with LD did not benefit. Rarely is this tenet assessed in either practice or research before a diagnosis is made. In this chapter we review three studies that investigated children’s responsiveness to general education reading instruction as an indicator of the need for more intensive interventions. Adequacy of instruction was quantified by children’s level and rate of progress as measured by curriculum-based measures of oral reading fluency. This model of identification was based on Fuchs and Fuchs (1998) treatment-validity model wherein children who do not respond to interventions provided in the general education classroom are potential candidates for special education services. The results of the studies reviewed indicate that the model is valid in that: (a) children who differ from their peers on level and Identification and Assessment Advances in Learning and Behavioral Disabilities, Volume 16, 37–50 Copyright © 2003 by Elsevier Science Ltd. All rights of reproduction in any form reserved ISSN: 0735-004x/doi:10.1016/S0735-004X(03)16002-8
37
38
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
slope of performance have more severe academic and behavioral problems than children who have IQ-achievement discrepancies or low achievement; (b) children who demonstrate persistent non-responsiveness over three years differ from other at-risk children on reading, reading-related, and behavioral measures; and (c) at-risk children who participated in speciallydesigned general education interventions had better outcomes than at-risk children who did not participate.
STARTING AT THE BEGINNING FOR LEARNING DISABILITIES IDENTIFICATION: RESPONSE TO INSTRUCTION IN GENERAL EDUCATION Twenty-five years ago, the U.S. Office of Education offered the following criteria for identifying learning disabilities (LD): (a) failure to benefit from adequate instruction; (b) a severe discrepancy between achievement and intellectual ability; and (c) exclusion of sensory impairments, mental retardation, emotional disturbance, or environmental, cultural, or economic disadvantage (U.S. Office of Education, 1977). Since that time, accumulated research calls into question the validity of the second criterion, IQ-achievement discrepancy (e.g. Fletcher et al., 1994; Share, McGee & Silva, 1989; Speece & Case, 2001; Stanovich & Siegel, 1994). These data have fueled interest in alternative methods of identifying LD, most of which center on responsiveness to instruction (e.g. Fuchs, 1995; Fuchs & Fuchs, 1998; Speece & Case, 2001; Vellutino et al., 1996). Further evidence that instructional responsiveness may be pertinent to LD identification was provided by a survey of experts that asked for components of a reading disabilities definition (Speece & Shekitka, 2002). A majority of these experts, not surprisingly, selected reading achievement (83.7%) and phonemic awareness (79.4%). Of note is that 67.3% of the respondents also selected responsiveness to instruction. The focus on instruction in reconceptualizing LD is refreshing. Prior to this point, LD was viewed strictly as a within-child deficit despite the fact that degree of learning is influenced by the context in which learning occurs (Carroll, 1963; Deno, 1989; Keogh & Speece, 1996). Response-to-instruction models make no assumptions about the underlying cause of the learning difficulty. Instead, such models recognize that the difficulty may lie within the child, within the instruction, or both. Only by systematically strengthening the quality of instruction and measuring a child’s response to that instruction can inferences be made about the possibility that child deficits contribute to learning difficulties. Thus, the long-held assumption that “processing” or “neurological” deficits play a role in LD is not
Starting at the Beginning for Learning Disabilities Identification
39
ruled out but rather set aside, awaiting the development of valid measurement procedures. The treatment-validity model (Fuchs, 1995; Fuchs & Fuchs, 1998) is the most fully developed conceptualization of how response to instruction may be applied to LD identification. In this model, curriculum-based measures (CBM) are used in general education classrooms to identify children whose level and rate (slope) of performance are below those of their classmates. This “dual discrepancy” of level and slope becomes the marker by which to judge responsiveness to instruction. Dually-discrepant children then receive instruction from their general education teachers that is reformulated to meet their needs. This instruction and continued placement in general education would be “treatment valid” for children who demonstrate improvement. For children who continue to demonstrate a dual discrepancy, the instruction and placement would not be valid, leading to a trial placement in special education to determine responsiveness, and hence, treatment validity, under more intensive instructional parameters. This model is not without difficulty or controversy (Fuchs, Fuchs & Speece, 2002). Full implementation would require weekly assessment of all children in all academic areas addressed by CBM (reading, mathematics, spelling, writing), development of instructional plans that are based on validated instructional protocols and that are feasible for general education classrooms, and fidelity of implementation across all interventions. Despite the enormity of the task, there are many benefits, including: (a) less reliance on teacher referral thereby reducing false negative identification and possible referral bias; (b) a focus on academic behavior rather than difficult-to-measure processing weaknesses; (c) a focus on growth in addition to absolute level of performance; (d) elimination of IQ-achievement discrepancies and correspondingly, the need to administer intelligence tests; (e) elimination of false-positive identification; and (f) the potential to improve general education environments for all children. This new perspective also addresses a criterion in the federal regulations that is generally disregarded in diagnoses of LD: the child has received adequate instruction in general education but failed to benefit from it. In the model proposed by Fuchs and Fuchs (Fuchs, 1995; Fuchs & Fuchs, 1998), classrooms that have a mean level and slope of performance that is substantially lower than other classrooms, would first receive classroom-level intervention geared to improving the academic performance of all children. Thus, children could not be identified as dually discrepant unless there was evidence that most children
40
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
in the classroom were responding to the curriculum, that is, that instruction was adequate.
ANALYSIS OF THE TREATMENT VALIDITY MODEL There is much to recommend a treatment validity approach to identifying learning disabilities and much to study regarding its implementation. We investigated several aspects of the model in a three-year longitudinal study of reading (Case, Speece & Molloy, in press; Speece & Case, 2001; Speece, Case & Molloy, 2002). Our primary objective was to assess the validity of the model from multiple perspectives. In a series of three studies we asked the following questions: (a) do first- and second-grade children who are identified as dually discrepant in reading differ from children who have IQ-reading achievement discrepancies or who have low reading achievement?; (b) across three years, do dually-discrepant children who are persistently nonresponsive to specially-designed general education instruction experience greater reading, reading-related, and behavioral problems than duallydiscrepant children who do respond? Further, are there differences in the instructional contexts?; and (c) does the implementation of a modified treatment validity model result in better achievement and behavioral outcomes for dually-discrepant children compared to dually-discrepant children who do not participate in the model? This final issue pertains to outcomes associated with an improved pre-referral system that is buttressed by specific and research-based intervention components. The remainder of this chapter summarizes the major results of these studies and ends with implications for research.
Overview of the Sample and Measures As noted, the goal of the three-year study was to investigate the validity of dual discrepancy and responsiveness to treatment as a method of operationalizing reading disability. Three suburban schools from one school district in the Mid-Atlantic states participated and were selected based on comparability of school size, strength of school leadership, ethnic representation, reading scores on district-administered tests, and mobility rate. From a fall CBM population screen of two cohorts of first-grade children and one cohort of second-grade children, we identified the longitudinal sample of at-risk children. Children who performed in
Starting at the Beginning for Learning Disabilities Identification
41
the lowest 25% of their class on Letter Sound Fluency (LSF, first grade) or Oral Reading Fluency (ORF, second grade) constituted the longitudinal sample. Each year we identified a Purposive Sample (PS) from each classroom to estimate ORF classroom slope and level, information necessary to identify children as dually discrepant. PS children represented the range of reading skill in each classroom. The measures administered varied for the AR and PS children. All children completed three reading subtests from the Woodcock–Johnson Psychoeducational Battery-Revised (WJ-R; 1989) and an abbreviated battery of the WISC-R (Wechsler, 1974) to estimate Full Scale IQ. The AR group, which was the focus group for the study, also received phonological processing tasks (phonological awareness, RAN, and word reading efficiency) and teacher ratings of their classroom behavior (social skills, problem behavior, and academic competence). At the end of every year, we collected data on services received by all children, including additional instruction outside the general education classroom and interaction with the special education system.
Is the Dually Discrepant Classification Valid? Based on data from the study’s first year (Speece & Case, 2001), we compared children who were dually discrepant (DD, N = 47) on ORF at the end of the year to children who exhibited IQ-reading achievement discrepancies (IQRDG, N = 17) and to children who exhibited low achievement (LA, N = 28), defined as a standard score at or below 90 on the Basic Reading Cluster Score of the WJ-R. We also compared these groups to children in both the Purposive Sample (PS, N = 123) and at-risk sample (AR, N = 86) who were not identified as members of any of the problem-reader groups. Importantly, all groups were defined by the researchers and were not based on teacher referrals. If dual discrepancy is a valid marker for reading disability, then children so identified should be distinguishable from PS and AR non-classified children on reading, reading-related, and behavioral measures. Further, they should exhibit skills that are comparable to or lower than children who compose the other problem-reader groups (IQRDG, LA). We were also interested in differences on age as well as gender and ethnic representation to address issues of early identification and overrepresentation. The results supported the validity of the dual-discrepancy classification. The DD, AR, and PS groups could be compared on IQ, reading, and age. The DD children had lower reading scores than the other two groups, lower IQ than the PS group but not the AR group, and the groups did not differ on age. DD and AR groups could be compared on phonological processing and classroom
42
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
behavior. The DD group exhibited lower performance on phonological awareness, word reading efficiency, and ratings of academic competence. The DD and AR groups did not differ on RAN, social skills, or problem behaviors. Thus, the DD classification demonstrated construct validity with respect to two critical groups: children who were not expected to demonstrate reading problems and children who demonstrated initial difficulty (like the DD group) but who kept pace with their classmates. With respect to the other two poor reader groups, the DD group was either comparable or experienced more difficulty. We expected that the DD and IQRDG groups would differ on IQ based on the criteria used for classification, and they did. However, despite an approximate 22 point mean difference on IQ, the two groups did not differ on word reading skills (ES = 0.10). This finding supports other evidence that IQ-achievement reading discrepancies do not capture all children who experience reading difficulty (e.g. Fletcher et al., 1994). Not all DD-IQRDG differences on phonological processing reached statistical significance but, in all cases, the DD group had lower performance, with effect sizes ranging from 0.33 to −1.39. These small to large effect sizes suggest that the differences were educationally meaningful. The two groups also differed on classroom behavior (ES range = 0.54–0.71), with a significant difference on academic competence. We observed fewer differences in comparing the DD and LA groups. We found significant differences for second-grade children on phonological awareness, favoring the LA group, and on teacher ratings of classroom behavior, also favoring LA children (ES = 0.21 to −0.29). Although modest, these differences are not trivial given the selection criteria for LA. These groups did not differ on either IQ or reading. The comparisons of age, gender, and ethnic proportions were revealing. For both sets of group comparisons (DD-IQRDG, DD-LA), the DD children were significantly younger (ES = 1.16 and 0.69, respectively). The proportions of females and males did not differ across any of the groups, contrasting with findings for school-identified samples. With respect to ethnicity, the DD and LA groups reflected the proportions of majority-minority representation in the school system from which the sample was drawn. Over 80% of the IQRDG group was from the majority group (Caucasian). Thus, only the DD group reflected all three aspects of social consequential validity (Messick, 1995) in that they were younger and reflected gender and racial equity. The overall picture is that the DD group can be differentiated from normallyachieving children and from children with equivalent scores on screening measures early in the school year. Moreover, meaningful differences existed between the DD group and the two other poor-reader groups and, importantly, the DD group did not reflect the negative social consequences associated with current school
Starting at the Beginning for Learning Disabilities Identification
43
identification procedures. Thus, by focusing on both level and growth in reading achievement as indexed by CBM, a valid group of children who experienced reading problems was identified. Although simpler identification methods would be preferred, other analyses indicated that single indicators of reading difficulty (LSF, ORF, phonological awareness) were not sensitive indicators of either DD or status as problem readers (DD, IQRDG, and LA) (Speece & Case, 2001). The dual-discrepancy method would require major changes in the way children are identified; however, our initial evidence suggests that benefits may outweigh the cost of change.
Does Persistent Non-responsiveness Signal Disability? The previous study represented the first phase of the treatment-validity model: the identification of children as dually discrepant. In that study, DD was based on reading performance across most of a school year. In the treatment validity model (Fuchs, 1995; Fuchs & Fuchs, 1998), this phase and ensuing phases are conceptualized as eight-week time frames. Thus, it is likely that our DD group represented children with more severe reading problems than would be experienced by children identified within a shorter time span. Our second study (Case, Speece & Molloy, in press) examined phases two and three of the treatment validity model during which general education instruction is modified for children as soon as they are identified as DD. This study took place in one of our participating schools across three years. DD identification procedures were conducted three to four times a year, roughly corresponding to eight-week intervals. When a child was identified as DD, the authors met with the child’s general education teacher to develop an intervention that was based on the child’s strengths and weaknesses, reflected research-based principles of effective reading instruction, and was considered feasible by the general education teacher. The plans were implemented for eight weeks, at which time the team met to review progress and design new instruction as needed. During the eight weeks, a researcher met with the teacher to discuss implementation problems, offer assistance, and discuss progress. A researcher also made unannounced visits to check fidelity of implementation. The focus of the study was to determine if DD children who were persistently non-responsive to general education interventions experienced more severe reading, reading-related, and behavioral problems than either DD children who were more responsive or at-risk children who were never identified as DD. The “persistent non-responders” would be equivalent to children who enter phase four of the treatment validity model, a trial placement in special education. Thus, these
44
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
children should have more severe problems if a response-to-instruction model is valid for LD identification. Our sample consisted of 36 children who were in the study for three years. These children were classified into three groups based on degree of responsiveness. The frequently-dually-discrepant group (FDD, N = 7) was composed of children who were identified four or more times as DD over the course of the study and were considered the persistently non-responsive group. The infrequently-duallydiscrepant group (IDD, N = 17) was identified three or fewer times, and the never-dually-discrepant group (NDD, N = 12) was composed of at-risk children defined by the fall screening (like the other two groups) but never identified as dually discrepant. These groups did not differ on age, IQ, or mother’s years of education. To address the primary research question, the groups were compared on the measures described previously: reading, phonological awareness, RAN, word reading efficiency, and teacher ratings of academic competence, social skills, and problem behaviors in a series of repeated measures ANOVAs. The FDD children performed significantly worse that the NDD children on Letter Word Identification, Word Attack, Word Reading Efficiency, Academic Competence, Social Skills, and Problem Behaviors and significantly worse than the IDD children on Letter Word Identification, Word Reading Efficiency, Academic Competence, and Problem Behaviors. The sensitivity of the responsiveness groupings was further apparent in the differences between the IDD and NDD groups on Academic Competence, and Problem Behaviors. Slope estimates provided by Deno, Fuchs, Marsten and Shin (2001) supported our proposal that children in the the FDD group would be candidates for special education services. Deno et al. estimated that oral reading fluency slopes for children in special education were 0.71 in first and second grades and 0.58 in third and fourth grades. The FDD children’s mean slopes were 0.28 and 0.67, respectively, which approximate the Deno et al. figures. Although the FDD children would, by definition, have the lowest slopes of the children in this study, the similarity of their slopes to a separately diagnosed sample provides further evidence supporting criterion-related validity in addition to that found in the group analyses reported above. We considered whether differences in instructional context may explain the academic and behavioral differences between the groups’ responsiveness. If the FDD group was in less-effective classrooms, then the assertion that they are candidates for special education services is less tenable in the treatment-validity model. We compared the groups on a measure of instructional effectiveness that was based on two-yearly classroom observations, teachers’ years of experience, and mean classroom slopes. No group differences or interactions with year
Starting at the Beginning for Learning Disabilities Identification
45
were obtained. Thus, the hypothesis that FDD children received poorer quality instruction, had less experienced teachers, or had peers who demonstrated lower growth in ORF could not be supported. The differences observed between the three groups were not a function of differential classroom environment, lending further credence to the validity of the classification defined as degree of responsiveness to instruction.
Does an Improved Pre-referral System Result in Better Outcomes for Dually-Discrepant Children? The previous two studies provided evidence that the dual-discrepancy approach is a valid index of children’s responsiveness to general education instruction. We demonstrated that dual-discrepancy status, based on reading performance across most of the school year, equated with severe reading and behavioral difficulties. Also, we found that children who consistently fail to respond to redesigned general education intervention have considerably poorer academic and behavioral outcomes over the early elementary school years. The final study to be reviewed compared dually-discrepant children who received general education intervention to dually-discrepant children in two other schools who did not receive special interventions (Speece, Case & Molloy, 2002). This study asked whether there was any benefit to implementing a pre-referral system that was strengthened by explicit intervention plans built on research-based principles with attention to fidelity of implementation. These plans were developed collaboratively by the authors and teachers. Our operationalization of pre-referral differs considerably from earlier research on the topic. Past studies typically relied on teacher referral, did not collect data on interventions within classrooms, and the primary dependent variable was frequency of special education placement, not student achievement. The intervention school was called the Full Model (FM) because interventions were implemented in general education classrooms and progress was monitored weekly. Two schools, Assessment Feedback (AF) and Assessment Only (AO), provided comparisons. Children in these schools received the same assessments, but received less frequent oral reading fluency monitoring (approximately 10 data points per year). In the AF condition, researchers met with teachers twice a year, providing assessment data and explaining the concept of dual discrepancy as needed. Researchers provided a written report that identified children who were dually discrepant, including graphs of performance, and discussed the children and project in general with teachers. We did not offer any advice on instruction, but rather emphasized that the DD children may require additional assistance. The
46
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
AO school was a true control condition. The research team did not meet with the teachers during the study to provide information on DD status. The AF and AO conditions were included to control, respectively, for the possibilities that: (a) feedback to teachers on CBM performance may affect student achievement independent of a specific intervention plan (Fuchs & Fuchs, 1986; Fuchs, Fuchs & Hamlet, 1989a); and (b) measurement alone may affect student achievement (Fuchs & Fuchs, 1986; Fuchs, Fuchs & Hamlet, 1989b; Jenkins, Mayhall, Peshka & Townsend, 1974; Wesson et al., 1988). Only children identified as DD at any point were included in the analyses. Analyses were conducted within year to include all three cohorts of children (Year one N = 75, Year two N = 110, Year three N = 107). The number of DD children within schools was roughly equivalent. As is often the case with school-based research, our design met with some problems. In the summer following Year 1 of the study, the participating school district made plans to intensify general education literacy instruction in first and second grades. In the modified program, two certified teachers staffed each first- and second-grade classroom to deliver 90 minutes of literacy instruction every day. Two of our schools, FM and AO, participated in the modified program which included two weeks of summer training that emphasized phonological awareness, the alphabetic principle, writing, and assessment using running records. Our third school, AF, participated in the school district’s modified program in Year 3. Thus, our Full Model condition faced a stiff test and the research question became what is the added value of the FM condition over the intensified general education literacy program? The most stringent comparisons were between FM and AO in Years 2 and 3 since they both participated in the new program for the same amount of time. Although this was a daunting prospect, we found that the FM/DD children received, over time, an academic benefit above and beyond the new literacy program. In Year 1, prior to the new program, the only difference between the conditions was for social skills and the effect favored AF over FM. Although not significant, the AF teachers also rated their DD children as exhibiting fewer problem behaviors compared to FM (ES = 0.72). In Year 2, the significant comparisons were for Letter Sound Fluency level and slope (first grade children) with FM > AF but not AO. Small to moderate effect size estimates indicated higher performance by FM on WJ-R reading cluster scores compared to both of the control conditions (ES range = 0.31–0.51). For Year 3, we found three significant effects: ORF level, ORF slope, and school attention. “School attention” referred to the sum of additional services (e.g. instruction from a reading specialist) and meetings held on behalf of the students (e.g. school assistance teams, IEP). For ORF level, FM read significantly
Starting at the Beginning for Learning Disabilities Identification
47
more words correctly than AF but not AO (ES = 0.48 and 0.29, respectively). For ORF slope, FM grew at a significantly faster rate than either of the two comparison conditions (ES = 0.79 and 0.51, respectively). On school attention, FM had significantly lower scores than AO (ES = 1.33) but not AF (ES = 0.75). The ORF slope and school attention results are especially important because they demonstrate that the FM condition, representing an enhanced pre-referral intervention strategy, did improve outcomes for DD children over and above any effects of the new literacy program implemented in the AO school. It is also true that, by Year 3, several variables (phonological awareness, RAN, WJ-R reading scores, and teacher ratings of classroom behavior) yielded no differences. Thus, the significant findings must be placed in the context of all the variables tested. Nonetheless, these findings represent the first demonstration that an enhanced pre-referral model results in improved academic outcomes for at-risk children.
DISCUSSION AND IMPLICATIONS Although the number of children is relatively small in the studies reviewed, they are representative of children in the participating schools because they were initially identified based on a population screen (approximately 1,000 children in the first two years to identify at-risk children) and further identified based on objective criteria for classification. These design features are critical in understanding alternative methods of identifying learning disabilities that are not confounded by sampling bias. Further research with larger samples of AR and DD children in more academic areas will be necessary to test and extend the reliability of our findings. We established that the dual-discrepancy criterion, based on CBM oral reading fluency measures, identifies a valid, younger group of poor readers free of gender and ethnic bias. We also demonstrated that DD children who are persistently non-responsive to general education interventions have poorer academic and behavioral outcomes and are in need of more intensive interventions than were provided in their general education classrooms. Finally, we found that DD children who received specially-designed general education instruction had better outcomes than DD children who did not participate and required fewer services beyond their general education classrooms. These findings set the foundation for further work with the treatment-validity model to identify LD. A major difference between our implementation and the conceptualization of the treatment-validity framework (Fuchs & Fuchs, 1998) is the length of time to identify candidates for special education. In our second study (Case, Speece & Molloy, in press), we reviewed DD status over a three-year
48
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
period to examine degree of responsiveness. This was due to several constraints that included time required to secure parent permission and the fact that our project co-existed with, rather than supplanted, the special education identification system. When the model is fully functional and operating on well-defined time frames for identification and intervention, a child who is DD early in the school year could be deemed eligible for special education services before the end of the school year. Research investigating this more efficient model is critical to understanding the characteristics of children identified and their instructional requirements. Another research need is determining the dual discrepancy cut points for level and slope. We used one standard deviation below classroom mean performances, consistent with findings for math (Fuchs & Fuchs, 1998). These criteria may need adjustment, depending on the child’s age. To capture older children who require assistance, reducing the size of the discrepancy may be required. In any event, the issue needs to be studied systematically. Finally, closer examination is needed of research-based interventions that can be realistically and reliably implemented by general education teachers. We relied on the extant literature on phonological awareness, the alphabetic principle, and peer tutoring (e.g. Adams, Foorman, Lundberg & Beeler, 1998; Fuchs, Fuchs, Mathes & Simmons, 1987) for the majority of the intervention plans. Intervention plans were tailored to the child’s needs and negotiated with the teacher. For the most part, the teachers were faithful implementers of the plans and worked diligently to improve student performance. We need to understand more about this process: the critical elements of research-based general education interventions, the professional development necessary to insure faithful implementation, and the relationship to child outcomes. It is clear that different methods to identify LD are needed. The response to treatment paradigm is in the forefront of recommended approaches at the national level (e.g. Donovan & Cross, 2002; President’s Commission on Excellence in Special Education, 2002). This represents a sea change in how identification in the schools will be accomplished and represents a new frontier in the history of the learning disabilities field. The work presented in this chapter indicates there is good reason to pursue such models. It also indicates how much remains to be learned, especially in the realm of research to practice.
ACKNOWLEDGMENTS The work reported in this chapter was supported by Grant HO23F97008 from the U.S. Department of Education, Office of Special Education Programs. This
Starting at the Beginning for Learning Disabilities Identification
49
chapter is based on a forthcoming article: Speece, D. L., Case, L. P. & Molloy, D. E. Responsiveness to general education instruction as the first gate to learning disabilities identification. Learning Disabilities Research & Practice. Correspondence concerning this chapter may be addressed to Deborah L. Speece, Department of Special Education, 1308 Benjamin Building, University of Maryland, College Park, MD 20742. Electronic mail may be sent to
[email protected].
REFERENCES Adams, M. J., Foorman, B. R., Lundberg, I., & Beeler, T. D. (1998). Phonemic awareness in young children. Baltimore: Paul H. Brookes. Case, L. P., Speece, D. L., & Molloy, D. E. (in press). The validity of a response to instruction paradigm to identify reading disabilities: A longitudinal analysis of individual differences and contextual factors. School Psychology Review. Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64, 321–339. Deno, S. L. (1989). Curriculum-based measurement and alternative special education services: A fundamental and direct relationship. In: M. R. Shinn (Ed.) Curriculum-Based Measurement: Assessing Special Children (pp. 1–17). New York: Guilford Press. Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524. Donovan, S., & Cross, C. (2002). Minority students in gifted and special education. Washington, DC: National Academy Press. Fletcher, J. M., Shaywitz, S. E., Shankweiler, D. P., Katz, L., Liberman, I. Y., Stuebing, K. K., Francis, D. J., Fowler, A. E., & Shaywitz, B. A. (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement definitions. Journal of Educational Psychology, 86, 6–23. Fuchs, L. S. (1995, May). Incorporating curriculum-based measurement into the eligibility decisionmaking process: A focus on treatment validity and student growth. Paper presented at the Workshop on IQ testing and educational decision making, national research council, national academy of science. Washington, DC. Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation on student achievement: A meta-analysis. Exceptional Children, 53, 199–208. Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13(4), 204–219. Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1989a). Monitoring reading growth using student recalls: Effects of two teacher feedback systems. Journal of Educational Research, 83, 103–111. Fuchs, L. S., Fuchs, D., & Hamlett, C. L. (1989b). Effects of alternative goal structures within curriculum-based measurement. Exceptional Children, 55, 229–238. Fuchs, D., Fuchs, L. S., Mathes, P. M., & Simmons, D. (1987). Peer-assisted learning strategies: Making classrooms more responsive to student diversity. American Educational Research Journal, 34, 174–206. Fuchs, L. S., Fuchs, D., & Speece, D. L. (2002). Treatment validity as a unifying construct for identifying learning disabilities. Learning Disability Quarterly, 25, 33–46.
50
D. L. SPEECE, D. E. MOLLOY AND L. P. CASE
Jenkins, J. R., Mayhall, W., Peshka, C., & Townsend, V. (1974). Using direct and daily measures to increase learning. Journal of Learning Disabilities, 10, 604–608. Keogh, B. K., & Speece, D. L. (1996). Learning disabilities within the context of schooling. In: D. L. Speece & B. K. Keogh (Eds), Research on Classroom Ecologies: Implications for Inclusion of Children with Learning Disabilities (pp. 1–14). Mahwah, NJ: Lawrence Erlbaum. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749. President’s Commission on excellence in special education (2002). A new era: Revitalizing special education for children and their families. Retrieved July 10, 2002 from PCESE. Web site: http://www.ed.gov/inits/commissionsboards/whspecialeducation Share, D. L., McGee, R., & Silva, P. (1989). IQ and reading progress: A test of the capacity notion of IQ. Journal of American Academy of Child and Adolescent Psychiatry, 28, 97–100. Speece, D. L., & Case, L. P. (2001). Classification in context: An alternative approach to identifying early reading disability. Journal of Educational Psychology, 93, 735–749. Speece, D. L., Case, L. P., & Molloy, D. E. (2002). Effects of an early identification/general education intervention program on children at risk for reading failure. Manuscript in preparation. Speece, D. L., & Shekitka, L. (2002). How should reading disabilities be operationalized? A survey of experts. Learning Disabilities Research & Practice, 17, 118–123. Stanovich, K. E., & Siegel, L. S. (1994). Phenotypic performance profile of children with reading disabilities: A regression-based test of the phonological-core variable-difference model. Journal of Educational Psychology, 86, 24–53. U.S. Office of Education (1977). Definition and criteria for defining students as learning disabled. Federal Register, 42(250) (pp. 65083). Washington, DC: U.S. Government Printing Office. Vellutino, F. R., Scanlon, D. M., Sipay, E. R., Small, S. G., Chen, R., Pratt, A., & Denckla, M. B. (1996). Cognitive profiles of difficult-to-remediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of specific reading disability. Journal of Educational Psychology, 88, 601–638. Wechsler, D. (1974). Wechsler intelligence scale for children – revised. New York: Psychological Corporation. Wesson, C., Deno, S. L., Mirkin, P. K., Maruyama, G., Skiba, R., King, R., & Sevcik, B. (1988). A causal analysis of the relationships among ongoing curriculum-based measurement and evaluation, the structure of instruction, and student achievement. The Journal of Special Education, 22, 330–343. Woodcock, R. W., & Johnson, M. B. (1989). Woodcock–Johnson psychoeducational battery – revised. Chicago: Riverside Publishing Company.
IDENTIFICATION OF NON-RESPONDERS: ARE THE CHILDREN “LEFT BEHIND” BY EARLY LITERACY INTERVENTION THE “TRULY” READING DISABLED? Stephanie Al Otaiba ABSTRACT The primary purpose of this chapter is to synthesize the existing research that describes children who are unresponsive to generally effective early literacy interventions. Studies were selected in which: (a) children ranged from preschoolers to third graders and were at-risk for reading disabilities; (b) treatments targeted early literacy; (c) outcomes reflected reading development; and (d) students’ unresponsiveness to intervention was described. The search yielded 23 studies, eight of which were designed primarily to identify characteristics of unresponsive students; the remaining 15 studies focused on treatment effectiveness, but also identified and described unresponsive students. A majority of unresponsive students had phonological awareness deficits; additional characteristics included phonological retrieval or encoding deficits, low verbal ability, behavior problems, and developmental delays. Methodological issues are discussed that complicate comparisons of non-responders across studies. A secondary purpose of this chapter is to
Identification and Assessment Advances in Learning and Behavioral Disabilities, Volume 16, 51–81 Copyright © 2003 by Elsevier Science Ltd. All rights of reproduction in any form reserved ISSN: 0735-004x/doi:10.1016/S0735-004X(03)16003-X
51
52
STEPHANIE AL OTAIBA
describe findings from recent longitudinal studies that support the hypothesis that non-responders may be the truly reading disabled. Implications for future research and practice are discussed.
INTRODUCTION At the start of this chapter, I want to emphasize two points. First, not all poor readers have reading disabilities. That is to say, reading difficulties are not the same thing as reading disabilities. Reading disabilities are neurologically based, are pervasive across the life span, are not caused by a lack of opportunity to learn or by mental disability, and are difficult to remediate. By contrast, reading difficulties can be prevented and may occur primarily due to a lack of instruction (Vellutino et al., 1996). Second, many students with reading disabilities can be helped, but only through ongoing intervention, or treatment, which is more intensive than classroom instruction. Although 38% of all fourth grade students are reading below the basic level of reading (National Center for Educational Statistics, 1996), current estimates suggest that only 5% of school-age students are actually reading disabled. Nonetheless, reading difficulties are the most common reason for referral for special education (e.g. Mastropieri, Lenhart & Scruggs, 1999) because research has shown that reading trajectories are established early, and once established, are difficult to change (Good, Simmons & Kame’enui, 2001). This gap between poor and strong readers widens over the elementary years (Stanovich, 1986), particularly after third grade (Fletcher & Foorman, 1994; Kennedy, Birman & Demaline, 1986; Lyon, 1985). Even when poor readers continue to grow in reading, they rarely catch up to strong readers because poor readers develop a negative attitude toward reading (Juel, 1988) and, subsequently, practice reading dramatically less than strong readers (Allington, 1994). Specifically, poor readers may read as few words in one year as strong readers read in two days (Cunningham & Stanovich, 1998). Given rising expectations for literacy in our increasingly technological workplace, the well-documented long-term costs of early reading difficulties are a societal concern (e.g. Adams, 1990). Preventing reading difficulties has become an urgent national priority.
HELPING POOR CRACK THE CODE Over the last two decades, researchers have shown that most poor readers do not understand phonological awareness at the level of the phoneme: phonemic
Identification of Non-responders
53
awareness. Without understanding that words are made of individual sounds that can be blended, segmented, and manipulated, children do not understand how to apply phonics, or letter-sound strategies, to sound out words. In other words, they do not understand the system of linkage between sounds, or phonemes, and letters, or graphemes. If children cannot identify words, they cannot comprehend written material, which is, of course, the ultimate goal of reading (Share & Stanovich, 1995). Researchers have also shown that explicit and systematic instruction in phonemic awareness and phonics can help prevent reading difficulties for many children. Explicit instruction, in contrast with implicit or incidental teaching, follows a conscious and logical sequence from easier to more difficult. For example, teaching children to blend syllables like “bat” and “man” to make “batman” would happen before teaching that “bat” is made up of three sounds: /b/ /a/ and /t/. Systematic instruction aims to teach the alphabetic code, which is the system or the relationship between sounds in speech and letters. Two recent influential reports, Teaching Children to Read (National Reading Panel, 2000) and Preventing Reading Difficulties for Young Children (Snow, Burns & Griffin, 1998), summarize converging and convincing evidence that early literacy intervention that includes explicit and systematic instruction in phonological awareness and phonemic decoding, is more effective than implicit approaches. In addition to phonological awareness and phonics training, both reports also identify three additional components of effective early literacy instruction: vocabulary, fluency, and comprehension. Findings from these two reports, known collectively either as “ScientificallyBased Reading Research” (SBRR), or as “evidence-based instruction,” are cited throughout No Child Left Behind Act of 2001 (PL 107–110), especially in Part B, also known as Reading First. To achieve the stated goal of all children reading by third grade, all schools receiving funding through Reading First will be required to provide SBRR instruction and more intensive intervention which includes all five core training components (phonological awareness, phonics, vocabulary, fluency, and comprehension) in order to reduce the proportion of students inappropriately identified for special education. In other words, good classroom instruction is necessary, but may not be sufficient, for all students. Therefore, schools have a financial incentive for selecting reading instructional materials including basal reading curricular programs as well as supplemental intervention materials that are consistent with SBRR. This brings me to the second point: Remediation of reading disabilities is not impossible. With sustained individualized intervention, or treatment, beyond classroom reading instruction some older students can catch up to their peers. In recent years, researchers have conducted very intensive interventions that have successfully remediated reading problems for a number of older children
54
STEPHANIE AL OTAIBA
(Alexander, Anderson, Heilman, Voeller & Torgesen, 1991; Lovett, 1997). These studies demonstrated that it took more than 60 hours of focused one-to-one, intensive (i.e. one to two hours per day) intervention by highly trained reading teachers to help children catch up to their peers (as evidenced by standard scores on word identification tasks). However, even when their word identification problems were successfully remediated, these students still lagged behind in fluency, or oral reading rates, and continued to experience difficulty with reading comprehension (Torgesen, 2000). Fluency may be constrained by lack of motivation, lack of practice, lack of vocabulary, or simply by slower processing. Some have questioned whether such “heroic” interventions can be delivered in general education or in special education classrooms (see Fuchs & Fuchs, 1998; O’Connor, 2000). Unfortunately, special education programs are often unable to provide intensive and individualized reading intervention (Vaughn, Moody & Shumm, 1998). In special education, teachers’ caseloads are high because many children placed in special education are not truly reading disabled but rather instructional casualties (Simmons, Fuchs & Fuchs, 1991). In addition, the context of special education is changing. Under the Individuals with Disabilities Education Act (IDEA, 1997), an increasing number of students with reading disabilities are primarily served in the general education classroom. Moreover, neither special educators nor their general education peers reportedly feel prepared to teach reading to students with such diverse abilities (Moats & Lyon, 1996). A growing number of reading and special education researchers and educators have expressed concern that as many as 30% of children at-risk for reading difficulties (e.g. Blachman, 1994, 1997; Brown & Felton, 1990; Juel, 1994; Mathes, Howard, Allen & Fuchs, 1998; Shannahan & Barr, 1995; Smith-Burke & Jaggar, 1994; Torgesen, Morgan & Davis, 1992) and as many as 50% of children with special needs (e.g. O’Connor, Jenkins, Leicester & Slocum, 1993; O’Connor, Jenkins & Slocum, 1995) may not benefit from generally effective early literacy interventions. These students have been called “treatment-resistors” or “non-responders” (Blachman, 1994, 1997; Torgesen, Wagner & Rashotte, 1994). So, how many children are still being “left behind?” Torgesen (2000) recently estimated that only 2–6% of the school population might be unresponsive to intervention efforts. By Torgesen’s definition, treatment resistors are children who have had access to preventative programs, but who still have not acquired word reading skills and remain below the 30th percentile on Word Attack and Word Identification subtests of the Woodcock Reading Mastery Test-Revised (Woodcock; WRMT, 1987). If Torgesen’s estimates are accurate, they are very similar to federal incidence figures for children with reading disabilities.
Identification of Non-responders
55
Overview The purpose of this chapter is to describe the extant knowledge about the characteristics of children who do not respond to well-implemented and generally effective early literacy interventions. A further purpose is to examine two recent longitudinal studies and a long-term follow-up investigation that provide tentative evidence that students unresponsive to treatment are indeed reading disabled. In a “leave no child behind” world, this emerging area of research is particularly important because response to intervention, or treatment, offers an alternative to the IQ-achievement discrepancy-based formulas currently used to identify students with reading disabilities (Berninger & Abbott, 1994; Fuchs & Fuchs, 1998; Lyon et al., 2001; Vellutino et al., 1996). This alternative approach or paradigm is more proactive than the discrepancy paradigm because it does not require students to fall behind in order to qualify for help. In the primary, or prevention stage, of this response to intervention paradigm (see Fuchs & Fuchs, 1998), all students are guaranteed the opportunity to learn through good classroom reading instruction, reflecting use of SBRR. Then, in a secondary prevention stage, children who do not respond to instruction are identified for one or more intensive classroom interventions or treatments. During this secondary prevention stage, student progress would be monitored and diagnostic information gathered to tailor the intervention to meet the students’ needs. This stage is akin to the current pre-referral stage. Only those students who do not make adequate progress following good instruction and individualized intensive intervention would then be considered “truly reading disabled.” A response to intervention approach is consistent with IDEA (1997) because special educators and reading specialists would support classroom teachers in their prevention efforts. This approach is clearly more proactive than the current discrepancy approach. Children with reading disabilities would not have to wait for reading support until they fail. Currently many students do not qualify for special services until they fall far behind their expected reading achievement when remediation becomes more difficult (Adams, 1990). Moreover, special educators would have access to student progress monitoring reports during prevention, which may help them design more effective intensive interventions. Using the response to intervention approach could reduce current special education case-loads and allow more individualized reading instruction for the truly reading disabled (Vaughn, Moody & Schumm, 1998). In the next three sections of the chapter, I will present a summary of the findings from a recent review of the non-responder literature (Al Otaiba & Fuchs, 2002a). Next, I will present findings from two longitudinal studies that provide
56
STEPHANIE AL OTAIBA
tentative confirmation that non-responders eventually are referred for special education. Finally, I will discuss implications and a state-wide model to reduce the incidence of non-responders by translating SBRR research into policy and practice.
WHAT IS KNOWN ABOUT THE CHARACTERISTICS OF NON-RESPONDERS? Al Otaiba and Fuchs (2002a) conducted a thorough review to synthesize the literature describing children unresponsive to generally effective early literacy interventions. Because this is a relatively new area of research, the authors included not only the eight most recent studies that were designed primarily to describe non-responders (Berninger et al., 1999; Hatcher & Hulme, 1999; Schneider, Ennemoser, Roth & Kuspert, 1999; Torgesen & Davis, 1996; Torgesen et al., 1999; Uhry & Shepherd, 1997; Vellutino et al., 1996; Vellutino, Scanlon & Lyon, 2000), but also 15 studies that were designed primarily to explore treatment effectiveness that provided limited information about non-responders (Ehri & Robbins, 1992; Fazio, 1997; Foorman, Francis, Fletcher, Schatschneider & Mehta, 1998; Foorman, Francis, Winikates, Mehta, Schatschneider & Fletcher, 1997; Fox & Routh, 1976; Hurford, 1990; Kasten, 1998; O’Connor et al., 1993; O’Connor, Notari-Syverson & Vadasy, 1996, 1998a, b; O’Shaughnessy & Swanson, 2000; Peterson & Haines, 1992; Snider, 1997; Vadasy, Jenkins, Antil, Wayne & O’Connor, 1997; Vandervelden & Siegel, 1997). In all, 23 studies were selected in which: (a) (b) (c) (d) (e)
participants ranged in grade level from preschool to third grade; children were included who were at-risk for reading disabilities; interventions targeted early literacy; outcomes reflected reading development; and students unresponsive to treatment were described.
Table 1 describes study participants, the definitions of unresponsiveness, the percentage and characteristics of unresponsive students. Table 2 provides background information about types of interventions, their intensity, duration, and effectiveness. I want to emphasize that the findings from the 23 studies reviewed by Al Otaiba and Fuchs (2002a) indicated that early literacy interventions helped most students, including many students with disabilities. However, depending on the individual study, sample selection criteria, and outcome measure, between 8 and 80% of students made little or no improvement.
Article
Demographics
Intervention studies conducted to describe unresponsive students Berninger et al. M age = 7 yrs., 2% Black, 8% (1999) Hispanic, 2% Asian, 4% Native American, M Verbal IQ = 91.60 Hatcher and M age = 7 yrs., M IQ = 68–122 Hulme (1999) Schneider et al. M age = 5 yrs. 7 mos., German (1999) Torgesen and Age: 5–6 yrs., 73% Black, low Davis (1996) SES, M Verbal IQ approximately 91 Torgesen et al. Age: 5–6 yrs., 26% Black, 2.1% (1999) Other, Verbal IQ > 75 Uhry and Shepherd (1997)
Age: 5–8 yrs., 17% Black, middle SES, IQ > 90
Vellutino et al. (1996, 2000)
Age: 5–8 yrs., Mostly white, middle SES, IQ > 90
Definition and Percentage of Unresponsive Students
Characteristics of Unresponsive Students
Growth slopes not different from 0. No growth on words trained: 52%; no growth on WRMT-R WA: 63%; no growth on WRMT-R WI: 75%. Not defined; no percentage reported.
Low PA, slow RAN, poor orthographic skills, and low verbal IQ. Low PA.
No gains during treatment, no percentage reported.
Slow RAN among students with lowest PA. Poor invented spelling, slow RAN, and low verbal ability.
No gains during treatment, segmenting: 30%; blending: 10%.
WRMT-R standard score <85. On the WA, WI and PC subtests, respectively: controls: 53%, 53%, 56%; PASP: 24%, 21%, 36%; EP: 47%, 28%, 47%; RCS: 44%, 31%, 56%. No gain during treatment on WRMT-R WI: 8%.
Students with lowest growth slopes on WRMT-R, WI and WA from K., fall of second grade: 26%. No improve beyond thirties percentile on WRMT-R, WI and WA: 33%.
Low PA, slow RAN, low SES, poor classroom behavior ratings, and poor phonological memory. Low PA, slow RAN, poor phonological memory, and poor attention. Low segmentation, slow RAN, poor phonological memory, and poor attention. No segmentation skills.
No segmentation skills. Poor segmentation. Low PA.
57
Studies exploring treatment effectiveness: Beginning readers without disabilities Ehri and Robbins Age: 5–7 yrs., middle SES Unable to read transfer words: 80% of students who could not (1992) segment and blend nonsense words; 0% of students who could segment. Fox and Routh (1976) M age = 4 yrs., middle SES, Did not improve during treatment on decoding: 50% overall; M IQ =112 100% of those who could not segment. Peterson and Haines Age: 5–6 yrs., PPVT > 85 Did not significantly improve word-reading: 33% overall; (1992) 100% of poor segmenters. Vandervelden and Age: 5–7 yrs., low SES No gains in PA: 13% overall; 18% of students with low PA. Siegel (1997) Could not read more than one word: 27% overall; 36% of students with low PA.
Identification of Non-responders
Table 1. Child Characteristics.
58
Table 1. (Continued ) Article
Demographics
Definition and Percentage of Unresponsive Students
Studies exploring treatment effectiveness: Pre-literate readers with disabilities Fazio (1997) Age: 4–6 yrs., M non-verbal IQ: Difficulty learning and recalling a rhyming poem: percentage 85–115 not reported. Kasten (1998) M age = 5 yrs., low SES, White Did not display significant reading growth on WJ-R subtests. No percentage reported. O’Connor et al. (1993) Age: 4–6 yrs. Did not learn to identify rhyming oddities: 8%. Did not learn to blend onset-rime: 36%; Did not learn to segment first sound: 46%. O’Connor et al. (1996, Age: 5–7 yrs., 56% Black, Made less than half the mean gain in PA: general education 1998a, b) M Verbal IQ = 67 students, 18%; students with disabilities, 33%; mild mental retardation, 66%; learning disabilities, 38%; behavior disorders: 50%. Studies exploring treatment effectiveness: Older students Foorman et al. (1997) Age: 7–9 yrs., 32% Black, 24% low SES, Verbal IQ > 79 Foorman et al. (1998) Grades 1–2, 60% Black, 20% Hispanic, low SES Age: 7–9 yrs., IQ > 90
Snider (1997)
Age: 7–9 yrs., 10% low SES
O’Shaughnessy and Swanson (2000)
M age = 7 yrs., 8 mos., 4.4% Black, 2.2% Asian, 28.9% Hispanic, low SES, M IQ = 89.9 Age: 5–8 yrs., 50% low SES
Vadasy et al. (1997)
Learned fewer than 2.5 words on a 50 word list: Implicit code, researchers, 46% implicit code, teachers, 38% embedded code, 44%; and direct code, 16%. Post-treatment segmentation skills are poorer than students without disabilities. No percentage reported. Did not significantly improve reading rate and accuracy on oral reading fluency: 10%. Did not significantly improve rate and accuracy on oral reading fluency: phonological training, 20%; word analogy training, 27%. Gained less than 8 points on the Reading and Spelling subtests of the WRAT-R: 35%.
Low PA and poor verbal ability. Low IQ. Low PA.
Low PA and low IQ.
Low PA poor spelling, Spanish ethnicity, and low verbal IQ.
Low PA and younger children. Poor attention. Poor attention.
Poor attention.
Notes: WRMT-R = Woodcook Reading Mastery Tests-Revised (Woodcooks, 1989). WA = Word Attack; WI = Word Identification; PA = Phonological Awareness; RAN = Rapid Automatized Naming; SES = Socioeconomic Status; PPVT = Peabody Picture Vocabulary Test; WJ-R = Woodcock–Johnson-Reading subtests; WRAT-R = Wide Range Achievement Test-Revised.
STEPHANIE AL OTAIBA
Hurford (1990)
Not defined, no percentage reported.
Characteristics of Unresponsive Students
Article
Treatment Description
Intervention studies conducted to describe unresponsive students Berninger et al. Graduate students trained children to read words (1999) using whole word instruction, phonemic decoding, or whole word + phonemic decoding. Hatcher and Hulme Teachers (not the children’s) trained students (1999) using either phonological awareness training (P), Reading Recovery (R), or both (P + R).
Schneider et al. (1999)
Classroom teachers trained their own students using phonological awareness training.
Torgesen and Davis (1996)
Graduate students conducted rhyming, blending, and segmenting training.
Torgesen et al. (1999)
Research staff and instructional aides trained students in phonological awareness + synthetic phonics (PASP) or embedded phonics (EP). A third group received regular classroom support (RCS). Graduate students trained students in letter – sound correspondence, segmentation and guided reading in phonics-controlled texts and narrative texts and writing. Research staff (certified teachers) trained students in phonemic awareness, phonetic decoding, reading in connected text, and writing.
Uhry and Shepherd (1997)
Vellutino et al. (1996, 2000)
Treatment Intensity, Duration, and Fidelity
Treatment Effectiveness
Individual tutorials in eight 30 min sessions. Total = 4 hrs. No fidelity reported.
All three groups made significant growth.
Individual tutorials in forty 30 min. sessions for 20 weeks. Total = 20 hrs. No fidelity reported.
Whole classroom instruction conducted daily in ten 15 min sessions for 6 mo. Total = 20 hrs. No fidelity reported. Small group format in four 20 min sessions per week for 3 mo. Total = 16 hrs. No fidelity reported. Individual tutorial in four 20 min weekly sessions (two with staff and two with instructional aide). Total = 88 hrs. No fidelity reported.
P group made greater gains on phonological skills than other groups. Only P +R group made greater reading gains than controls and differences continued at 9-mo. follow-up. Treatment students outperformed controls on phonological awareness at posttest, and on reading at end of Grade 2. Treatment students made significantly more growth in blending and segmenting than control students. PASP group scored higher than the other three groups on WRMT-R, WA, and WI; PASP group also scored higher than controls and RCS on WRMT-R, PC.
Individual tutorial two 1 hr sessions per week for 5 mos. Total = 32 hrs. No fidelity reported.
Most students made significant growth in WRMT-R, WI and WA.
Individual tutorial five 30 min sessions per week for 1–2 semesters. Total = 35–40 hrs. No fidelity reported.
67% of tutored students improved beyond lowest 30th percentile on WRMT-R, WI and WA.
Identification of Non-responders
Table 2. Treatment Characteristics.
59
60
Table 2. (Continued ) Article
Treatment Description
Treatment Intensity, Duration, and Fidelity
Studies exploring treatment effectiveness: Beginning readers without disabilities Individual tutorial in four 15 min sessions Ehri and Robbins Research staff trained experimental students to across 1 mos. Total = 1 hr. No fidelity (1992) read words by analogy (e.g. cave, save) and reported. trained control students to read unrelated words (e.g. rain, save). Individual tutorial in two 30 min sessions for Fox and Routh Research staff trained students to read short (1976) words by blending two sounds together (e.g. 1 week. Total = 1 hr. No fidelity reported. me). Peterson and Research staff trained experimental students to Individual tutorial in seven 15 min sessions Haines (1992) read words by analogy (e.g. /b/ /all/, /ball/; /f/ across 1 mos. Total = 2 hrs. No fidelity /all/, /fall/). reported. Vandervelden and Research staff trained students in phonological Individual or small group training. In one Siegel (1997) awareness and reading and spelling using initial 30–45 min session per week. For 3 mos. sounds and rimes. Total = 6–9 hrs. No fidelity reported.
O’Connor et al. (1993) O’Connor et al. (1996, 1998a, b)
Research staff trained students in either rhyming, blending, or segmenting using direct instruction. Classroom teachers trained their students in phonological and print awareness.
Significant differences favored experimental students on word reading.
Significant effects on decoding for experimental students who could segment. Significant growth in decoding for experimental students with mid- to high-level segmentation skill. Significantly greater growth on phonological awareness and reading for experimental students.
Individual tutoring in four 15 min sessions across 1 mos. Total = 1 hr. No fidelity reported.
Children with speech or language impairments in the hand motions condition outperformed children in other conditions.
3-yr duration. No total reported. No fidelity reported.
Student’s scores on Reading Miscue Inventory showed he was becoming a developing reader. Experimental students made significantly more growth on rhyming, blending, and segmenting. Experimental students with and without disabilities outperformed controls on blending, segmenting, and word reading on WRMT. Effects continued to favor experimental students with disabilities 1-yr after training.
Small group training four 10 min sessions per week for 7 weeks. Total = 4.5 hrs. No fidelity reported. Whole classroom instruction 100–281 sessions over 6 mos. No total reported. No fidelity reported.
STEPHANIE AL OTAIBA
Studies exploring treatment effectiveness: Preliterate children with disabilities Fazio (1997) Research staff taught children to recall and retell a poem using direct instruction under four conditions: with or without hand motions and with or without singing the poem (phonological encoding). Kasten (1998) Phonics training in resource room setting; whole language classroom instruction.
Treatment Effectiveness
O’Shaughnessy and Swanson (2000) Vadasy et al. (1997)
Instructional aides conducted either phonological awareness and phonemic decoding training (PAT), word analogy training (WAT), or math training (MAT). Community volunteers trained students on phonological awareness, letter – sound correspondence, phonogram activities with magnetic letters, reading, and writing.
Whole classroom instruction 1 hr daily for 6 mos. Total = 120 hrs. No fidelity reported.
Students in synthetic phonics condition performed better than students in sight word condition in segmentation and reading.
Whole classroom instruction was 90 min daily; small group or individual tutorial was 30 min daily. 6 mos. Duration No fidelity reported.
Positive effects favored the direct code group on WJ-R Basic Skills and Passage Comprehension subtests.
Sessions were 30–45 min per day for 3–4 days. Total = 2.5–4 hrs. No fidelity reported.
Experimental students displayed significant improvement in segmentation.
Grouping format not reported. 30–45 min. daily across 9 mos. Total = 90 hrs. No fidelity reported. Small-group format. Three 30-min sessions across 6 weeks. Fidelity reported (94–97%).
Students improved their reading rate and accuracy on oral reading fluency.
Individual tutorial for 100 sessions across 6 mos. Total = 54 hrs. No fidelity reported.
Identification of Non-responders
Studies exploring treatment effectiveness: Older children Foorman et al. Resource room teachers taught their students (1997) using synthetic phonics (multi-sensory and systematic at the phoneme level), analytic phonics (implicit and at the onset-rime level), or sight word (whole word) approaches. Foorman et al. Classroom teachers used either direct code, (1998) embedded code, or implicit code. Research staff also conducted an implicit code training group. In addition, Title 1 teachers provided students with additional tutoring that was implicit. Hurford (1990) Research staff trained students to use a computer program that tried to help students discriminate between phonemes (e.g. /ri /li/). Snider (1997) Students’ resource room teacher used direct instruction.
PAT & WAT groups outperformed the MAT group on PA, phonological memory, and on word attack (WRMT-R). Effect sizes for segmentation and reading on WRMT favored experimental students.
Notes: WRMT-R = Woodcock Reading Mastery Test-Revised (Wookcock, 1987). WA = Word Attack; WI = Word Identification; PC = Passage Comprehensive; WJ-R = Woodcook–Johnson-Reading subtest.
61
62
STEPHANIE AL OTAIBA
Variation in Treatment-related Characteristics Complicates Interpretation Table 2 demonstrates the extreme variation in treatments from one investigation to another along a number of dimensions, including duration, intensity, and background of trainers. In seven studies, treatments were brief (1–9 hours), in seven additional investigations, the duration was longer (20–55 hours); and in five studies, treatments ran for more than 80 hours (up to three years). Although more than half of the investigators conducted one-to-one tutorials, researchers in five studies used small groups (Fazio, 1997; O’Connor et al., 1993; O’Shaughnessy & Swanson, 2000; Torgesen & Davis, 1996; Vandervelden & Siegel, 1997), and in six studies, they used whole-classroom formats (Foorman et al., 1997; Kasten, 1998; O’Connor et al., 1996, 1998a, b; Schneider et al., 1999; Snider, 1997). Whereas most trainers were graduate students or research staff, O’Shaughnessy and Swanson used paraprofessionals and Vadasy et al. (1997) used community volunteers as trainers. In only four studies (Foorman et al., 1997; Kasten, 1998; O’Connor et al., 1996; Snider, 1997) special educators conducted the treatments, and general educators conducted the treatment in only one study (Schneider et al., 1999). This small number of interventions conducted by teachers in “real classrooms” undermines generalizability of the findings. Furthermore, the large variation in treatment-related characteristics complicates interpretations of the 23 studies as a group. Consequently, the percentage of non-responders may possibly be related to treatment duration, treatment intensity, or the background or skill of the trainers rather than to the characteristics of unresponsive students. In addition, only one of the 23 studies reported fidelity of treatment data (see Table 2). Without information detailing how faithfully treatments were conducted, we cannot assume that treatments were responsible for observed positive change in reading behavior.
A Closer Look at a Subset of Eight of the Twenty-three Studies To provide further information about possible relationships between treatmentrelated issues and the percentage of non-responders, a subset of eight studies in which interventions lasted more than 10 hours and that reported the percentage of non-responders was selected (Foorman et al., 1998; O’Connor et al., 1996; Snider, 1997; Torgesen & Davis, 1996; Torgesen et al., 1999; Uhry & Shepherd, 1997; Vadasy et al., 1997; Vellutino et al., 1996). These studies were used to explore the relationship between percentage of non-responders and four treatment-related issues: (a) who implemented treatment; (b) the total hours of treatment; (c) the explicitness of the treatment; and (d) grouping size. First, each of these
Identification of Non-responders
63
four variables was coded and entered into a database. Studies with more than one treatment were coded separately. The only treatment-related issue that was statistically significantly correlated with the percentage of non-responders was the explicitness of treatment (r = 0.80 and p = 0.000). The mean percentage of non-responders in implicit approaches was 43.80% and, in explicit approaches, this percentage was 28.00%. These finding underscores the importance of providing explicit evidence-based interventions. Child Characteristics of Non-responders A preliminary examination of Table 3 shows the seven clusters of child characteristics that were associated with unresponsiveness to treatment: (a) (b) (c) (d)
phonological awareness; phonological memory; rapid naming; general intelligence (note that this category subsumes verbal ability measured by PPVT-R); (e) attention and behavior problems; (f) orthographic awareness; and (g) demographic information (e.g. SES, parent education). Most notably, all but two teams of investigators explored the relationship between phonological awareness and treatment unresponsiveness, and in 70% of the studies phonological awareness was found to be a clear and important correlate. The importance of intelligence to treatment responsiveness is less clear: 22% of researchers reported a relationship, but 30% did not. This relationship, however, has only been explored within a limited range of intelligence (i.e. >75), with five Table 3. Number of Studies in which Learner Characteristics were Significantly Associated with Non-responding Students. Findings
Yes No Mixed
Phonological Awareness (n = 21)
Phonological Memory (n = 7)
Rapid Naming (n = 7)
IQ or Verbal Ability (n = 15)
Attention (n = 9)
Orthography or Spelling (n = 7)
Demographics (n = 4)
16 1 4
4 2 1
5 1 1
5 7 3
7 2 0
3 4 0
2 0 2
Notes: The “phonological awareness” column should be read this way: In 16 of 21 studies exploring the importance of phonological awareness to treatment non-responsiveness, a statistically significant relationship was obtained; in one study, the relationship was non-significant; in four studies, results were mixed; and in studies (23 − 21 = 2), the researcher did not explore the relationship. Total number of studies + 23. Yes = Statistically significant relationship. No = Non-significant relationship. Mixed = Combination of significant and non-significant relationships.
64
STEPHANIE AL OTAIBA
notable exceptions (Kasten, 1998; O’Connor et al., 1993, 1996, 1998; Snider, 1997). All reported high frequencies of unresponsiveness, and among O’Connor et al.’s students with mild mental retardation, the percentage of non-responders reached 66%. Because the language ability of students in these studies was not reported, it is impossible to know whether treatment unresponsiveness was related specifically to low verbal ability or to a more general developmental disability. The connections among the remaining five characteristics and treatment unresponsiveness have been explored even less consistently. Sixty percent of research teams did not address the importance of phonological memory, and 70, 61, 70, and 80% of the research teams did not explore rapid naming, attention or behavior, orthographic processing, or demographics, respectively (see Table 3). No study provided a direct test of the dual (or multiple) deficit hypothesis (Wolf & Bowers, 1999), which posits that students with combined deficits are more likely to be unresponsive than students with a single deficit. Thus, although there is suggestive evidence of the importance of this last set of characteristics, future research is obviously needed. Limitations of the Current Non-responder Database An even closer inspection of Tables 1–3 show additional concerns about the current knowledge base. First and foremost, no common definition of the construct treatment unresponsiveness emerged from this review. Currently, children may be termed unresponsive because they make no growth, low growth, or because they do not catch up to average-achieving peers. This lack of an agreed-upon absolute standard or criterion for what it means to be a non-responder makes it difficult to make meaningful comparisons across studies and limits researchers’ ability to guide policy and practice. I would like to illustrate these differences in criteria by contrasting the definitions used in two of the studies. Torgesen et al. (1999), as previously described, defined unresponsiveness as standard scores below 85 on word attack and word identification subtests of the WRMT-R. Defining unresponsiveness in terms of performance level may have an important drawback, especially for very low performing students because performance level is insensitive to students’ growth. For example, it may not be realistic to expect a first grade student with mental disabilities to catch up to her typically-developing peers, or to have a standard score above 85 on the WRMT-R. However, it is realistic to expect her to show progress, for example, to show improvement on her reading goals on her IEP (e.g. decoding a word like “cat” or recognizing letter-sounds, or sight words). O’Connor et al. (1996, 1998a, b), on the other hand, defined unresponsiveness as growth of less than half the mean gain of treatment students. Defining unresponsiveness exclusively in terms of growth may also be problematic. Higher
Identification of Non-responders
65
performing students may show little growth, but still demonstrate acceptable performance levels; lower performing students, including many with disabilities, may make relatively impressive growth, but have unacceptably low performance levels. When it comes to identifying non-responders, policy makers and educators will need a more meaningful approach to defining unresponsiveness that provides valid benchmarks that relate to future reading performance (e.g. success or failure). A second concern apparent upon close inspection of Table 1 is that participants were drawn from very different sampling procedures and are, therefore a heterogeneous group. These differences make it difficult to compare proportions and to compare the characteristics of non-responders across studies. Five investigations (Foorman et al., 1997; Hurford, 1990; O’Shaughnessy & Swanson, 2000; Snider, 1997; Uhry & Shepherd, 1997) included students with reading disabilities. Five additional studies included children with speech and language impairments (Fazio, 1997), cognitive delays, or behavior disorders (Kasten, 1998; O’Connor et al., 1993, 1996, 1998a, b). As previously reported, children with disabilities, had much higher percentages of unresponsiveness than typically-developing peers (see also Fuchs, Fuchs & Thompson et al., 2002). Further, not all researchers specified their “cut-scores” for at-risk criteria; those who did reported a range from the lowest 35th percentile to the lowest 10th percentile of the distribution on either reading or phonological awareness outcomes. Moreover, these percentiles were based on study samples, not on normative populations. Thus, the lowest 10th percentile from the respective samples of two studies could define different groups of children. To date, the knowledge base about non-responders is still emerging. Recent research has shown that with explicit phonemic awareness and phonics training, many, but still not all, children can be helped to “catch up” to peers in terms of word recognition. The next section of this chapter will describe longitudinal investigations that provide tentative confirmation that children who do not catch up are indeed reading disabled.
ARE NON-RESPONDERS THE “TRULY” READING DISABLED? The findings from two recent longitudinal studies and a long-term follow-up investigation extend the current data base about non-responders. The purpose of O’Connor’s (2000) study was to determine whether the percentage of nonresponders could be reduced by providing layers of intervention that increased in intensity over kindergarten and first grade. Al Otaiba (2001) tracked nonresponders for two years, across kindergarten and first grade, and then conducted
66
STEPHANIE AL OTAIBA
a follow-up study (Al Otaiba & Fuchs, 2002b) at the end of third grade. Together, the objectives of these two studies were: (a) to compare the percentages of non-responders to implicit classroom instruction vs. explicit intervention; (b) to examine the child characteristics of non-responders in order to develop a tentative set of markers; and (c) to determine how well non-responder status predicted school-identified reading difficulties by third grade.
O’Connor (2000) The participants in O’Connor’s study were 189 children, 59 of whom were considered at-risk for reading difficulties. These at-risk students named less than 15 letters and less than four segments correctly in one minute, and scored below 86 on the combined letter-word and dictation subtests of the Woodcock Johnson (Woodcock & Johnson; WJ, 1990). By January of kindergarten, non-responders were identified who still scored below 86 on the WJ literacy subtests (letter-word identification and dictation) and who made limited growth on segmentation and letter-naming. In first grade, O’Connor adapted the non-responder criteria by adding the WJ word attack subtest, eliminating rapid letter naming, and using limited growth on blending and segmenting. During Layer 1, classroom teachers conducted whole-class explicit phonological awareness instruction using the Ladders to Literacy activities (O’Connor et al., 1998a, b). Participating students included children with and without disabilities. In Layer 2, the 25 kindergarten students who did not respond to Layer 1 received more intensive intervention in one-to-one tutorials. These 12-minute sessions were conducted three times a week for three months. In Layer 3, which began in first grade, 20 students were identified who either responded poorly to Layer 2 or whose standard scores on the Woodcock Johnson reading measures fell below 86. Then, these students received 14 weeks of small group instruction (four 30-minute sessions per week) that continued phonological training but also included explicit phonetic blending and decoding. In Layer 4, children were tutored individually by O’Connor herself for four weeks (four 15-minute sessions per week) using a similar intervention. By the end of second grade, all six of the non-responders to this fourth layer of intervention were identified for special education. O’Connor (2000) reported that 7% of children overall were non-responders. Among children with disabilities, this percentage was 30%. Only 1% of the initially at-risk students performed in the normal range of the WJ subtests without
Identification of Non-responders
67
receiving O’Connors’s interventions. Moreover, O’Connor reported that across the two years, some children responded well to intensive treatment, but when they returned to regular classroom instruction, they lost ground. These findings suggest the need for ongoing and sustained intensive intervention (especially for children with disabilities) and the need to track children’s progress over time. She also cautioned that although the school reported that as a result of classroom instructional changes the proportion of children in the lowest quartile on the high-stakes Iowa Test of Basic Skills dropped from 40 to 17%, the number of children identified as needing special education for reading problems did not decrease. O’Connor did not focus on the characteristics of her non-responders.
Al Otaiba (2001) and Al Otaiba and Fuchs (2002b) These studies were part of a larger investigation conducted in a southern metropolitan school district that was designed to examine the two-year effects of early literacy intervention on the reading acquisition of 312 children (see Fuchs et al., 2001, 2002). The students’ teachers in kindergarten and first grade were assigned randomly to treatment conditions; therefore, students received one of the following: (a) (b) (c) (d)
two years of treatment; only one year of treatment in kindergarten; only one year of treatment in first grade; or typical classroom instruction with no additional treatment.
All teachers used the district adopted grade-level basal reading programs from Harcourt Brace (e.g. Farr & Strickland, 1995). These basal reading programs took an implicit approach to phonological awareness and to phonics instruction (Stein, Johnson & Gutlohn, 1999). By contrast, the kindergarten treatments were explicit and systematic interventions. They consisted of either teacher-directed phonological awareness activities derived from Ladders to Literacy (O’Connor, Notari-Syverson & Vadasy, 1998a) or a combination of these teacher-directed activities and peer-mediated phonological awareness and decoding practice (Peer-Assisted Learning Strategies for Kindergarten; K-PALS). K-PALS sessions were conducted three times per week for 16 weeks. Each session lasted approximately 30 minutes. The first grade treatment was also explicit and peer-mediated. It consisted of Peer-Assisted Learning Strategies (PALS), which was conducted three times a week for approximately 20 weeks. First-Grade PALS lessons include phonological awareness,
68
STEPHANIE AL OTAIBA
decoding, and sight word training as well as reading short stories (i.e. reading connected text). Identification of Non-responders At the end of kindergarten, unresponsiveness to kindergarten treatment was defined as performing in the lowest 30th percentile of treatment students in terms of amount of growth on letter-sound and segmentation fluency measures. The Rapid Letter-Sound test (RLS) from the work of Levy and Lysynchuk (1997) was adapted to assess the number of letter-sounds a student named correctly in one minute. Students were shown a sheet with lower case letters and asked to tell each letter sound. The Yopp-Singer segmentation test (1994) required children to say the sounds in words (e.g. say the sounds in cat). This test strongly correlated with the Dynamic Indicators of Beginning Literacy Skill (DIBELS, Kaminsky & Good, 1996) phoneme segmentation fluency measure (r = 0.77; Good, Simmons & Kame’enui, 2001). None of the unresponsive students could segment more than 12 phonemes in one minute or identify more than 11 letter-sounds per minute. Recently, Good, Simmons and Kame’enui (2001) reported that students who named fewer than 10 segments per minute on the DIBELS phoneme segmentation fluency measure at the end of kindergarten were at-risk for poor reading outcomes in first grade. Unresponsiveness to treatment in first grade was also defined in terms of fluencyoral reading fluency. However, evaluating students’ growth on this measure was not possible because oral reading fluency measures are too difficult in fall of first grade. Therefore, a criterion or benchmark of 40 words per minute was selected because reading at or above 40 words per minute on unseen first grade text predicts good future reading (Good et al., 1999). Oral reading fluency on text was assessed by having students read aloud for one minute on three grade-level reading passages. To calculate the number of words read correctly in one minute, examiners averaged the number of correct words read across the three samples. Test-retest reliability on oral reading fluency measures range from 0.93 to 0.96 (Fuchs, Deno & Marston, 1983). By contrast, students were identified as either sometimes or always responsive if they were responsive to one or to both years of treatment, respectively. Students were considered responsive to kindergarten treatment whose growth on the segmentation and rapid letter sound fluency measures was at or above the treatment mean and were considered responsive to first grade treatment if their oral reading fluency was at or above the treatment mean. What Proportion of Students were Non-responders? Of the 312 students in the larger study, 132 students met either the unresponsive or the responsive criteria. However, 28 students were lost to the study because
Identification of Non-responders
69
they moved, leaving a total of 104 participants. At the end of first grade, these 104 students were classified into three groups in terms of their responsiveness to kindergarten and first grade treatment: never responsive, sometimes responsive, and always responsive. Typical classroom control students were also categorized according to the same unreponsiveness/responsiveness criteria used for treatment students in order to compare proportions of non-responders across conditions. Of the students who received one or two years of treatment in the context of the larger study, only 7.05% were never responsive to treatment. Recall that this is the same percentage of non-responders reported by O’Connor (2000). A higher proportion (i.e. 25.35%) of the students in the typical classroom control group, by contrast, was never responsive. This finding supports Vellutino et al. (1996) assertion that many children who appear to have reading difficulties may not have received adequate instruction or practice on important pre-reading skills. However, among children with disabilities (mostly speech or language disorders) the proportion of never responsive students was higher. Twenty-seven percent of children with disabilities were non-responders, slightly lower than the 30% O’Connor (2000) reported, but still much lower than the 67% of non-responders in the typical classroom control condition. While these findings should be interpreted with caution due to the small sample size, they do suggest that children with speech and language disorders who are unresponsive to kindergarten treatment may require more intensive intervention than whole class phonological training or peer-mediation. This is congruent with findings from a best evidence synthesis of the literature on the efficacy of peer tutoring for students with high-incidence disabilities (Mathes & Fuchs, 1994). Mathes and Fuchs (1994) reported that although peer tutoring was more effective than typical reading instruction, it was less effective than one-to-one teacher tutoring or teacher-led small group explicit interventions. Children unresponsive in kindergarten rarely changed status in first grade. Most (i.e. 91.89%) children, and all children with IEPs who were identified as unresponsive in kindergarten, remained unresponsive in first grade. Thus, it appears that unresponsiveness to treatment is relatively stable. This finding is consistent with Vellutino et al. (1996) suggestion that unresponsiveness to early intervention may be an important indicator of reading disability. Measures Selected to Describe Characteristics of Non-responders Measures were included to reflect characteristics associated with unresponsiveness by previous research (Al Otaiba & Fuchs, 2002a, b). Fidelity of teacher and student implementation was also measured to determine whether differences
70
STEPHANIE AL OTAIBA
in how accurately treatments were implemented affected responsiveness to treatment. Phonological encoding was evaluated using the Word Sequences and Sentence Imitation of the Detroit Tests of Learning Aptitudes-3 (Hammil, 1991; DTLA-3) in order to assess two types of phonological encoding: words that are unrelated (e.g. cold, late, full) and words that are in the context of a sentence (e.g. I saw a fire on the way to school). Phonological discrimination was evaluated using the Word Discrimination subtest of the Test of Language Development 2-Primary (Hammil & Newcomer, 1988; TOLD-P:2). In this subtest, students are asked to tell whether pairs of words are the same or different (e.g. chop-shop). For naming speed, the Rapid Letter Naming Test (RLN) was used to determine the number of letters a child can name in one minute, which is a strong predictor of future reading ability (Adams, 1990; Juel, 1988). In this test, students are shown upper and lower case letters and asked to name them as quickly as they can. Three types of verbal ability were evaluated. General verbal ability was measured with the Peabody Picture Vocabulary-Revised (Dunn & Dunn, 1981; PPVT-R), which is a widely-used individually administered, norm-referenced test of receptive vocabulary. Syntactic knowledge was also included because students unresponsive to treatment were differentiated on syntax in at least one study. In the Grammatic Completion subtest of the TOLD-P:2 (Hammil & Newcomer, 1988; TOLD-P:2), students are asked to supply the missing word in a sentence (e.g. John likes to cook every day. Yesterday, he . “cooked”). The Achenbach Child Behavior Checklist (Achenbach, 1994) was administered by the classroom teacher to address students’ attention and classroom behavior. Teachers also reported if students were receiving special education services, in which case students’ records were examined for goals on students’ Individualized Education Plans (IEPs). Which Characteristics Differentiated Non-responders? Findings from a discriminant function analysis suggest that the variables that reliably differentiated never, sometimes, and always responsive students were: naming speed, attention/behavior, verbal ability, and phonological encoding. This combination of markers, in conjunction with access to treatment, correctly identified 82.4% of the non-responders. Although these markers correctly identified most never responsive students, 17.6% of students were not predicted to be never responsive using these markers. These false negatives included four never responsive students predicted to be sometimes responsive and one student predicted to be always responsive. Thus, such students would not have been identified as likely to need more intensive treatment. In addition, the markers identified eight students as never responsive who actually were sometimes responsive and four students as
Identification of Non-responders
71
never responsive who were, in fact, always responsive. Practical implications of such false positives would be of less concern, however, than false negatives. Such children might receive more intensive treatment (conceivably at a higher cost) than needed. There is an important caveat: Non-responders were more often in kindergarten classrooms where teachers had lower ratings on instructional adaptations (adapting level of difficulty for students or allowing lower-performing students to respond) and in first grade classrooms where PALS lessons were conducted with lower treatment fidelity. These issues raise concerns about whether students were unresponsive due to poor treatment fidelity. Moreover, a large proportion of never responsive students were in the classroom control group. No data were collected regarding the quality or type of classroom instruction received by these students. The characteristics that distinguished non-responders from responders were relatively consistent with other investigations and with current reading theory. However, these studies’ findings may extend the literature because classroom teachers, rather than research staff, conducted treatment in their own real-world (diverse) classrooms. This is important given that the general education is the default placement for increasing numbers of young children with disabilities as a result of the Individuals with Disabilities Education Act (IDEA). Other researchers who provided clinical or pull-out interventions have also reported that non-responders were characterized by naming speed deficits (e.g. Berninger et al., 1999; Torgesen & Davis, 1996; Torgesen et al., 1999; Uhry & Shepherd, 1997; Vellutino et al., 1996). This finding is also consistent with the theory that deficits in naming speed may interact with other child characteristics to produce intractable reading disabilities (e.g. Wolf, 1991). Similarly, poor attention and conduct have been attributed to non-responders (Snider, 1997; Uhry & Shepherd, 1997; Vadasy et al., 1997; Vellutino et al., 1996). General verbal ability also differentiated non-responders from responders. Prior findings with regard to the impact of general verbal ability on responsiveness to treatment have been equivocal. On the one hand, several investigations of diverse populations reported that low general verbal ability was associated with treatment unresponsiveness (e.g. Fazio, 1997; Foorman et al., 1997; Torgesen & Davis, 1996). However, Vellutino et al. (1996) did not find PPVT-R, general verbal intelligence, or semantic deficits to be related to treatment responsiveness with mostly Caucasian and middle class participants. Thus, for children from lower income and multicultural families, verbal ability may be associated with responsiveness to treatment. Other researchers (e.g. Adams, 1990; Wolf & Bowers, 1999) have suggested that vocabulary may play a role in reading fluency as it is easier to decode familiar rather than unfamiliar words.
72
STEPHANIE AL OTAIBA
In comparison with responders, non-responders had more difficulty with remembering words in sentences, or phonological encoding. Children unable to recognize and make use of syntactic information in oral language would likely fail to use syntactic or grammatical context clues while reading text, which in turn may have implications for their reading fluency. On the other hand, there were no significant differences among groups on phonological encoding of abstract words. Thus, it appears that students’ memory span for words in abstract, or unrelated, lists did not differentiate responsiveness to treatment, whereas their memory for sentences did. This finding differs from Vellutino et al. (1996), who reported differences between responsive vs. unresponsive students using a measure that assessed immediate and delayed recall of abstract words. Nor were there any significant differences among never, sometimes, and always responsive students on demographic variables such as race, age, or socioeconomic status. This finding differs from Foorman et al. (1997), but the two populations differed because Foorman et al. participants included students with English as a second language. Study 2: Third Grade Follow-up As indicated in the prior section, a follow-up study was conducted at the end of third grade to determine whether at this critical time non-responders differed significantly from responders in their reading ability, retention rates, and participation in special education. To my knowledge, this was the first follow-up study to confirm whether non-responders were in fact identified by schools as having reading difficulties or reading disabilities. At the end of third grade, despite a good deal of detective work and assistance from the school district, only 51 of the original 104 Study 1 participants were located. The remaining students were not accessible because they had moved out of the school district, were home-schooled, or attended private school. Of the 51 participants located, only one child’s parent did not consent for her to be tested. The 50 follow-up participants were tested on reading performance using the WRMT-R word attack and word identification subtests, as well as an oral reading fluency measure. In addition, participants’ classroom teachers were interviewed to determine whether students had an IEP with reading goals, had received tutoring services in reading, had been retained at grade level, or had required a student study team and subsequent pre-referral reading intervention. All second and third grade teachers used the same District-adopted implicit basal reading program as did the kindergarten and first grade teachers: Harcourt Brace (e.g. Farr & Strickland, 1995). It was not possible, however, to observe classroom instruction or special education instruction. Table 4 describes the attrition rates, responsiveness status, and the third grade educational placement history for each of the remaining 50 participants. The
Identification of Non-responders
73
Table 4. Attrition, Number of Remaining Participants, and Third Grade Educational Placement by Responsiveness Status. 5/99–5/01
Never (n = 34–21)
Always (n = 44–21)
Sometimes (n = 26–12)
Attrition
19 moved 1 LD private 1 no-consent 4 5 4
16 moved 2 no consent 3 home school 23 0 0
11 moved 1 home school 13 1 0
39 6 5
13
23
14
50
Third Grade IEP reading goals Retained/Referred for reading diff. Total
Total (n = 104–54)
significant differences among never, sometimes, and always responsive students were still evident at the end of third grade in terms of word identification, word attack, and oral reading fluency. Table 4 shows four never responsive students were not identified by schools as having reading difficulties. However, all four were in the no-treatment group and had only received typical classroom instruction. By contrast, every non-responder who received treatment in either kindergarten or first grade was identified by schools as having reading difficulties by the end of third grade. Because of their reading difficulties, non-responders were retained or referred for special education. Thus, the predictive accuracy of responsiveness status was slightly lower (92%) when students who received only the more implicit typical classroom instruction were included.
Summary and Some Limitations of the Studies On the one hand, the O’Connor (2000) and the Al Otaiba and Fuchs (2002b) findings demonstrate the percentages of non-responders can be reduced. On the other hand, they also suggest how intensive interventions must be to for the most difficult-to-reach children, including many students with disabilities. For instance, Al Otaiba and Fuchs reported that non-responders had many characteristics of students with reading disabilities and that their responsiveness status rarely improved. Moreover, all of the kindergarten non-responders with IEPs (for speech and language) remained unresponsive in first grade. O’Connor expressed surprise and concern that two of her non-responders had not yet been identified by the school as having reading difficulties by the end of first grade. Had she been able to track students through second and third grade, however, these students may have eventually qualified for services. However, all of the non-responders to treatment
74
STEPHANIE AL OTAIBA
identified by Al Otaiba and Fuchs were identified by schools as having reading difficulties. Their findings, however, must be interpreted with caution because of the high attrition rates. Clearly, the schools who participated in Al Otaiba and Fuchs (2002b) attempted to provide non-responders with additional help: by providing them with tutoring, by retaining them in grade 1 or grade 2, by conducting pre-referral meetings, or by identifying them for special education services. Unfortunately, limited resources precluded observations of these additional services or detailed descriptions of classroom instruction. The same limitation is true in O’Connor’s (2000) study. Further research is needed to see if the characteristics of non-responders can provide guidance in differentiating or individualizing interventions. For example, non-responders with behavior problems might benefit from motivational support. In a similar vein, research is needed to explore how much instructional time should be allotted to each of the five components and how instruction can be differentiated. For example, a logical research question might be: Do non-responders with very low general verbal ability improve in reading when they receive specialized vocabulary instruction?
USING WHAT WE HAVE LEARNED: A MODEL OF RESEARCH-TO-POLICY-TO-PRACTICE Although unresponsiveness to treatment is still a relatively new area of research, it has become highly visible in current efforts to prevent reading difficulties. This nascent research has already influenced policy, and is embodied in the Leave No Child Behind Act of 2001, which aims to help every child read by third grade. In Florida, the Florida Center for Reading Research is helping coordinate efforts to use this research. Policy makers, informed by research, will provide financial incentives for educational practice to change. In my conclusions, I will focus on this model.
Providing Explicit and Systematic Instruction Clearly, the good news is that we know that explicit and systematic reading instruction will help the vast majority of all students learn to read by the end of third grade. Moreover, findings from the review of the literature (Al Otaiba & Fuchs, 2002b) suggest that classroom teachers could lower the percent of non-responders simply by providing more explicit classroom reading instruction. To that end, teachers need explicit basal reading programs for instruction as well
Identification of Non-responders
75
as supplemental materials for intervention. Basal programs should include aligned teacher and student materials that are clearly-organized and that include all five essential SBRR components: phonemic awareness, phonics, fluency, vocabulary, and comprehension. Teachers in Florida will also receive professional development to help them learn why the research shows explicit and systematic instruction is critical for at-risk students. The strongest basal reading programs could serve as an effective support to teacher professional development by providing a concise update of the current research. Teachers and researchers will work together to improve instructional strategies, to intensify instruction, and to individualize instruction for struggling readers. Identifying Non-responders Although no precise definition of non-responder emerged from the review of the literature, it is clear that a combination of early screening and progress monitoring are needed to identify children who are not responding. In Florida, students will be tested four times a year to select students for more intensive intervention. Good, Simmons and Kame’enui (2001) have recently shown that benchmarks or cut-off scores for DIBELS and oral reading fluency measures have good predictive validity for high stakes reading tests at grades three and four. Al Otaiba (2001) and O’Connor (2000) also have shown that phoneme segmentation fluency measures and rapid letter naming may be sensitive enough to distinguish non-responders from responders, even among young children with disabilities. Florida will join other states in conducting large-scale research that is needed to confirm the predictive and concurrent validity of these measures with high stakes testing. Progress monitoring on alternate forms of the same curriculum-based measure will allow teachers and principals to decide whether students are catching up with their peers. Additional diagnostic measures will then be used as needed to further guide intervention. These might include teacher-made tests, behavior observations, or standardized individual tests of phonological awareness or vocabulary. All three types of measures (screening, progress monitoring, and diagnostic measures) need to be valid and reliable and relatively easy and time-efficient for teachers to administer.
Reducing the Percentage of Non-responders Through Immediate and Individualized Intervention We do not yet know the exact instructional conditions that must be available for schools to leave no child behind. However, we can “stack the cards” by focusing on
76
STEPHANIE AL OTAIBA
strategies to reduce the percentage of non-responders. First, we need to maximize the intensity of instruction and intervention for these children for as long as it takes for them to succeed. Even when students are reading successfully and may not need close monitoring of their progress, routine screenings should still be used to ensure they do not fall behind once they return to less-intensive instructional programs. Future research like O’Connor’s (2000) investigation is needed to examine the degree of intensity of instruction non-responders need and then to find effective and efficient ways to make that possible in classrooms with diverse students. In addition, because researchers generally report findings “on average,” such as the NRP report which suggested small group instruction is as effective as one-to-one tutoring, these findings may not necessarily generalize to nonresponders. Therefore, non-responders to less-intensive interventions should be identified early as they are not likely to benefit from “more of the same.” This recommendation appears particularly important for students with disabilities. For example, while PALS was successful on average for most children with disabilities and dramatically reduced the proportion of non-responders among children with disabilities (from two thirds to one quarter), it is important to keep in mind that another year of PALS did not help those children who did not benefit from kindergarten treatment. Future research could identify which children benefit from: (a) explicit classroom instruction; (b) explicit peer mediated intervention; (c) explicit individual or small-group intervention; and which children might need all three. To date, although relatively low-intensity tutorial interventions using peers (Al Otaiba & Fuchs, 2002b) or volunteers (Vadasy et al., 1997) appear more effective than typical implicit classroom instruction, this comparison could be misleading. It is important to recognize that both interventions were explicit, and have not yet been compared directly to classroom instruction that is also explicit. This point is relevant because for non-responders, without such a comparison, it is not yet certain where and how to direct schools’ limited resources. Second, we need more research to identify ways to enhance interventions by differentiating or individualizing based on children’s characteristics. For example, providing extra emotional support (praise or motivation) or scaffolding (breaking down more difficult tasks into easier steps) could reduce the number of non-responders. Juel (1996) has reported that the level of support offered by tutors during beginning reading instruction is an important variable in evaluating treatment effectiveness. Al Otaiba (2001) found that teachers who used more instructional adaptations (i.e. adapted the level of difficulty for students, gave more explicit help, and allowed more low-performing students to respond) had fewer non-responders than teachers who did not use such adaptations. In conclusion, perhaps the most important implication of this chapter is that even with financial incentives for change that will be funded by Reading First, we have a long way
Identification of Non-responders
77
to go to create the necessary conditions and commitment in all schools to provide the kind of intensive, individual, and sustained support needed for all children to read by third grade.
ACKNOWLEDGMENTS This research was supported in part by Grant #H324D000033 and Grant #H324B0049 from the Office of Special Education Programs in the U.S. Department of Education to Vanderbilt University. The paper does not necessarily reflect the position or policy of the funding agency and no official endorsement by it should be inferred. Portions of this chapter were presented at the annual meetings of the Society for the Scientific Study of Reading in Boulder, CO, and Council for Exceptional Children in Kansas City, MO, both in 2001. Correspondence should be addressed to Stephanie Al Otaiba, Florida State University, Department of Special Education, 205 Stone Building, Tallahassee, FL 32306-4459 (or
[email protected]).
REFERENCES Achenbach, T. M. (1994). Child behavior checklist. Burlington, VT: University Medical Education Associates. Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press. Alexander, A., Anderson, H., Heilman, P. C., Voeller, K. S., & Torgesen, J. K. (1991). Phonological awareness training and remediation of analytic decoding deficits in a group of severe dyslexics. Annals of Dyslexia, 41, 193–206. Allington, R. L. (1994). The schools we have. The schools we need. The Reading Teacher, 48, 14–29. Al Otaiba, S. (2001). IRA outstanding dissertation award for 2001: Children who do not respond to early literacy instruction: A longitudinal study across kindergarten and first grade. (Abstract). Reading Research Quarterly, 36, 344–345. Al Otaiba, S., & Fuchs, D. (2002a). Characteristics of children who are unresponsive to early literacy intervention: A review of the literature. Remedial and Special Education, 23, 300–316. Al Otaiba, S., & Fuchs, D. (2002b). Non-responder: A synonym for reading disabled? Can third grade reading disabilities be predicted by responsiveness to early literacy intervention? Paper presented at the annual meeting of the Society for Scientific Study of Reading, Chicago, IL. Berninger, V. W., & Abbott, R. D. (1994). Redefining learning disabilities: Moving beyond aptitudeachievement discrepancies to failure to respond to validated treatment protocols. In: G. R. Lyon (Ed.), Frames of Reference for the Assessment of Learning Disabilities: New Views on Measurement Issues (pp. 163–184). Baltimore, MD: Paul Brookes. Berninger, V. W., Abbott, R. D., Zook, D., Ogier, S., Lemos-Britton, Z., & Brooksher, R. (1999). Early intervention for reading disabilities: Teaching the alphabet principle in a connectionist framework. Journal of Learning Disabilities, 32, 491–503.
78
STEPHANIE AL OTAIBA
Blachman, B. A. (1994). What we have learned from longitudinal studies of phonological processing and reading, and some unanswered questions: A response to Torgesen, Wagner and Rashotte. Journal of Learning Disabilities, 27, 287–291. Blachman, B. (1997). Early intervention and phonological awareness: A cautionary tale. In: B. Blachman (Ed.), Foundations of Reading Acquisition and Dyslexia (pp. 408–430). Mahwah, NJ: Erlbaum. Brown, I. S., & Felton, R. H. (1990). Effects of instruction on beginning reading skills in children at risk for reading disability. Reading and Writing: An Interdisciplinary Journal, 2, 223–241. Cunningham, A. E., & Stanovich, K. E. (1998). What reading does for the mind. American Educator, 22(Spring/Summer), 8–15. Dunn, L. M., & Dunn, L. M. (1981). Peabody picture vocabulary test – revised. Circle Pines, MN: American Guidance Service. Ehri, L. C., & Robbins, C. (1992). Beginners need some decoding skill to read words by analogy. Reading Research Quarterly, 27, 13–26. Farr, R. C., & Strickland, D. S. (1995). Treasury of literature: First street. Orlando, FL: Harcourt Brace & Co. Fazio, B. B. (1997). Learning a new poem: Memory for connected speech and phonological awareness in low-income children with and without specific language impairment. Journal of Speech, Language, and Hearing Research, 40, 1285–1297. Fletcher, J. M., & Foorman, B. R. (1994). Issues in definition and measurement of learning disabilities: The need for early intervention. In: G. R. Lyon (Ed.), Frames of Reference for the Assessment of Learning Disabilities: New Views on Measurement Issues (pp. 185–200). Baltimore, MD: Brookes. Foorman, B. R., Francis, D. J., Fletcher, J. M., Schatschneider, C., & Mehta, P. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Psychology, 90, 37–55. Foorman, B. R., Francis, D. J., Winikates, D., Mehta, P., Schatschneider, C., & Fletcher, J. M. (1997). Early interventions for children with disabilities. Scientific Studies of Reading, 1, 255–276. Fox, B., & Routh, D. K. (1976). Phonemic analysis and synthesis as word-attack skills. Journal of Educational Psychology, 68, 70–74. Fuchs, L. S., Deno, S. L., & Marston, D. (1983). Improving the reliability of curriculum-based measures of academic skills for psychoeducational decision making. Diagnostique, 6, 135–149. Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13(4), 204–219. Fuchs, D., Fuchs, L. S., Thompson, A., Al Otaiba, S., Yen, L., Yang, N., Braun, M., & O’Connor, R. (2001). Is reading important in reading-readiness programs? A randomized field trial with teachers as program implementers. Journal of Educational Psychology, 93, 251–267. Fuchs, D., Fuchs, L. S., Thompson, A., Al Otaiba, S., Yen, L., Yang, N., Braun, M., & O’Connor, R. (2002). Exploring the importance of reading programs for kindergartners with disabilities in mainstream classrooms. Exceptional Children, 68, 295–311. Good, R. H., Simmons, D. C., & Kame’enui, E. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade highstakes outcomes. Scientific Studies of Rading, 5, 257–288. Good, R. H., Simmons, D. C., & Smith, A. (1999). Effective academic interventions in the United States: evaluating and enhancing the acquisition of early reading skills. Educational and Child Psychology, 15, 56–70.
Identification of Non-responders
79
Hammil, D. D. (1991). Detroit tests of learning aptitude – 3. Austin, TX: ProEd. Hammil, D. D., & Newcomer, P. L. (1988). Test of language development-primary. Austin, TX: ProEd. Hatcher, P. J., & Hulme, C. (1999). Phonemes, rhymes, and intelligence as predictors of children’s responsiveness to remedial reading instruction: Evidence from a longitudinal study. Journal of Experimental Child Psychology, 72, 130–153. Hurford, D. P. (1990). Training phonemic segmentation ability with a phonemic discrimination intervention in second- and third-grade children with reading disabilities. Journal of Learning Disabilities, 23, 564–569. Juel, C. (1988). Learning to read and write: A longitudinal study of fifty-four children from first through fourth grades. Journal of Educational Psychology, 80, 437–447. Juel, C. (1994). At-risk university students tutoring at-risk elementary school children: What factors make it effective? In: E. H. Hiebert & B. M. Taylor (Eds), Getting Reading Right from the Start: Effect Early Interventions (pp. 39–62). Boston, MA: Allyn & Bacon. Juel, C. (1996). What makes literacy tutoring effective? Reading Research Quarterly, 31, 268–289. Kaminsky, R. R., & Good, R. H. (1996). Toward a technology for assessing basic early literacy skills. School Psychology Review, 25, 215–227. Kasten, W. C. (1998). One learner, two paradigms. Reading and Writing Quarterly, 14, 335–353. Kennedy, M. M., Birman, M., & Demaline, B. (1986). The effectiveness of Chapter 1 services. Second interim report from the National Assessment of chapter 1. Office of Educational Research and Improvement (OERI), Washington, DC. Leave No Child Behind Act of 2001. Pub. L. No. 107–110 (H. R. 1). Levy, B. A., & Lysynchuk, L. (1997). Beginning word recognition: Benefits of training by segmentation and whole word methods. Scientific Studies of Reading, 1(4), 359–387. Lovett, M. W. (1997). The effectiveness of remedial programs for reading disabled children of different ages: Does the benefit decrease for older children? Learning Disability Quarterly Sum 1997, 20(3), 189–210. Lyon, G. R. (1985). Identification and remediation of learning disability subtypes: Preliminary findings. Learning Disability Focus, 1, 21–35. Lyon, G. R., Fletcher, J. M., Shaywitz, S. E., Shaywitz, B. A., Torgesen, J. K., Wood, F. B., Schulte, A., & Olson, R. (2001). Rethinking learning disabilities. Washington, DC: Hudson Institute. Mastropieri, M. A., Leinart, A., & Scruggs, T. E. (1999). Strategies to increase reading fluency. Intervention in School and Clinic, 34, 278–283, 292. Mathes, P. G., & Fuchs, L. (1994). The efficacy of peer tutoring in reading for students with mild disabilities: A best-evidence synthesis. School Psychology Review, 23, 59–80. Mathes, P. G., Howard, J. K., Allen, S. H., & Fuchs, D. (1998). Peer-assisted learning strategies for first grade readers: Responding to the needs of diverse learners. Reading Research Quarterly, 33, 62–94. Moats, L. C., & Lyon, G. R. (1996). Wanted: Teachers with knowledge of language. Topics in Language Disorders, 16, 73–86. National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. National Institute of Child Health and Human Development, Washington, DC. O’Connor, R. E. (2000). Increasing the intensity of intervention in kindergarten and first grade. Learning Disabilities Research and Practice, 15, 43–54. O’Connor, R. E., Jenkins, J., Leicester, N., & Slocum, T. (1993). Teaching phonological awareness to young children with learning disabilities. Exceptional Children, 59, 532–546.
80
STEPHANIE AL OTAIBA
O’Connor, R. E., Jenkins, J., & Slocum, T. (1995). Transfer among phonological tasks in kindergarten: Essential instructional content. Journal of Educational Psychology, 87, 202–217. O’Connor, R. E., Notari-Syverson, A., & Vadasy, P. F. (1996). Ladders to literacy: The effects of teacherled phonological activities for kindergarten children with and without disabilities. Exceptional Children, 63, 117–130. O’Connor, R. E., Notari-Syverson, A., & Vadasy, P. F. (1998a). Ladders to literacy: A kindergarten activity book. Baltimore, MD: Paul Brookes. O’Connor, R. E., Notari-Syverson, A., & Vadasy, P. F. (1998b). First-grade effects of teacher-led phonological activities in kindergarten for children with mild disabilities: A follow-up study. Learning Disabilities Research & Practice, 13, 43–52. O’Shaughnessy, T. E., & Swanson, H. L. (2000). A comparison of two reading interventions for children with reading disabilities. Journal of Learning Disabilities, 33, 257–277. Peterson, M. E., & Haines, L. P. (1992). Orthographic analogy training with kindergarten children: Effects on analogy use, phonemic segmentation, and letter-sound knowledge. Journal of Reading Behavior, 24, 109–127. Schneider, W., Ennemoser, M., Roth, E., & Kuspert, P. (1999). Kindergarten prevention of dyslexia: Does training in phonological awareness work for everybody? Journal of Learning Disabilities, 32, 429–436. Shannahan, T., & Barr, R. (1995). Reading recovery: An independent evaluation of the effects of an early intervention for at-risk learners. Reading Research Quarterly, 30, 958–996. Share, D. L., & Stanovich, K. E. (1995). Cognitive process in early reading development: A model of acquisition and individual differences. Issues in Eduacation: Contributions from Educational Psychology, 1, 1–57. Simmons, D. C., Fuchs, D., & Fuchs, L. S. (1991). Instructional and curricular requisites of mainstreamed students with learning disabilities. Journal of Learning Disabilities, 24, 354–360. Smith-Burke, M. T., & Jaggar, A. M. (1994). Implementing reading recovery in New York: Insights from the first two years. In: E. H. Hiebert & B. M. Taylor (Eds), Getting Reading Right From the Start: Effect Early Interventions (pp. 63–84). Boston, MA: Allyn & Bacon. Snider, V. E. (1997). Transfer of decoding skills to a literature basal. Learning Disabilities Practice, 1, 54–62. Snow, C. E., Burns, M. S., & Griffin, P. (Eds). (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press. Stanovich, K. E. (1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360–407. Stein, M., Johnson, B., & Gutlohn, L. (1999). Analyzing beginning reading programs: The relationship between decoding instruction and text. Remedial and Special Education, 20(5), 275–287. Torgesen, J. (2000). Individual differences in response to early interventions in reading: The lingering problem of treatment resisters. Learning Disabilities Research and Practice, 15, 55–64. Torgesen, J. K., & Davis, C. (1996). Individual difference variables that predict response to training in phonological awareness. Journal of Experimental Child Psychology, 63, 1–21. Torgesen, J. K., Morgan, S., & Davis, C. (1992). The effects of two types of phonological awareness training on word learning in kindergarten children. Journal of Educational Psychology, 84, 364–370. Torgesen, J. K., Wagner, R. K., & Rashotte, C. A. (1994). Longitudinal studies of phonological processing and reading. Journal of Learning Disabilities, 27, 276–286. Torgesen, J. K., Wagner, R. K., Rashotte, C. A., Lindamood, P., Rose, E., Conway, T., & Garvan, C. (1999). Preventing reading failure in young children with phonological processing disabilities: Group and individual responses to instruction. Journal of Educational Psychology, 91, 579–593.
Identification of Non-responders
81
Uhry, J. K., & Shepherd, M. (1997). Teaching phonological recoding to young children with phonological processing deficits: the effect on sight word acquisition. Learning Disability Quarterly, 20, 104–125. Vadasy, P. F., Jenkins, J. R., Antil, L. R., Wayne, S. K., & O’Connor, R. E. (1997). Community-based early reading intervention for at-risk first graders. Learning Disabilities Research & Practice, 12, 29–39. Vandervelden, M. C., & Siegel, L. S. (1997). Teaching phonological processing skills in early literacy: A developmental approach. Learning Disability Quarterly, 20, 63–81. Vaughn, S. R., Moody, S. W., & Shuman, J. S. (1998). Broken promises: Reading instruction the resource room. Exceptional Children, 64, 211–225. Vellutino, F. R., Scanlon, D. M., & Lyon, G. R. (2000). Differentiating between difficult-to-remediate and readily remediated poor readers. Journal of Learning Disabilities, 33, 223–238. Vellutino, F. R., Scanlon, D. M., Sipay, E. R., Small, S., Chen, R., Pratt, A., & Denckla, M. B. (1996). Cognitive profiles of difficult-to-remediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of specific reading disability. Journal of Educational Psychology, 88, 601–638. Wolf, M. (1991). Naming speed and reading: The contribution of the cognitive neurosciences. Reading Research Quarterly, 26, 123–141. Wolf, M., & Bowers, P. G. (1999). The double-deficit hypothesis for the developmental dyslexias. Journal of Educational Psychology, 91, 415–438. Woodcock, R. (1987). Woodcock reading mastery test – revised. Circle Pines, MN: American Guidance Service. Woodcock, R. W., & Johnson, M. B. (1990). Woodcock–Johnson tests of achievement. Allen, TX: DLM. Yopp, H. K. (1994). A test for assessing phonemic awareness in young children. The Reading Teacher, 49, 20–29.
THE ROLE OF READING INTERVENTION RESEARCH IN THE IDENTIFICATION OF CHILDREN WITH READING DIFFICULTIES: A META-ANALYSIS OF THE LITERATURE FUNDED BY THE NICHD Denise M. Necoechea and H. Lee Swanson ABSTRACT There has been much discussion in the literature in recent years on the problems involved in the identification of children with reading disabilities. One of the most influential sources of knowledge in the field of learning disabilities is the National Institute of Child Health and Human Development (NICHD). This agency has typically been a major funding source for methodologically rigorous reading intervention research. Further, such research has contributed significantly to the validity of identifying children suspected of learning disabilities as “treatment resistors” (e.g. Vellutino et al., 1996). Yet, the NICHD has recently been the focus of some controversy. The purpose of this chapter was to synthesize NICHD funded research conducted over the
Identification and Assessment Advances in Learning and Behavioral Disabilities, Volume 16, 83–161 Copyright © 2003 by Elsevier Science Ltd. All rights of reproduction in any form reserved ISSN: 0735-004x/doi:10.1016/S0735-004X(03)16004-1
83
84
DENISE M. NECOECHEA AND H. LEE SWANSON
past 10 years via a meta-analysis to determine what can be generalized from this body of research that can be applied to the identification of students with learning disabilities in reading. The results of the synthesis were that a prototypical intervention study has a mean effect size (ES) of 0.67 (SD = 0.42), indicating that most interventions designed to increase reading skills were effective. The overall ES ranged, however, from 0.19 to 1.76, and therefore some criterion could be established for identifying treatment resistors. Performance below an overall ES of 0.25 was suggested as one of several criteria for identifying children with potential reading disabilities. However, this suggestion must be put in the context of intervention outcomes. The synthesis indicated that: (a) performance was more pronounced on skill or process measures (e.g. ES varies from 0.45 to 1.28 on measures of segmentation and pseudoword reading) than on measures of actual reading (ES varies from 0.17 to 0.60 on real word and comprehension measures); (b) the magnitude of effect sizes were more related to instructional activity (e.g. explicit instruction/practice) than to the content of instruction (e.g. type of phonics instruction); and (c) the bulk of intervention studies focused on a narrow range of reading behaviors (i.e. phonological awareness). Implications related to identification and sound teaching practice versus content training of reading instruction (e.g. phonological skills, comprehension skills) are discussed.
INTRODUCTION Research funded by the National Institute of Health and Human Development (NICHD) has had a profound effect on the field of learning disabilities. One area that has been strongly influenced by NICHD research is the assessment and identification of children with or at-risk for reading disabilities. Specifically, researchers affiliated with NICHD (among others funded from other sources, for example, Share, McGee, McKenzie, Williams & Silva, 1987; Stanovich & Siegel, 1994) have raised serious questions about the current methodology (e.g. discrepancy, heterogeneity, exclusionary criteria) used to identify children with learning disabilities (Fletcher et al., 1994; Shaywitz, Fletcher, Holahan & Shaywitz, 1992; Vellutino, Scanlon & Lyon, 2000; Vellutino, Scanlon & Tanzman, 1994; Vellutino et al., 1994). This body of research has directed the field’s attention to investigating the critical cognitive components of reading acquisition (i.e. phonological awareness, rapid naming, verbal memory, phonological processing) to accurately identify those individuals with reading deficits. Hence, a wealth of evidence now indicates that reading disability is a language-based disorder,
The Role of Reading Intervention Research in the Identification of Children
85
manifested when phonological processing and phonological awareness skills are lacking (Adams, 1990; Foorman, Francis, Fletcher & Lynn, 1996; Liberman, Shankweiler & Liberman, 1989; Share & Stanovich, 1995; Stanovich, 1988; Stanovich & Siegel, 1994) and that measures of these components are better indicators of reading ability than is the traditional use of measures of intelligence. Despite the significant base of research from which we can derive information regarding the identification of reading disabled learners, a critical area that has not been addressed adequately in the current literature is the learner’s response to reading treatment. Recent NICHD research findings indicate that many reading disabled individuals can improve in achievement with appropriate intervention efforts (Lovett et al., 1996). Yet, intervention research supported by NICHD has suggested that some children with learning disabilities may be conceptualized as treatment resistors (Vellutino et al., 1996), typically defined as those individuals who fail to demonstrate adequate improvement in reading achievement despite having access and exposure to effective and intense intervention procedures (Torgesen, 2000, 2002; Torgesen, Wagner, Rashotte & Heron, 2001). What remains unclear is how responsive (or resistant) reading deficits are to various reading intervention treatments. The current research base is also unclear as to how certain reading treatments can add to our ability to establish early and accurate identification procedures of students struggling with reading skills. Given the suggestion that alternative diagnostic tools should be developed, knowledge about response to reading treatments may provide the field of learning disabilities with a basis for identifying children at-risk for reading disabilities. That is, the field may benefit from a baseline of outcomes related to concentrated reading interventions. Such baselines can serve as a reference point for identifying children with learning disabilities. This chapter addresses the question of what can be generalized from reading intervention studies funded by NICHD. Clearly, a number of well-designed reading intervention studies have been funded by other sources (see Swanson, 1999 for a review) and more comprehensive syntheses of reading interventions have been recently published (e.g. Ehri, Nunes, Stahl & Willows, 2002; Swanson, 1999). However, the current synthesis provides a much more comprehensive and exhaustive analysis of instructional components and parameters than has been previously published. Also, in the current synthesis we limit our review to reading intervention research funded exclusively by the NICHD. This was proposed for a number of reasons. First, NICHD reading intervention research has been used as an informative tool in important decision making, specifically that which involves changes in policy (for a discussion of this issue, see Allington & Woodside-Jiron, 1999; Taylor, 1998; Taylor,
86
DENISE M. NECOECHEA AND H. LEE SWANSON
Anderson, Au & Raphael, 2000). Second, the NICHD reading research program advocates a highly rigorous and methodologically sound research agenda (Lyon, 1999a, b). This agency utilizes a thorough and rigid peer-review system to evaluate research proposals seeking funds for reading research (see Lyon, 1999c for a description of this process). Third, NICHD is unique in that it funds studies that wish to establish reliability of previous findings (replication efforts) across a broad range of areas of literacy development. Moreover, NICHD actively supports research looking at reading development over time (those projects that employ longitudinal designs). Finally, NICHD studies have not only looked at normal reading development, but have included a myriad of studies focusing on disabled readers and the problems they encounter (Lyon, 2002). Recently, commendations were given to the NICHD by Congress for the precedence placed on reading development research and its labors to explicate reading difficulties and reading disabilities (Maloney, 2001). How do children learn to read? Why do some individuals have difficulty with the reading process? Which teaching approaches are necessary for which type of student? Since 1965, the NICHD has attempted to address these questions, and in doing so, has arrived at many conclusions. Over 2,000 refereed journal articles and books have been published in the past 30 years addressing these topics (Adams, 1997). More than 100 researchers from 18 different university-based sites across the nation are involved in the efforts. Yet, the NICHD has arrived at some conclusions not well accepted by others in the scientific community (e.g. Pressley & Allington, 1999a, b; Strauss, 2001). Some research scientists state that NICHD has given priority to word recognition studies, and in doing so, has promoted a narrow conceptualization of reading research (e.g. Pressley & Allington, 1999a; see Issues in Education: Contributions of Educational Psychology, vol. 5, issue 1, for several reviews on these issues). Arguments have been made that substantial monies have been allocated to research focusing on phonemic awareness and word recognition models of reading performance, at the expense of an immensely rich and diverse body of literature on literacy development. Moreover, some critics assert that NICHD has put undue emphasis on research focusing on disabled learners, rather than deriving conclusions based on proficient readers (Pressley & Allington, 1999a). Therefore, some research results obtained from NICHD-funded studies may be too limited in scope to be applied in a catholic manner (Taylor et al., 2000) and should not be used to generate unequivocal conclusions as to how children learn to read. Other researchers also contend that literacy research should espouse a broad perspective of reading development, perhaps employing both qualitative and quantitative methodology, rather than the narrow focus adopted by the NICHD (Alexander & Buehl, 1999; Borkowski, 1999; Morrow, 1999; Pressley & Allington, 1999a; Williams, 1999). Critics of the NICHD reading research program
The Role of Reading Intervention Research in the Identification of Children
87
claim that federal funding continues to be channeled down to scientists studying these overemphasized areas (i.e. phonological awareness) of reading (Hoffman, 1999; Pressley & Allington, 1999b) and suggest that NICHD-funded researchers emphasize word level skills because they insist that comprehension abilities will surface once recognition skills are improved (Pressley & Allington, 1999b). Furthermore, Pressley and Allington (1999a) also question the methodological quality of some of the reading studies funded by NICHD (e.g. Foorman, Francis, Novy & Liberman, 1991; Foorman, Francis, Winikates, Mehta, Schatschneider & Fletcher, 1997; Lovett, Borden, DeLuca, Lacerenza, Benson & Brackstone, 1994; Olson, Wise, Johnson & Ring, 1997; Torgesen, Wagner & Rashotte, 1997; Vellutino et al., 1996; Wise & Olson, 1995). Specifically, these studies are criticized for external and internal validity imperfections (Pressley & Allington, 1999a; Troia, 1999). Given the high visibility of NICHD-funded research in reading, as well as criticisms about what can be generalized from such studies, a synthesis of this research is necessary. The intent of this chapter is to synthesize some of the empirical evidence derived from recent NICHD reading prevention and intervention research. In the present review, the research literature is synthesized through meta-analytic procedures. Previous meta-analyses suggest that not all forms of interventions in reading work equally well (e.g. Swanson, 1999; Swanson & Hoskyn, 1998). In this synthesis, we were interested in isolating the instructional interventions yielding the largest effects, as well as determining what types of individuals benefit from which intervention programs. For example, phonemic awareness instruction has recently become a popular pedagogy adopted by several teachers to train individuals in early reading skills. Yet, meta-analyses effect sizes for phonemic awareness interventions indicate that these treatments have rather modest long-term effects (Bus & van IJzendoorn, 1999). The intent of this synthesis, then, is to analyze the research results from several reading intervention studies in an effort to facilitate our understanding about what can be generalized from the NICHD-research findings and the contribution of reading interventions to the identification of reading disabled or at-risk learners. Another purpose of the synthesis was to evaluate the NICHD reading research program in light of the criticisms recently voiced (see Issues in Education, vol. 5, issue 1, for example). We were interested in looking at the breadth and depth of NICHD-funded studies in the area of reading interventions and evaluate the distribution of the research foci. Further, we wanted to analyze the methodological worth of the studies; methodological differences between studies could account for some of the variance and disparity in reading intervention research. Specifically, we wanted to determine if the quality of research methodology common to each study impacted the size of the effect of the reading outcomes.
88
DENISE M. NECOECHEA AND H. LEE SWANSON
The current synthesis addresses five main questions: (1) What conclusions can be drawn from the NICHD reading research literature that have application to the identification of learning disabilities?; (2) What are the domains of reading (e.g. phonemic awareness, comprehension) that are most susceptible and resistant to change?; (3) What instructional components, or combinations of components, yield the best reading outcomes, and therefore should be included in instructional programs to better identify those children less responsive to treatment?; (4) What methodological variations influence the magnitude of reading outcomes?; and (5) What can we determine regarding the distribution of the NICHD reading research foci?
METHOD Data Collection The online databases of PsycINFO, ERIC, and MEDline were systematically reviewed for studies that were published from 1990 to September of 2000. The databases were searched using the following descriptors: reading, learning disabled (disabilities), reading disabled (disabilities), dyslexia, and at-risk combined with each of the following terms or word pairs: reading remediation, reading instruction, reading strategies, reading failure, word recognition, phonological awareness, reading and treatment, reading and improvement, reading and training, and reading and prevention. In addition, two lists of researchers affiliated with NICHD, as cited in Fletcher and Lyon (1998) and Lyon (1999c) were used for descriptors (author’s last name) for searches on the databases. This search yielded approximately 1,325 citations, from journal articles, book chapters, dissertations, and technical reports. The pool of relevant literature was narrowed down to studies that were funded by the NICHD, as is indicated in the “article acknowledgement” section or footnote of each study. Funding may have been directed to a primary investigator or to a research center. Each study was reviewed and evaluated according to six additional criteria that had been previously established for inclusion in the meta-analysis: (1) Each study included an experimental and control-group design in which participants received treatment to augment their reading performance. Treatment was defined as reading instruction or assistance, given over a
The Role of Reading Intervention Research in the Identification of Children
(2)
(3)
(4)
(5)
(6)
89
minimum of three sessions, which extended beyond what would normally be encountered during regular classroom periods. Each study provided treatment for children, ranging from preschool to high school grade level. Studies with participants older than 18 years of age were excluded. Each study emphasized reading instruction and included at least one dependent measure of reading achievement (i.e. any measure of phonological skills, word recognition, and/or reading comprehension). Studies that focused on other academic domains (i.e. spelling or writing instruction), without directly focusing on reading were excluded. Each study reported sufficient information to calculate the ES between the control and treatment group(s) on reading achievement measures. These were calculated from the means and standard deviations of the performance outcomes for the treatment and control groups, or from tests of the significance of the differences in performance between the instruction conditions (e.g. t or F tests, χ 2 ). Each study included original data not reported elsewhere. Overlapping studies that reported on the same data were excluded. Studies included were either the most recent publication of the same data set or the study that reported the broadest number of outcome measures. Each study had to be published in a refereed journal.
Twenty-one studies met the selection criteria.1 Those studies funded by NICHD that were excluded from the analyses reported duplicate findings (identical studies), failed to include a comparison control group, or failed to include in an intervention specifically focused on reading.
Coding Each of the 21 studies was coded for the following information: (a) sample characteristics (see Appendix A), (b) components of treatment intervention and control condition (see Appendix A), (c) treatment assignment procedures (see Appendix B), (d) instructional and curricular materials (see Appendix B), (e) treatment sessions and length, including overall time exposed to intervention (see Appendix B), (f) internal validity (see Appendix B), and (g) sample variation (see Appendix B). Interrater agreement for the coding of study items exceeded 95% for some components of the articles (dependent measures, sample characteristics, statistical data) and 80% for other areas (internal validity and sample variation).
90
DENISE M. NECOECHEA AND H. LEE SWANSON
Categorization of Dependent Measures Dependent measures were classified as one of five reading categories: (1) reading recognition skills, which included all measures of real-word identification (e.g. WRMT-R word identification subtest; WRAT-3 reading recognition subtest); (2) non-word and pseudo-word reading (e.g. WRMT-R word attack subtest; Goldman-Fristoe-Woodcock non-word reading subtest); (3) phonological skills, which included all measures of blending (synthesis), manipulation, rhyme, and phonemic awareness (e.g. TOPA; Lindamood Auditory Conceptualization test); (4) phonological analysis/word segmentation (e.g. Rosner Test of Auditory Analysis; Torgesen-Wagner Battery of Phonological Analysis); and (5) reading comprehension, which included all measures of silent reading or oral reading comprehension (e.g. WRMT-R passage comprehension subtest; WJ-R passage comprehension subtest). Several studies reported multiple dependent measures within one or more categories. To partially control for statistical independence within the meta-analysis, the effect sizes within each category were averaged within categories before further analysis occurred (see Gleser & Olkin, 1994). For example, if a study provided results for more than one reading recognition measure in the real-word reading measure category, then the effect sizes were averaged across all real-word measures prior to subsequent statistical analyses. Categorization of Treatment Components One hundred fifty-five instructional components were identified (coded 1 for occurrence of components and 0 for non-occurrence) within the 21 studies (see Appendices E and F) in the present study. Instructional components were divided into two broad categories: those reflecting curricular and instructional variables and those reflecting organizational and instructional management variables. Curricular and instructional components were defined as variables involving teaching techniques, content of instruction and practice, instructional methods, and teacher/learner behaviors during which the intervention was administered. Organizational and instructional management components were defined as variables involving the size of the instructional group, the setting of the instruction, and the personnel or mechanism by which the instruction was introduced. By coding instructional components across the studies, we hoped to identify those that were influential in producing larger effect sizes.
The Role of Reading Intervention Research in the Identification of Children
91
Based on previous research that have identified instructional components that influenced student outcomes (Adams, 1990; Berninger & Traweek, 1991; Byrne & Fielding-Barnesley, 1991; Lovett et al., 1994; Lundberg, Frost & Peterson, 1988; Pikulski, 1994; Rosenshine, 1995; Slavin, Karweit & Madden, 1989; Swanson, 1999; Swanson & Hoskyn, 1998), as well as patterns that emerged in the current study, we clustered a subset of the 155 individual instructional activities into 20 cluster categories used for further analyses (see Appendix G). We coded the occurrence of instructional component clusters, as follows: (1) Articulatory awareness activities. Statements in the procedural description reporting instructional activities showing how words are produced by articulatory gestures of the oral region. (2) Phonological skills. Statements in the procedural description incorporating the alphabetic principle, word/syllable/phoneme segmentation (analysis), letter-sound correspondence, manipulation of word parts, phonological/ phonemic/syllable awareness, phonics instruction, word family training, and/or blending sounds (synthesis). (3) Questioning/summarizing. Statements in the procedural description reporting answering questions, retelling information, summarizing text, and/or using comprehension strategies. (4) Orthographic skills. Statements in the procedural description incorporating orthographic awareness activities, code-name correspondence, and/or spelling instruction and practice. (5) Reading activities. Statements in the procedural description reporting reading real- and pseudowords, connected and controlled text, and activities involving choral, guided, repeated, independent, shared, silent and/or oral reading of stories. (6) Metacognitive/strategy instruction. Statements in the procedural description reporting instructional “rules,” self-monitoring, error detection, and/or the use of strategies or mnemonic devices. (7) Technology. Statements in the procedural description reporting computerized instruction or practice and/or the use of media to facilitate presentation of material and feedback of performance. (8) Explicit instruction and practice. Statements in the procedural description reporting explicit instruction, detailed instructional steps, feedback of performance, distributed review and practice, monitoring of practice, fluency-building exercises, and/or overlearning of skills. (9) Teacher/student interchange. Statements in the procedural description about directing students to ask questions, teacher/student dialogue, and/or teacher eliciting questions.
92
DENISE M. NECOECHEA AND H. LEE SWANSON
(10) One-to-one instruction. Statements in the procedural description reporting independent practice, tutoring, individually-paced instruction, and/or instruction that was individually tailored. (11) Small group instruction. Statements in the procedural description reporting a small group instructional setting (2–5 individuals). (12) Large group instruction. Statements in the procedural description reporting a large group instructional setting (>5 members). (13) Regular classroom instruction. Statements in the procedural description indicating that instruction occurred in the regular classroom (where instruction normally occurs for the student). (14) Alternative instructional setting. Statements in the procedural description indicating that instruction occurred outside of the regular classroom (e.g. office area, library). (15) Regular classroom teacher-led instruction. Statements in the procedural description indicating that the regular classroom teacher (the person who normally delivers instruction) introduced the instruction. (16) Non-regular classroom teacher instruction. Statements in the procedural description indicating that a researcher or other adult introduced the instruction, rather than the regular classroom teacher. (17) Computer-mediated instruction. Statements in the procedural description indicating that a computer program introduced the instruction. (18) Skill-modeling. Statements in the procedural description that reported modeling of skills by a teacher, peer, adult, or computer, and/or requiring the student to repeat processes, responses, and/or actions first imitated by the instructor. (19) Control difficulty. Statements in the procedural description reporting the use of systematic training (from simple to complex) of instructional steps, providing assistance, fading prompts or cues, adjusting the level of difficulty to the student, and conducting probes of learning. (20) Advanced organizers. Statements in the procedural description about directing students to attend to the material, explaining the benefits of instruction, pre-training or prerequisite skill building prior to instruction, and/or stating the objectives of instruction. Each study was coded twice for the treatment and control instructional components; this was done to refine code descriptions and to ensure accuracy of the coding procedure. A few studies reported more than one experiment within the same article. Hence, when treatment procedures from a subsequent experiment were described as being similar to the proceeding experiment, unless otherwise specified, it was assumed that components in both experiments were alike and were coded identically. If a targeted study referenced treatment procedures in a
The Role of Reading Intervention Research in the Identification of Children
93
different article, however, without providing details, then only the components listed in the target study were coded. Interrater agreement exceeded 88% for instructional components common to experimental and control conditions across all studies. Coding time for each article ranged from 3.0 to 5.5 hours.
Control Group Parameters In most studies, the control group was identified according the study’s description. When the study failed to indicate the control group directly, we referred to the study’s hypothesis statement to assign condition. If no hypothesis was stated (N = 1), the condition that least resembled a phonological treatment was designated as the control group. When multiple control groups were available within a study, only one group was chosen for comparison; the condition that was most removed from reading instruction was designated the control group. The coding of the control condition was similar to the procedures used for treatment conditions in that all curricular, instructional, and organizational variables were coded as present or not present.
Effect Size Calculation Cohen’s d (Hedges & Olkin, 1985) was the primary index of ES. Cohen’s d was calculated as the difference between control and experimental treatment post-test mean scores (partialed for the influence of pre-test scores if that information was available) divided by the average standard deviation (see Appendix C for ES calculations computed for each reading measure within each of the 21 experiments and Appendix D for ES calculations on follow-up studies). To provide a common standard deviation across various studies, we calculated the pooled (average) standard deviation from the post-test performance of the treatment and control conditions. When post-test standard deviations could not be calculated (such as with repeated measures, covariance, and gain scores), adjustments were made in standard deviations (see Rosenthal, 1994, p. 241, for formula). The primary unit of analysis was the average ES for each study (N = 21). Each study was weighted to its precision, giving more weight to the more precise estimates, weighing each study level according to the inverse of the sampling variance (attaching more weight to studies with larger samples). The dependent measure for the weighted estimate of ES was defined as est = (d/l/v), where v is the inverse of the sampling variance, v = (N trt + N crtl )/(N trt ×N crtl ) + d 2 / [2(N trt + N crtl )].
94
DENISE M. NECOECHEA AND H. LEE SWANSON
Effect sizes vary in value. An ES of +1.00 indicates that the performance of the treatment group exceeded the performance of the control condition by one standard deviation, while an ES of −1.00 indicates the reverse. Positive effect sizes imply that the performance of the treatment group was favorable compared to the control group; negative effect sizes imply that the performance of the control group was favorable compared to the treatment group. The magnitude of the ES may vary as well. Effect sizes less than 0.40 indicated a small effect, between 0.40 and 0.79 indicated a moderate effect, and 0.80 or greater indicated a large effect.
RESULTS Prototypical Study Twenty-one studies yielded 300 effect sizes. When the unit of analysis was aggregated for each study (N = 21), the mean ES across all studies was 0.67 (SD = 0.42), with a range of 0.19–1.76. When the number of reading-related dependent measures was used as the unit of analysis (N = 300), the mean ES across all measures was 0.70 (SD = 0.67), with a range of −0.85 to 3.16. The participants’ chronological ages ranged from 5.4 to 11.5 years old, with a mean of 8.21 (SD = 2.03). Because of the mixtures of grade levels in the participants, a mean grade level was not computed. For the 16 studies that reported participant’s gender, there were 964 male participants and 605 female participants. The mean male sample size was 60.3 (SD = 44.0) and the mean female sample size was 37.8 (SD = 29.4). Of those studies that reported information regarding the intelligence level of the participants (N = 16), the average intelligence score across all of the samples was 97.5 (SD = 7.6) and the mean real word reading achievement standard score (N = 8) was 76.7 (SD = 6.5). Instructional sessions were conducted an average of 4.0 (SD = 1.13) times per week and lasted for a mean of 35.1 (SD = 15.8) minutes. The mean total number of sessions across all studies was 74.3 (SD = 78.4), with a range of 4–300 sessions.
Article Characteristics Articles were published between 1991 and 2000 in the United States, with an average year of publication as 1995 (SD = 2.9). The most frequent publication outlets were the Journal of Educational Psychology (29%), Annals of Dyslexia (19%), Journal of Learning Disabilities (9.5%), and Reading and Writing: An Interdisciplinary Journal (9.5%). The articles were primarily coauthored with
The Role of Reading Intervention Research in the Identification of Children
95
two to seven authors, with an average of 4.09 authors. Seventy-one percent of the primary authors were female. Six primary authors (Berninger, Foorman, Lovett, Olson, Torgesen & Wise) had multiple studies used in the current metaanalysis. Methodological Variations The total sample across the 21 studies was comprised of 1,859 subjects, ranging from 12 to 285 subjects in the combined treatment and control groups. This yielded a mean of 88.5 (SD = 65.8) participants per study. The treatment groups ranged from 6 to 119 participants, with a mean of 34.9 (SD = 28.1) subjects. The control groups ranged from 6 to 60 subjects, having an average of 26.1 (SD = 14.2) per group. Sixty percent of the studies selected participants from public school programs. Ten percent of the studies acquired sample participants from University clinic settings, and another 10% indicated that samples originated from clinic/hospital sources. An additional 5% of the studies used participants who were derived from an early childhood care facility. Approximately 15% of the studies failed to specify the source of their sample. Settings where the treatments were administered included the regular classroom (14.29%), resource room (4.76%), pullout area different from the resource room (19.05%), hospital/clinic (14.29%), and University settings (9.52%). A combination of two or more settings was used in 4.76% of the studies. No information concerning instructional setting for treatment was reported in 33% of the studies. The reader is directed to Table 1 for a complete listing of study characteristics. Comparisons were noted between treatment and control group participants relative to instructional parameters and curricular materials. Along with the widely varied treatments used as interventions, 85.7% of the studies made variations in the materials used in the experimental treatment(s) when compared to the control conditions. The sample size varied between treatment and control conditions in 61.9% of the studies. Approximately 33% of the 21 studies reported differences in the proportion of males to females who participated. A different teacher/experimenter was used to administer treatment and control conditions in 61.9% of the studies. A total of 10 studies, or 47.6%, reported variations in the setting where the treatment and control conditions were administered. Only a small number of studies reported substantial variations in sample age, ethnicity, intelligence scores, and reading achievement scores (range of N = 1–3). Approximately 9.5% of the studies indicated that the experimental group/condition did not differ in any way from the control group/condition. A summary of variations across all studies can be found in Table 2.
96
DENISE M. NECOECHEA AND H. LEE SWANSON
Table 1. Description of Study Characteristics. Description Study sample size Treatment group Control group Intervention settings Regular classroom Resource room Pullout area Clinic/Hospital Other Combination No information
N
%
1859 664 495 3 1 4 3 2 1 7
964 605 290
M
SD
12–285 6–119 6–60
88.5 34.9 26.1
65.8 28.2 14.2
18–65 2–5 4–300
35.1 4.0 74.3
15.8 1.1 78.4
6–174 5–111
60.3 37.8
44.0 29.4
5.4–11.5
8.2
2.02
14.29 4.76 19.05 14.29 9.52 4.76 33.3
Session length (minutes) Sessions per week Number of sessions Subjects Males Females Not cited
Range
51.9 32.5 15.6
Age (in years)
Table 2. Summary of Sample Variations between Control and Treatment Group. Description Sample size Proportion of males to females Age Ethnicity Intelligence Achievement (reading scores) Materials Person administering treatment Setting of instruction Socioeconomic status a Number
No. of Studies Reportinga
% of Studies
13 7 3 2 1 3 18 13 10 0
61.9 33.3 14.3 9.5 4.8 14.3 85.7 61.9 47.6 0.0
of studies that show variations in sample characteristics (N = 21).
The Role of Reading Intervention Research in the Identification of Children
97
Internal validity ratings were calculated for each study. Internal validity was rated with a numeric scale on 11 items, which included controls for: (1) participant mortality; (2) Hawthorne effects and selection bias; (3) comparable exposure between treatment and control groups in terms of materials; (4) instruction time between treatment and control groups; (5) instructional delivery (same teachers) between treatment and control groups; (6) fidelity of treatment administration; (7) practice effects on the dependent measure; (8) floor and ceiling effects; (9) interrater reliability; (10) regression to the mean ruled out as an alternative explanation of findings; and (11) homogeneity of variance between groups (see Lysunchuk, Pressley, D’Ailly, Smith & Cake, 1989; Swanson & Hoskyn, 1998, for further coding detail). In reviewing each study, a score of one reflected internal validity control, two reflected no validity control, and three reflected no validity (or unclear) information was provided in the article. The possible range in ratings was 11–33. The lower the point score, the higher the internal validity of the study. The data generated by the scale indicated a total score range from 7 to 28 points. The mean point total for internal validity was 23.33 (SD = 2.69). This finding is similar to the mean of 22.16 that Swanson (1999) found in a synthesis of 93 reading intervention studies in the field of learning disabilities. Thus, the methodological rigor of studies reported in this synthesis does not differ from others that have been previously analyzed. The most common internal validity item controlled for was the amount of time that the control and experimental groups were exposed to the intervention conditions (66.7% of the studies reported this feature). However, only 48% of the studies reported using checks to determine if the teachers and the students did as they were instructed (treatment fidelity) during the treatment session. The percentage of studies reporting interrater reliabilities was 33.3%. Approximately 9% of the studies reported that the controls were exposed to the same instructional materials as trained subjects. Only 28.6% of the studies controlled for teacher effects (employing the same experimenter to provide treatment for all conditions). Only one study reported ceiling or floor effects. That is, that the mean scores were between 90% or 0.90 for the ceiling or the mean scores were between 0 and 10% or 0.10 for the floor. Summary results of internal validity across all of the studies are found in Table 3.
98
DENISE M. NECOECHEA AND H. LEE SWANSON
Table 3. Summary of Internal Validity Rating. No. of Studies Reportinga
Internal Validity Criteria
1. Subject mortality equal in control and experimental groups 2. Control subjects believed receiving a treatment 3. Control group exposed to same materials as treatment 4. Control group exposed to equal amount of time as treatment 5. Same experimenter provided treatment for all conditions 6. Checks were employed to determine treatment fidelity 7. Alternate forms were used if a dependent measure was repeated 8. There were ceiling or floor effects 9. Interrater reliabilities were reported 10. Regression to the mean ruled out as an alternative explanation 11. Correlations were computed within groups a1
1
2
3
10 2 3 14 6 10 3 1 7 1 7
10 19 18 1 13 2 1 2 8 1 1
1 – – 6 2 9 17 18 6 19 13
= yes; 2 = no; 3 = could not be determined.
Effect Size Effect sizes were separated into five broad categories of dependent measures. These included: (a) phonological skills; (b) segmentation; (c) real word reading; (d) reading comprehension; and (e) pseudoword reading. One hundred and seventy-one standardized dependent reading measures and 134 experimental measures were administered across all studies. Table 4 shows the unweighted and weighted means and standard deviations of Cohen’s d effect sizes as a function of the type of dependent measure across studies. These categories were further subgrouped into either standardized (normreferenced measures) or experimental (researcher-developed) measures. As seen in Table 4, the most frequent dependent measures reflected across the various studies were measures of phonological skills (e.g. phonemic awareness, phonological synthesis, phonological deletion, rhyme). Also frequently used were measures of real word reading (e.g. word identification, regular words, exception words). One of the most striking findings in the analysis is the degree to which the effect sizes varied across categorical domains, with a weighted ES range of 0.17–1.28. As shown in Table 4, marginal effect sizes emerged in standardized measures of reading comprehension, as well as experimental measures of real word reading, reading comprehension, and pseudoword reading. Moderate weighted effect sizes occurred on standardized measures of phonological skills (0.66), real word reading (0.60) and pseudoword reading (0.59). Moreover, experimental measures of phonological skills (0.57) had moderate ES estimates. Based on the weighted
The Role of Reading Intervention Research in the Identification of Children
99
Table 4. Weighted Mean Effect Sizes for Studies as a Function of Dependent Measure Category. N
K
Treatment vs. Control
Effect Size d Unweighted Phonological skills Standardized 9 Experimental 11
95% Confidence Interval for Weighted Effects
Effect Size d Weighted
Lower
Upper
Standard Error
Homogeneity (Q)
48 60
1.04 0.60
(0.79) (0.60)
0.66 0.57
0.46 0.38
0.86 0.77
0.10 0.09
37.95*** 28.82***
3 2
9 3
0.58 1.42
(0.92) (0.68)
0.79 1.28
0.31 0.87
1.26 1.70
0.24 0.21
11.92** 7.81**
Real word reading Standardized 13 Experimental 6
66 38
0.76 0.15
(0.56) (0.34)
0.60 0.17
0.46 –0.06
0.75 0.40
0.07 0.12
22.66* 5.24
4 4
0.39 0.36
(0.06) (0.21)
0.41 0.36
0.15 –0.29
0.66 1.01
0.13 0.33
0.23 0.30
31 7
0.89 0.52
(0.62) (0.45)
0.59 0.40
0.39 0.08
0.80 0.79
0.10 0.20
10.51 3.40
Segmentation Standardized Experimental
Reading comprehension Standardized 3 Experimental 1 Pseudoword reading Standardized 6 Experimental 2
Note: N = number of studies; K = number of dependent measures; (·) = standard deviation. ∗ p < 0.05. ∗∗ p < 0.01. ∗∗∗ p < 0.001.
effect sizes, the areas that approached Cohen’s (1988) threshold of 0.80 for a large effect were standardized (0.79) and experimental (1.28) measures of segmentation. Because the homogeneity tests were significant for a subset of the dependent measures, further analyses were necessary to ascertain some of the variables that moderate the effect sizes. Table 5 shows the correlation between aggregated effect sizes (N = 21) and study characteristics. The relationships between ES and publication date, sample characteristics, treatment session data, and the composite score for internal validity were considered. As shown in Table 5, the important finding was that no significant correlations emerged (alpha set at 0.01) for participant, intervention, or internal validity characteristics. Instructional Component Categories As shown in Appendix E, 155 instructional components were coded. The mean number of instructional components reported in the experimental condition was
100
DENISE M. NECOECHEA AND H. LEE SWANSON
Table 5. Means, Standard Deviations, and Correlation Coefficients between Effect Size and Selected Study Characteristics. Study Characteristics
N
M
(SD)
r
p
Year of publication
21
1996
(2.88)
−0.39
0.07
Participants Male Female Mean IQ score Mean real word reading Mean age
16 16 16 08 17
60.3 37.8 97.5 76.7 8.2
(44.0) (29.4) (7.6) (6.5) (2.0)
−0.40 −0.27 0.30 −0.55 −0.27
0.15 0.31 0.27 0.15 0.29
Instructional intervention Number of sessions (total) Length of session Number of sessions per week Method compositea
21 21 15 21
74.3 35.1 4.0 23.3
(78.4) (15.8) (1.13) (2.69)
−0.35 −0.13 −0.18 0.08
0.57 0.57 0.53 0.72
a Method
was computed from the sum of internal validity items 1 through 11.
30.71 (SD = 16.18) and the mean number reported in the control condition was 14.27 (SD = 14.39). Table 6 shows the mean number of instructional components as a function of the 20 cluster categories for the treatment conditions. For this analysis, all treatment conditions (N = 36) across the 21 studies served as the unit of analysis. The first column shows the component category, the second column shows the mean number of components reported in the treatment condition, the third column shows the possible range in component scores, and the fourth column shows the correlation between the number of components in a category and the ES. As shown in column three, infrequently reported components were those associated with instruction from the regular teacher (M = 0.13, SD = 0.34), instruction in the regular class (M = 0.13, SD = 0.34), large group instruction (M = 0.18, SD = 0.39), articulatory awareness activities (M = 0.24, SD = 0.59), and reading comprehension (M = 0.29, SD = 0.73). Frequently represented categories include phonological skills (M = 5.8, SD = 4.7), reading activities (M = 2.8, SD = 2.1), explicit instruction/practice (M = 2.3, SD = 1.3), technology (M = 2.2, SD = 3.1), one-to-one instruction (M = 1.3, SD = 1.2), and control of task difficulty (M = 1.2, SD = 1.38). The magnitude of the correlation between the reported frequency of the instructional components and ES are reported in column four. As shown, all correlations ranged from low to moderate. Because of the multiple comparisons, alpha was set at 0.01. As shown in Table 6, reported frequency positively affected the ES of the condition. That is, the more certain components were found in a
The Role of Reading Intervention Research in the Identification of Children
101
Table 6. Frequency and Effect Size of Component Cluster Categories for Treatment Condition (N = 36). Component Categories
M (SD)
Range
r
Reported
1. Articulatory awareness 2. Phonological skills 3. Reading comprehension 4. Orthographic skills 5. Reading activities 6. Metacognitive/strategy 7. Technology 8. Explicit instruction/practice 9. Teacher-student interchange 10. One-to-one instruction 11. Small group instruction 12. Large group instruction 13. Instruction in regular class 14. Instruction in alterative setting 15. Instruction from teacher 16. Instruction from other adult 17. Instruction from computer 18. Skill modeling 19. Control difficulty 20. Advanced organizers
0.24 (0.59) 5.8 (4.7) 0.29 (0.73) 0.74 (0.95) 2.8 (2.1) 1.1 (1.4) 2.2 (3.1) 2.3 (1.3) 0.53 (0.73) 1.3 (1.2) 0.45 (0.50) 0.18 (0.39) 0.13 (0.34) 0.63 (0.49) 0.13 (0.34) 0.71 (0.46) 0.34 (0.48) 1.00 (1.09) 1.2 (1.38) 0.74 (0.69)
0–2 0–17 0–4 0–3 0–7 0–4 0–8 0–5 0–2 0–3 0–1 0–1 0–1 0–1 0–1 0–1 0–1 0–3 0–5 0–2
0.14 −0.27 0.27 −0.006 0.27 0.26 0.20 0.41 0.21 0.35 0.22 −0.15 0.34 0.29 0.07 −0.07 0.13 −0.21 0.14 −0.22
χ2
Effect Size Not Reported
N
LSM (SD)
N
LSM (SD)
5 19 4 8 18 10 6 20 9 14 7 5 3 11 3 15 5 10 12 10
0.75 0.56 0.88 0.80 0.59 0.80 0.66 0.62 0.71 0.66 0.61 0.66 1.06 0.60 0.83 0.58 0.61 0.56 0.65 0.48
16 2 17 13 3 11 15 1 12 7 14 16 18 10 18 6 16 11 9 11
0.56 0.96 0.54 0.53 0.62 0.44 0.57 0.26 0.49 0.45 0.59 0.59 0.53 0.59 0.57 0.63 0.60 0.64 0.54 0.82
1.40 3.05 3.90 3.30 0.05 8.04∗ 0.45 1.97 3.17 2.38 0.008 0.18 8.00∗ 0.03 1.78 0.09 0.02 0.45 0.82 6.36∗
Note: An asterisk (∗ ) denotes significance at the 0.01 level.
treatment condition, the higher the effect of the treatment. Specifically, having reported more statements in the treatment condition related to explicit instruction/practice (r = 0.41), one-to-one instruction (r = 0.35), and/or instruction in the regular class (r = 0.34) increased the magnitude of the effect for the particular treatment condition. The only significant correlation was related to the explicit instruction/practice component, r(36) = 0.41, p < 0.01. One limitation to this analysis is that studies vary greatly in terms of the detail reported on instructional components. Further, multiple treatment conditions within studies are not independent (Gleser & Olkin, 1994). Thus, we collapsed instructional components across treatments and compared those studies (N = 21) that reported components related to each category and those that did not. A weighted least squares analysis was used to compare the effect sizes of studies that reported instructional components related to each category (no limitations were place on the frequency) and those studies that did not report any instructional components related to a particular category. Because there were 20 multiple comparisons, a prespecified alpha was set at 0.01. Table 6 shows that treatments
102
DENISE M. NECOECHEA AND H. LEE SWANSON
which include metacognitive/strategy instruction, advanced organizers, and instruction in a regular class setting yielded significantly larger effect sizes than those studies which do not include such components. No other significant effects emerged. Although the differences between reported and not reported components did not differ across categories, the magnitude of the ES was high for a number of components. As shown in column six of Table 6, the majority of categories had at least moderate effect sizes. According to Cohen’s criteria, those treatment conditions reporting the inclusion of orthographic skills (ES = 0.80), metacognitive/strategy instruction (ES = 0.80), instruction from a teacher, rather than another adult (e.g. peer) (ES = 0.83), reading comprehension (ES = 0.88), and instruction in the regular class (ES = 1.06) show large effects.
DISCUSSION This chapter provides a synthesis of reading intervention research conducted over the past decade by researchers funded by the NICHD. Across the 21 studies included in the analysis, the average ES was 0.67, and when effect sizes were averaged across dependent measures (N = 300), the mean ES was 0.70. These findings suggest that these instructional treatments designed to improve reading skills in students in studies that were funded by NICHD were effective. Five important additional findings, however, qualify these results. First, given that the typical effect size across reading treatments is 0.67, the results of the synthesis suggest that treatment resistors would exhibit outcomes below this level on reading measures. When trying to identify children with or atrisk for learning disabilities in the reading domain, professionals could therefore use response to treatment as a guide. Current practice suggests that IQ scores greater than 80 and word recognition scores below the 25th percentile (standard score of 90) serves as an initial beginning point in the identification of children at risk reading disabilities (e.g. Fletcher et al., 1994; Stanovich & Siegel, 1994). Assuming the 25th percentile is an appropriate cut-off point for identifying children at risk for reading disabilities, we would suggest that children whose aggregate reading scores are below an ES of 0.36 are suffering from a reading disability. An ES of 0.36 reflects the first quartile (25th percentile) in this data set. Thus, a criterion of 0.36 ES may be an appropriate cut-off score in which to identify those students with reading deficits. However, an ES of 0.36 approaches Cohen’s criterion for a moderate effect size. Clearly a more rigorous cut-off score should be applied, such as 0.25 (e.g. an ES of 0.25 reflects the 10th percentile in this data set). This criterion approximates that of Swanson (1999, see footnote 1, p. 525) who suggested that a criterion of
The Role of Reading Intervention Research in the Identification of Children
103
0.20 ES be used as a criterion for judging processing difficulties after intervention. It is important to note, however, that we are not arguing that response to treatment below a particular outcome effect (ES), alone, should qualify a child as reading disabled. Rather, we believe that treatment response has practical utility, unlike information derived from IQ tests, in that this indicator can provide additional information to make accurate and sound decisions regarding the instruction and placement needs of the individual. Second, the magnitude of change related to treatment is greater in some reading measures than others. Effect sizes considered marginal according to Cohen’s criteria (effect sizes near or below 0.40) occurred in the domains of reading comprehension and experimental measures of real word recognition and pseudoword reading. Those areas that approached Cohen’s (1988) threshold of 0.80 for a large effect were standardized measures of phonological skills (0.66) and segmentation (0.79). Experimental measures of segmentation exceeded Cohen’s large effect criterion (1.28). Studies that demonstrated effect sizes in the moderate range included standardized measures of pseudoword reading and reading comprehension (0.59 and 0.41, respectively). Therefore, these results suggest that treatment resistance is less likely to be seen in measures of the phonological processes (e.g. segmentation) than on measures of actual reading (e.g. real word reading and reading comprehension). Third, only a few key instructional components moderated the magnitude of effect sizes. We found a positive moderate correlation between ES and the number of explicit instruction/practice components reported in the treatment. These include statements in the treatment description related to distributed review and practice, repeated practice, sequenced reviews, daily feedback, and/or weekly reviews. Other meta-analyses in intervention research corroborate these findings (see Vaughn, Gersten & Chard’s, 2000, review of outcomes from research syntheses, for example). Therefore, treatment conditions that provide explicit, detailed instructional steps, with distributive review and practice, and in conjunction with continual feedback of performance, yield higher reading outcomes. This finding is important because only one of the 20 cluster categories contributed to ES estimates, suggesting that not all treatments or instructional components are equally effective (additive) in predicting ES estimates. Thus, the multiple component approaches that are typically found in a number of intervention studies must be carefully contrasted with a component analysis approach that involves the systematic combination of instructional components known to have an additive effect on performance. Finding that extended practice enhances intervention outcomes is consistent with the existing literature. Long-term retention of all kinds of information and skills, for example, is greatly enhanced by distributed practice plus corrective feedback at different time periods.
104
DENISE M. NECOECHEA AND H. LEE SWANSON
When data were simply analyzed for reporting any component related to a particular category (e.g. a category may include six activities, but the study is given credit for the complete category even if only one activity is reported), larger effects were associated with treatment conditions that incorporated one-to-one instruction and instruction in the regular class setting when compared to studies that did not report such information. Further, two pedagogical features unique to large effects were metacognitive/strategy instruction and advanced organizers. We also found ES estimates to be significantly related to an organizational management feature, namely instruction given in the regular class setting. Most importantly, curricular components, such as phonological skills, spelling, and oral/silent reading activities failed to significantly contribute to positive student outcomes when conditions reporting these components were compared with conditions not reporting these components. Therefore, no outcomes in reading achievement were related to the content of instruction. Thus, it appears that the positive effects of reading intervention research are more related to effective instructional practices than to the specific skills emphasized during the instructional session. This is not to say that instructional content is unimportant. Rather, the results suggest that children may best be taught how to read when teachers focus on good teaching in the context of reading skills instruction. Fourth, variations in research methodology emerged. Given the fact that stringent criteria were used to qualify a reading intervention study for inclusion in the current analysis, it was not expected that the methodological variability of the studies would be great. However, the results show that the 21 studies are heterogeneous in methodological features. Specifically, approximately 68% of the studies failed to control for teacher effects and almost 90% failed to control for exposure to novel materials. Thus, when generalizing from the findings it is important to note whether the same teacher administered both the control and treatment conditions, or if the same materials were involved during the treatment and control conditions. The implication of these findings is that treatments for reading cannot be interpreted accurately unless methodological variations are taken into consideration. Artifacts related to methodology have a profound influence on treatment outcomes (also see Swanson & Hoskyn’s, 1998, discussion of various methodological variables influencing outcomes). Lastly, the distribution of NICHD reading intervention research is narrowly focused on word decoding skills. More than 90% (N = 19) of the 21 studies included standardized dependent measures of real word reading, and another 38% (N = 8) used experimental measures in this instructional domain. In addition, across the 21 studies included in the analysis, approximately 95% (N = 20)
The Role of Reading Intervention Research in the Identification of Children
105
included standardized or experimental dependent measures of phonological skills. Unfortunately, measures of reading comprehension (or precursors to reading comprehension, such as vocabulary, oral comprehension) were represented in only 19% (N = 4) of the studies. When dependent measure categories were analyzed as a function of total number of dependent measures (N = 300), 36% (N = 108) of the dependent measures assessed phonological skills, while fewer than 3% (N = 8) focused on reading comprehension skills. Given the fact that the average student studied was eight years old, it is not surprising that the focus of intervention has been on word recognition skills rather than comprehension. Nevertheless, even some of the precursors to comprehension (e.g. vocabulary) were under-represented. Therefore, the results suggest that rather few interventions funded by NICHD in the last decade are focusing on comprehension or older students (e.g. adolescents). This is interesting given the National Reading Panel’s (2000) recommendation that instruction in early reading include phonological processing, as well as text processing, reading comprehension strategies, oral language vocabulary, and other skills instruction. Moreover, previous research points to the emerging evidence on effective reading comprehension treatments. For example, in a previous metaanalysis, Mastropieri, Scruggs, Bakken and Whedon (1996) found a mean ES of 0.98 of treatments designed to augment comprehension skills in students with learning disabilities. In regards to application of these findings in the classroom, the question arises as to where do these explicit instruction, advanced organizers, metacognitive/strategy, and small regular classroom instruction components fit within the context of instruction? Several authors in both regular (e.g. Slavin, Karweit & Madden, 1989) and special education (e.g. Elbaum, Vaughn, Hughes, Moody & Schumm, 2000; Kucan & Beck, 1997; Pressley & Harris, 1994) suggest that effective instruction includes the following: (1) Provide opportunities for students to learn and work in small groups. (2) State the learning objectives and orient the students to what they will be learning and what performance will be expected of them. (3) Review the skills necessary to understand the concept. (4) Present the new information, give examples, and demonstrate the concepts/materials. (5) Teach procedures that encourage students to use strategies and “think aloud” during problem solving. (6) Pose questions (probes) to students and assess their level of understanding and correct misconceptions. (7) Provide extended practice. Give students an opportunity to demonstrate new skills and learn the new information on their own.
106
DENISE M. NECOECHEA AND H. LEE SWANSON
(8) Assess performance and provide feedback. Review the independent work and give a quiz. Give feedback for correct answers and re-teach skills if answers are incorrect. (9) Provide distributed practice and review. No doubt, the above components have variations within instructional models. The majority of these aforementioned steps are encapsulated in what we have defined as explicit instruction/practice, metacognitive/strategy instruction, advanced organizers, and small group instruction. There are, of course, caveats to our analysis. One obvious caveat is that we collapsed our analysis across all dependent measures in evaluating the importance of instructional components. This is because the measures are correlated (not independent) with other reading measures within a particular study (see Gleser & Olkin’s, 1994, discussion on stochastically dependent effect sizes). Unfortunately, because of the limited number of studies, we did not investigate if a particular instructional component interacts with a particular category of dependent measure in predicting the magnitude of treatment outcomes. Another caveat was that the components we coded might not have matched the components emphasized by the primary authors. For example, some authors, although providing detailed descriptions of certain treatment procedures, failed to give adequate information about the details of the context (i.e. within a “balanced” literacy curriculum). Further, descriptions of the same teaching practice may vary considerably by authors of different theoretical orientations, thereby introducing some artifacts in our coding procedure. In conclusion, the magnitude of effect sizes suggests that some treatments are effective in improving reading. However, most of the studies are directed toward reading processes and therefore outcomes are more isolated to reading skills than to actual reading. Further, only a few instructional components are related to the magnitude of treatment outcomes. These components appear more related to instructional activities (i.e. explicit practice) than to the type of instruction.
NOTE 1. After analysis had been completed, we realized that a study (Torgesen & Davis, 1996) had been overlooked in our search, and thus was not included in the final analysis. In addition, because we relied on footnotes as a basis for determining NICHD as a funding source, some studies may have been inadvertently included. For example, the study by Fuchs et al. (1997) lists NICHD as a funding source although funding is tied to a research center rather than directly to their study. Further, the footnote in the Fuchs et al. study notes other funding sources. However, for consistency of coding, the study was included in the analysis.
The Role of Reading Intervention Research in the Identification of Children
107
ACKNOWLEDGMENTS This chapter is an expanded version of the analysis on the same data set as recorded in Necoechea and Swanson (2003, manuscript in preparation). Portions of the results of this study have been presented at the California Learning Disabilities Association Conference (November, 2001) and the National Learning Disabilities Association Conference (February, 2002), which were supported in part by Peloy Endowment funds awarded to the second author.
REFERENCES ∗ Abbott,
S. P., & Berninger, V. W. (1999). It’s never too late to remediate: Teaching word recognition to students with reading disabilities in grades 4–7. Annals of Dyslexia, 49, 223–250. Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press. Adams, M. J. (1997). About the NICHD program of research on reading development and disorders. Perspectives: The International Dyslexia Association, 23, 3–37. Alexander, P. A., & Buehl, M. M. (1999). An escherian perspective: The recursive nature of reading research. Issues in Education, 5(1), 37–43. Allington, R. L., & Woodside-Jiron, H. (1999). The politics of literacy teaching: How “research” shaped educational policy. Educational Researcher, 28(8), 4–13. ∗ Berninger, V. W., Abbott, R. D., Zook, D., Ogier, S., Lemos-Britton, Z., & Brooksher, R. (1999). Early intervention for reading disabilities: Teaching the alphabet principle in aconnectionist framework. Journal of Learning Disabilities, 32(6), 491–503. Berninger, V. W., & Traweek, D. (1991). Effects of two-phase intervention on three orthographicphonological code connections. Learning and Individual Differences, 3, 323–338. Borkowski, J. G. (1999). Finding the right balance for research on reading instruction. Issues in Education, 5(1), 55–58. ∗ Brady, S., Fowler, A., Stone, B., & Winbury, N. (1994). Training phonological awareness: A study with inner-city kindergarten children. Annals of Dyslexia, 44, 26–59. Bus, A. G., & van IJzendoorn, M. H. (1999). Phonological awareness and early reading: A meta-analysis of experimental training studies. Journal of Educational Psychology, 91(3), 403–414. Byrne, B., & Fielding-Barnesley, R. (1991). Evaluation of a program to teach phonemic awareness to young children. Journal of Educational Psychology, 83, 451–455. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Ed.). New York: Academic Press. Ehri, L. C., Nunes, S., Stahl, S., & Willows, D. (2002). Systematic phonics instruction helps students learn to read: Evidence from the National Reading Panel’s meta-analysis. Review of Educational Research, 71, 393–448. Elbaum, B., Vaughn, S., Hughes, M., Moody, S. W., & Schumm, J. S. (2000). How reading outcomes of students with disabilities are related to instructional grouping formats: A meta-analytic review. In: R. Gersten, E. Schiller & S. Vaughn (Eds), Contemporary Special Education Research (pp. 105–135). Mahawah, NJ: Lawrence Erlbaum Associates.
108
DENISE M. NECOECHEA AND H. LEE SWANSON
Fletcher, J. M., & Lyon, G. R. (1998). Reading: A research-based approach. In: W. Evers (Ed.), What’s Gone Wrong in America’s Classrooms (pp. 49–90). Stanford, CA: Hoover Institution Press. Fletcher, J. M., Shaywitz, S. E., Shankwieler, D., Katz, L., Liberman, I., Stuebing, K., Francis, D., Fowler, A., & Shaywitz, B. A. (1994). Cognitive profiles of reading disability: Comparisons of discrepancy and low achievement definitions. Journal of Educational Psychology, 86, 6–23. Foorman, B. R., Francis, D. J., Fletcher, J. M., & Lynn, A. (1996). Relation of phonological and orthographic processing to early reading: Comparing two approaches to regression-based, reading-level-match designs. Journal of Educational Psychology, 88(4), 639–652. ∗ Foorman, B. R., Francis, D. J., Fletcher, J. M., Schatschneider, C., & Mehta, P. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Psychology, 90(1), 37–55. ∗ Foorman, B. R., Francis, D. J., Novy, D. M., & Liberman, D. (1991). How letter-sound instruction mediates progress in first-grade reading and spelling. Journal of Educational Psychology, 83(4), 456–469. ∗ Foorman, B. R., Francis, D. J., Winikates, D., Mehta, P., Schatschneider, C., & Fletcher, J. M. (1997). Early interventions for children with reading disabilities. Scientific Studies of Reading, 1(3), 255–276. ∗∗ Foster, K. C., Erickson, G. C., Foster, D. F., Brinkman, D., & Torgesen, J. K. (1994). Computer administered instruction in phonological awareness: Evaluation of the Daisyquest program. The Journal of Research and Development in Education, 27(2), 126–137. ∗ Fuchs, D., Fuchs, L. S., Mathes, P. G., & Simmons, D. C. (1997). Peer-assisted learning strategies: Making classrooms more responsive to diversity. American Educational Research Journal, 34(1), 174–206. Gleser, L. J., & Olkin, I. (1994). Stochastically dependent effect sizes. In: H. Cooper & L. V. Hedges (Eds), The Handbook of Research Synthesis (pp. 339–355). New York: Russell Sage Foundation. ∗ Hart, T. M., Berninger, V. M., & Abbott, R. D. (1997). Comparison of teaching single or multiple orthographic-phonological connections for word recognition and spelling: Implications for instructional consultation. School Psychology Review, 26(2), 279–297. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press. Hoffman, J. V. (1999). What do reading teacher educators want from reading research? A call from the hall. Issues in Education, 5(1), 77–83. Kucan, L., & Beck, I. L. (1997). Thinking aloud and reading comprehension research: Inquiry, instruction, and social interaction. Review of Educational Research, 67, 271–299. Liberman, I. Y., Shankweiler, D. P., & Liberman, A. M. (1989). The alphabetic principal and learning to read. In: D. P. Shankweiler & I. Y. Liberman (Eds), Phonology and Reading Disability: Solving the Reading Puzzle (pp. 1–22). Ann Arbor, MI: University of Michigan Press. Lovett, M. W., Borden, S. L., DeLuca, T., Lacerenza, L., Benson, N. J., & Brackstone, D. (1994). Treating the core deficits of developmental dyslexia: Evidence of transfer of learning after phonologically – and strategy-based reading training program. Developmental Psychology, 30, 805–822. Lovett, M. W., Borden, S. L., Warren-Chaplin, P. M., Lacerenza, L., DeLuca, T., & Giovinazzo, R. (1996). Text comprehension training for disabled readers: An evaluation of reciprocal teaching and text analysis to training programs. Brain and Language, 54, 447–480. ∗ Lovett, M. W., Lacerenza, L., Border, S. L., Frijters, J. C., Steinbach, K. A., & De Palma, M. (2000). Components of effective remediation for developmental reading disabilities: Combining
The Role of Reading Intervention Research in the Identification of Children
109
phonological and strategy-based instruction to improve outcomes. Journal of Educational Psychology, 92(2), 263–283. ∗ Lovett, M. W., Steinbach, K. A., & Frijters, J. C. (2000). Remediating the core deficits of developmental reading disability: A double-deficit perspective. Journal of Learning Disabilities, 33(4), 334–358. Lundberg, I., Frost, J., & Peterson, O. (1988). Effects of an extensive program for stimulating phonological awareness in preschool children. Reading Research Quarterly, 23, 263–284. Lyon, G. R. (1999a, July). Hearing on Title 1 of the Elementary and Secondary Education Act: Congressional testimony to the U.S. House of Representatives Committee on Education and the Workforce. Washington, DC. Lyon, G. R. (1999b, October). Education research: Is what we don’t know hurting our children. Congressional testimony to the U.S. House of Representatives House Science Committee and Subcommittee on Basic Research. Washington, DC. Lyon, G. R. (1999c). In celebration of science in the study of reading development, reading difficulties, and reading instruction: The NICHD perspective. Issues in Education, 5(1), 85–115. Lyon, G. R. (2002). Reading development, reading difficulties, and reading instruction: Educational and public health issues. Journal of School Psychology, 40(1), 3–6. Lysunchuk, L. M., Pressley, M., D’Ailly, H., Smith, M., & Cake, H. (1989). A methodological analysis of experimental evaluation