Multidisciplinary studies presented to Edgar Zurif on his 60th birthday
This . Page Intentionally Left Blank
LANGUA...
138 downloads
2033 Views
2MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Multidisciplinary studies presented to Edgar Zurif on his 60th birthday
This . Page Intentionally Left Blank
LANGUAGE AND THE BRAIN
FOUNDATIONS OF NEUROPSYCHOLOGY A Series of Textbooks, Monographs, and Treatises
Series Editor LAIRD S. CERMAK Memory Disorders Research Center, Boston Veterans Administration, Medical Center, Boston, Massachusetts
LANGUAGE AND THE BRAIN Representation and Processing
Edited by
Yosef Grodzinsky Aphasia Research Center Department of Neurology Boston University School of Medicine Boston, Massachusetts, and Department of Psychology Tel Aviv University Tel Aviv, Israel
Lewis P. Shapiro Department of Communicative Disorders San Diego State University San Diego, California
David Swinney Department of Psychology University of California, San Diego La Jolla, California
San Diego
San Francisco New York
Boston
London
Sydney
Tokyo
This . Page Intentionally Left Blank
Contents Contributors
xiii
Preface
xv
In Dedication to Edgar Basil Zurif
xix
PART I
Architecture of the Language System
Chapter 1
Fodorian Modularity and Representational Modularity Ray Jackendoff Problems with Fodorian Modularity on the Input Side . . . . . . . . . . . . . 4 What Is the Output of Language Perception? . . . . . . . . . . . . . . . . . . . . 8 Representational Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Differences between F-Modularity and Representational Modularity . . 17 The Bidomain Specificity of Interface Modules . . . . . . . . . . . . . . . . . . 19 The Relation of Processing to the Linguist’s Grammar . . . . . . . . . . . . . 22 Degrees of Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Chapter 2
Remarks on the Architecture of Language Processing Systems Merrill Garrett The Goals of the Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 A Background Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Some Experimental Examples and Perspectives from Parsing Studies . . 32 The Electrophysiological Record . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Language Production Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Another Path: Language Production as a Comprehension Filter . . . . . . 47 Summary Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
vii
Contents
viii
PART II
The Comprehension–Production Interface
Chapter 3
Overarching Agrammatism Yosef Grodzinsky Production: A Tree-Pruning Hypothesis . . . . . . . . . . . . . . . Receptive Abilities: A Restrictive Trace-Deletion Hypothesis Overarching Agrammatism Examined . . . . . . . . . . . . . . . A Tribute to Edgar Zurif . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 4
. . . . . . . . 75 . . . . . . . . 79 . . . . . . . . 82 . . . . . . . . 84 . . . . . . . . 85
Speech Perception, Conduction Aphasia, and the Functional Neuroanatomy of Language Gregory Hickok Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Evidence for Bilateral Organization of Speech Perception Systems . . . . 90 Evidence for Left Posterior Supratemporal Plane Participation in Speech Production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Relation between Phonological Encoding, Repetition, and Naming Deficits in Conduction Aphasia . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Other Areas Previously Implicated in Speech Perception May Reflect a Network Supporting Auditory–Motor Interaction . . . . . . . . . . . . . 96 Implications for Anatomical Models of Language . . . . . . . . . . . . . . . . 97 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Chapter 5
Overarching Agrammatism: When Comprehension Involves Production Janet L. Nicol and Tracy Love Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Experiment 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 General Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
PART III
The Memory–Language Interface
Chapter 6
Verbal Working Memory and Its Connections to Language Processing Edward E. Smith and Anat Geva Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Neuroimaging Studies of Working Memory . . . . . . . . . . . . . . . . . . . 124
Contents
ix Verbal Working Memory and Language Processing . . . . . . . . . . Other Kinds of Working Memories for Language Understanding Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 128 . . . . 133 . . . . 138 . . . . 139
Chapter 7
Sentence Memory in Amnesia Laird S. Cermak Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Encoding Deficits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Retrieval of Stored Material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Sentence Comprehension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Part IV
The Lexical–Structural Interface
Chapter 8
Toward a Neurochemistry of Naming and Anomia Martin L. Albert Neurochemistry of Output Anomia . . . . . . . Neurochemistry of Lexical Semantic Anomia Conclusions . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 9
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
158 160 162 163
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
167 169 170 175 179 181
. . . . .
. . . . .
. . . . .
185 188 191 195 199
Language Deficits in Broca’s and Wernicke’s Aphasia: A Singular Impairment Sheila E. Blumstein and William P. Milberg Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Theoretical Framework . . . . . . . . . . . . . . . . . . . . . . Lexical Processing Deficits in Aphasia . . . . . . . . . . . Syntactic Deficits in Aphasia: A Preliminary Proposal Semantic Integration . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 10
. . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
Right Hemisphere Contributions to Understanding Lexical Connotation and Metaphor Hiram Brownell Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . Neuropsychological Investigation of Connotation Neuropsychological Investigation of Metaphor . . Summary and Some Unresolved Issues . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
Contents
x
Chapter 11
Aspects of Lexical Access: Evidence from Aphasia Alfonso Caramazza Modality-Specific Deficits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 The Organization of the Lexical System: Must We Postulate Modality-Neutral Lexical Nodes? . . . . . . . . . . . . . . . . . . . . . . The Organization of the Lexical System: Morphology in Spoken and Written Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Evaluating Computational Models of Lexical Access in Aphasia . . . How Good a Fit? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . And If the Fits Had Been Good? . . . . . . . . . . . . . . . . . . . . . . . . . The Globality Assumption and the Data from Modality-Specific Lexical Deficits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 210 . . 215 . . 219 . . 221 . . 222 . . 224 . . 225 . . 226
Chapter 12
Using the Recording of Event-Related Brain Potentials in the Study of Sentence Processing Enriqueta Canseco-Gonzalez Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Event-Related Brain Potentials and Semantic Processing . . . . . . . . . . 231 Event-Related Brain Potentials and Grammatical Processing . . . . . . . 232 Slow Potentials in the Study of Language Processing . . . . . . . . . . . . . 260 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Chapter 13
Grammatical Gender Is Not Always Syntactic Harold Goodglass The Nature of Grammatical Gender . . . . . . . . . . . . . . . . . . . . . . . . . 268 How Does a Speaker Know a Word’s Gender . . . . . . . . . . . . . . . . . . 269 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
Chapter 14
The Time-Course of Lexical Access and the Role of Context: Converging Evidence from Normal and Aphasic Processing David Swinney, Penny Prather, and Tracy Love Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 The Basic Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Lexical Access and Context Effects: Evidence from Neurologically Involved and Non-neurologically Involved Populations . . . . . . Differential Cerebral Hemispheric Contributions to Lexical Access (and Context Effects) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 285 . . . 287
Contents
xi An Integrated View of Lexical Access and Context Effects: Converging Evidence from Normal and Aphasic Populations References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 289 . . . . . 290
Part V
The Syntax–Discourse Interface
Chapter 15
Comprehension of Discourse-Linked and Non-DiscourseLinked Questions by Children and Broca’s Aphasics Sergey Avrutin Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 Consistency with Other Findings . . . . . . . . . . . . . . . . . . . . . . . . . . 308 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Chapter 16
Positron Emission Tomographic Studies of Syntactic Processing David Caplan Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 Experiments with Young Subjects Using Relative Clauses . . . . . . . . . 317 Experiments with Young Subjects Using Active and Passive Sentences . . . . . . . . . . . . . . . . . . . . . . . . . . . Experiments with Elderly Subjects Using Relative Clauses Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 17
. . . . . . . . . 322 . . . . . . . . . 322 . . . . . . . . . 323 . . . . . . . . . 325
Canonicity in Broca’s Sentence Comprehension: The Case of Psychological Verbs Maria Mercedes Piñango Psychological Verbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 The Argument Linking Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . 334 Predictions by the Trace Deletion Hypothesis and the Argument Linking Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Previous and New Empirical Evidence . . . . . . . . . . . . . . . . . Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 18
. . . . . 337 . . . . . 341 . . . . . 345 . . . . . 348
A Simple Argument for Subject Gaps Joan Maling References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
xii
Chapter 19
Index
Contents
Some Recent Investigations of Gap Filling in Normal Listeners: Implications for Normal and Disordered Language Processing Lewis P. Shapiro Study 1: VP–Ellipsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 The Present Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 Study 2: Wh-Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364 Experiment 1: Who and What Questions . . . . . . . . . . . . . . . . . . . . . . 369 Experiment 2: Which-NP Questions . . . . . . . . . . . . . . . . . . . . . . . . . 372 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 377
Contributors Numbers in parentheses indicate the pages on which the authors’ contributions begin.
Martin L. Albert (157) Aphasia Research Center, Department of Neurology, Boston University School of Medicine, Boston Veterans Administration Medical Center, Boston, Massachusetts 02130. Sergey Avrutin (295) Department of Linguistics, Yale University, New Haven, Connecticut 06520. Sheila E. Blumstein (167) Department of Cognitive and Linguistic Sciences, Providence, Rhode Island 02912. Hiram Brownell (185) Department of Psychology, Boston College, Chestnut Hill, Massachusetts 02467; and Harold Goodglass Aphasia Research Center, Department of Neurology, Boston University School of Medicine, Boston, Massachusetts 02130. Enriqueta Canseco-Gonzalez (229) Department of Psychology, Reed College, Portland, Oregon 97202. David Caplan (315) Neuropsychology Laboratory, Massachusetts General Hospital, Boston, Massachusetts 02114. Alfonso Caramazza (203) Department of Psychology, Harvard University, Cambridge, Massachusetts 02138. Laird S. Cermak (143) Center for Memory Disorders, Boston University School of Medicine, Boston, Massachusetts 02130. Merrill Garrett (31) Department of Cognitive Science, University of Arizona, Tucson, Arizona 85721. Anat Geva (123) Department of Psychology, University of Michigan, Ann Arbor, Michigan 48109. Harold Goodglass (267) Department of Neurology, Boston University School of Medicine, Boston, Massachusetts 02130.
xiii
xiv
Contributors
Yosef Grodzinsky (73) Aphasia Research Center, Department of Neurology, Boston University School of Medicine, Boston, Massachusetts 02130; and Department of Psychology, Tel Aviv University, Tel Aviv 69978, Israel. Gregory Hickok (87) Department of Cognitive Sciences, University of California, Irvine, Irvine, California 92697. Ray Jackendoff (3) Center for Complex Systems, Brandeis University, Waltham, Massachusetts 02324. Tracy Love (105, 273) The National Center for Neurogenic Communication Disorders, University of Arizona, Tucson, Arizona 85721; and Department of Psychology, University of California, San Diego, La Jolla, California 92093. Joan Maling (351) Volen Center, Brandeis University, Waltham, Massachusetts 02454. William P. Milberg (167) Geriatric Research, Education and Clinical Center, Veterans Administration Medical Center, West Roxbury, Massachusetts 02132. Janet L. Nicol (105) Department of Cognitive Science, University of Arizona, Tucson, Arizona 85721. Maria Mercedes Piñango (327) Linguistics and Cognitive Science Program, Department of Psychology, Brandeis University, Waltham, Massachusetts 02254. Penny Prather (273) Boston Veterans Administration Medical Center, Boston, Massachusetts 02130. Lewis P. Shapiro (357) Department of Communicative Disorders, San Diego State University, San Diego, California 92182. Edward E. Smith (123) Department of Psychology, University of Michigan, Ann Arbor, Michigan 48109. David Swinney (273) Department of Psychology, University of California, San Diego, La Jolla, California 92093.
Preface Language, a core intellectual ability, is supported by complex neural and psychological mechanisms. In this book, we offer a state-of-the-art multidisciplinary perspective on the inner workings of these mechanisms. This book encompasses work ranging from the latest neuroimaging techniques to detailed syntactic argumentation, from considerations of the brain bases of online lexical access to discourse processing, from memory to metaphor. This type of mix is typical of our field today: to be a player, one must have one’s fingers dipped in linguistic theory (at all its levels), cognitive psychology, cognitive neuroscience, communicative disorders, and experiments “wet” and “dry.” This volume represents this mix. We feel, therefore, that it is most appropriate to dedicate this book to Edgar Zurif, whose work has touched upon all these themes. We have organized the book along the lines of functional interfaces between levels of language representation and processing. We begin with considerations of the architecture of the language system. Jackendoff (Chapter 1) addresses the fundamental issue of mental modularity, proposing a different (non-Fodorian) approach to “representational modularity,” which focuses on two putative kinds of modules—integrative and interface—and the roles they may play in language processing. Garrett (Chapter 2) addresses the conflicts between evidence that appears to support interactive accounts and evidence supporting modular accounts of structural processing. He argues that resolution of these conflicts may be found in a “filtering model” that holds that an essentially modular production system is the source of the apparent interactive constraints on parsing. We next move to consideration of the comprehension–production interface in language. Grodzinsky (Chapter 3) examines apparently separate linguistic descriptions of production and comprehension deficits in aphasia and attempts to derive one from the other in support of an “overarching agrammatism.” Hickok (Chapter 4) argues that speech perception systems are organized bilaterally in the posterior supratemporal plane and that these systems participate in both speech perception and
xv
xvi
Preface
speech production; conduction aphasia is linked to damage to these systems. Nicol and Love (Chapter 5) link the impairment in comprehension found in Broca’s aphasia to a deficit in rehearsal and hence to production in this population. Consideration of the memory–language interface begins with Smith and Geva’s (Chapter 6) description of neuroimaging and behavioral evidence demonstrating that verbal working memory is implemented by the same neural mechanisms involved in language processing. They describe the level of interdependence of these two functions. Cermak (Chapter 7) examines the language deficits found in amnesic patients and describes how these reveal the role of memory in language processing. We next consider the lexical–structural interface in language processing. Albert (Chapter 8) proposes that lexical–semantic deficits in anomia result from deficiencies in cholinergic neurotransmission, whereas deficits in phonological/articulatory output result from deficiencies in dopaminergic neurotransmission, providing both clinical and experimental support. Blumstein and Milberg (Chapter 9) connect level of lexical activation to deficits found in both Broca’s and Wernicke’s aphasia. Broca’s aphasics are claimed to underactivate and Wernicke’s aphasics to overactivate, effects that are argued to extend to phonological and syntactic processing. Brownell (Chapter 10) considers how the right hemisphere (and different regions within the right hemisphere) contributes to the comprehension of metaphor and connotation. Caramazza (Chapter 11) examines modalityspecific lexical processing deficits to inform a model of the general architecture of the lexical processing system. Canseco-Gonzalez (Chapter 12) provides an in-depth review of the electrophysiological (ERP) evidence underlying both lexical and structural (first- and second-pass) processing. Goodglass (Chapter 13) assembles evidence to argue that the ability to identify the grammatical gender of a noun is essentially a semantic operation and is not the same as accessing the syntactic features involved in gender agreement. Swinney, Prather, and Love (Chapter 14) present evidence from both aphasic processing and from left- and right-hemifield studies (with neurologically intact subjects) that support and consolidate a modular view of lexical access. Our final section deals with the syntax–discourse interface in language processing. Avrutin (Chapter 15) examines the comprehension of both discourse-linked and non-discourse-linked Wh- questions. He argues that children as well as Broca’s aphasic patients evince an interface deficiency that emerges during resource-intensive syntax–discourse operations. Caplan (Chapter 16) uses evidence from PET studies to reveal the neural localization of the processing of relative clauses. Piñango (Chapter 17) argues that Broca’s aphasics’ comprehension deficits are attributable to a
Preface
xvii
mismatch between the ordering of thematic roles and the order of roles in syntactic representation. Maling (Chapter 18) provides linguistic argumentation for string-vacuous movement in subject relative sentences, a ding that underlies experimental evidence demonstrating filler-gap reaccess effects in these constructions. Shapiro (Chapter 19) investigates the time course of processing gap-filling constructions and concludes that lexical, structural, and discourse processing are demonstrably independent. The volume was written by Edgar Zurif’s colleagues and students. Our current knowledge and understanding of the neurological underpinnings of language would not have been as rich as they are today had Edgar Zurif not forcefully entered this field and shared his zeal and insights with us. All of us have benefited from and appreciated Edgar’s curiosity and quest for deeper understanding; most of all we value his friendship. Edgar ’s career reflects the range of science found in this book. He began his research career with studies of lateralization, proceeded to linguistically informed deficit analyses of aphasia, later encompassed online lexical and structural processing, and most recently engaged in studies utilizing advanced neuroimaging techniques. Many of his experiments have become landmark studies in the field; several constitute the background literature to papers in this volume. His numerous contributions have played an important role in shaping this field and making it what it is today. Howard Gardner ’s dedication is a personal tribute to Edgar, one that captures the flavor of Edgar’s contributions to the field and to his colleagues. This volume is a small tribute to an outstanding scientist and friend. Joyous moments of celebration are sometimes lessened by sad events. As this book was on its way to the printer, we learned of the death of our friend, colleague, series editor, and contributor to this book, Laird Cermak, who passed away at age 57 on November 4th, 1999. Without him this book would never have come into being. Director of the Boston Center for Memory Disorders, Laird was a good friend, neighbor, and partner of the Aphasia Research Center, home to Edgar, as well as to many of this book’s contributors at one time or another. We all appreciated his wisdom and good cheer, and we shall always miss him. Yosef Grodzinsky Lewis P. Shapiro David Swinney
This . Page Intentionally Left Blank
In Dedication to Edgar Basil Zurif Edgar Basil Zurif was born in Montreal on January 9, 1940. This biographical information is readily available in resource books and on the Web (though Edgar is unlikely to look it—or anything else—up there). I did not have to look it up, however, because I will never forget Edgar’s birthday. More on that later. For twenty years, I had the pleasure of working with Edgar, side by side in the same dingy (and that is being generous) office at the Boston Veterans Administration Aphasia Research Center. We shared the space, research assistants, a shower that was converted into a closet, and equipment that did not work most of the time. But beyond these creature comforts and discomforts, I was privileged to grow professionally with Edgar. Roughly the same age and both near the start of our research careers, we had the equivalent of a professional marriage, one that, perhaps not surprisingly in this turbulent age, lasted beyond the marriages in which we were involved when we first joined forces in the fall of 1971. Truth to tell this was an arranged marriage. Edgar and I had not known each other before, and we had simply been placed in the same space as a convenience by Harold Goodglass, the director of the Aphasia Research Center. But this arranged marriage took. We soon applied for a grant together and shared this grant for the next two decades. We attended courses together, wrote papers together, and gave talks and seminars together. And our relationship became personal as we became good friends. We shared affinities, gossip, prejudices, tall tales, trips, parties, books, movies, trivial pursuits, wonderful ups, and less frequent but no less painful downs. In 1982 we even took a European trip, together with our wives. . . but while we all got along, something told us not to mix professional and personal marriages. I can’t speak for marriages approved by the state or the church, but I can testify to why Edgar is a good partner in research. To begin with, he knows a tremendous amount not only about the areas of his designated
xix
xx
In Dedication to Edgar Basil Zurif
expertise, but also about many regions that would seem quite peripheral. He reads, he thinks, he remembers. Edgar is also a compulsive learner. He wants to know more, he writes it down, he asks questions, and he debates with himself and with others. Even more important, Edgar is a wonderfully lucid thinker. He does not tolerate sloppiness in himself or in others—indeed, he insists doggedly on clarity in his own thinking, speaking, and writing. As is evident from the contributions to this volume, Edgar is a wonderful collaborator in neurolinguistics and allied fields. He has worked with many of the leading figures in the field and has had a strong hand in the formation of the most talented figures in the next generation of workers. (Truth to tell, he has played a “nontrival” role as well in the development of those from rival schools.) He is an enviably effective mentor. But I am here to affirm that Edgar is an equally skillful collaborator with someone whose interests are complementary to his. Edgar and I worked comfortably in language and cognition for twenty years. Aping the hemispheric specializations that attracted Edgar to the field in the 1960s, he took the phonological and syntactic aspects of language, while I took the semantic and pragmatic components; he favored the brief, carefully timed presentations and processes, while I explored the lengthier, more leisurely forms of processing; he focused on language in a strict sense, while I emphasized communication and other forms of cognition—all right, he (and Dave Swinney and Penny Prather and Yosef and Lew) took the left hemisphere and I (and Hiram Brownell) took the right. Edgar’s contributions to neurolinguisitics are significant and will last. While he neither started the field nor coined the term, he was as important as anyone in ushering the Modern Era of Neurolinguistics and has been as important as anyone in sustaining it and giving it direction and intellectual force. As I write, he is probably the most central figure in the field. And while his own interests have remained true to his first love of syntactic processing and its fate in those with Broca-style aphasias, he has left an imprint on a wide range of topics that run the virtual gamut of cognitive neuropsychology. To be a central figure, one must have a powerful intellect, strong theoretical interests, and ingenious empirical gifts. These Edgar has in abundance. These do not earn one a Festschrift at age 60, however. Edgar ’s personal traits have been equally important in sustaining his leadership position: his willingness to discuss, to debate, and even (on occasion) to admit that he has been wrong; his patience in designing studies, explaining concepts, and drawing out the best in colleagues and students, even wayward ones; his enthusiasm for the subject; his funda-
In Dedication to Edgar Basil Zurif
xxi
mental theoretical commitments, coupled with the capacity to change course or argument when the data or new arguments so dictate; his friendship, mentoring, and loyalty to so many “happy collaborators.” You would not know it from his calm and modest demeanor, but Edgar cares about what he wears . . . perhaps almost as much as he cares about the nature of syntactic parsing. Occasionally, he buys new clothing. Because he hates to accumulate things, and because he is by nature generous, Edgar sometimes gives away his clothing. At least 15 years ago, he decided he no longer wanted to wear his black pinstriped winter suit. So he gave it to me, and it fit—very well, in fact. As the new year clicks in every twelve months, I move Edgar’s suit to the front of my closet and when January 9 rolls around, I don the suit and give Edgar a call. We confirm our friendship, laugh a little, commiserate as necessary, looking back to the days of glory at the Boston VA and ahead to our “senior moments” as latter-day Sunshine Boys. If one is to grow old—though 60 is not old any more, Edgar—it is good to have collaborators and friends like Edgar Basil Zurif. Howard Gardner
This . Page Intentionally Left Blank
Part
Architecture of the Language System
I
This . Page Intentionally Left Blank
Chapter
1
Fodorian Modularity and Representational Modularity Ray Jackendoff
Jerry Fodor’s overall agenda in The Modularity of Mind (1983) is that we should not conceive of the mind as a giant maximally interconnected net, such that, for instance, your language processing can potentially be affected by what you ate for breakfast or the color of the speaker’s hair or millions of other ridiculous things. The book is to be taken as an argument against generalized association, cognition as general-purpose problem solver, general connectionism, heavily top-down AI, and top-down “new look” psychology. It is an argument for the specialization of mental functional architecture. I agree completely with Fodor’s (1983) overall agenda. But I think that Fodor’s realization of this agenda as what has come to be called the modularity hypothesis has some serious difficulties that need to be addressed. Unfortunately the issue of modularity has come to be identified with Fodor’s realization, so if one does not agree with him in detail one is liable to be thought of as opposed to modularity. However, one can in fact imagine other conceptions of modularity that preserve the basic agenda and that therefore also deserve to fall under the rubric of modularity theory. In this chapter I work out one such variant, representational modularity, and show how it meets my objections to Fodor ’s (1983) approach while preserving his essential insight. (The essentials of this version of modularity were proposed in Jackendoff, 1987, especially chapter 12; Jackendoff, 1997, works out consequences of this position for linguistic theory.) An important innovation in this approach is a distinction between two kinds of modules: integrative modules, which are responsible for integrating particular levels of representation, and interface modules, which are responsible for communication among representations. The latter have not been clearly recognized in previous literature with which I am familiar; but they prove to be key in understanding many of the crucial issues. Language and the Brain
3
Copyright © 2000 by Academic Press. All rights of reproduction in any form reserved.
4
Ray Jackendoff
In order to keep terminology clear, I will use the term “Fodorian modularity” or “F-modularity” for Fodor ’s (1983) theory, leaving the term “modularity” free for related variants.
PROBLEMS WITH FODORIAN MODULARITY ON THE INPUT SIDE Suppose Fodor (1983) is right (as I think he is) that there is a device for language perception in the brain (or a collection of devices) that is domain specific, that is, devoted to language perception but not to churchbell or symphony or cow perception and not to proving theorems. And suppose he is right that the operation of this device is fast (i.e., measured in milliseconds from input to output) and mandatory (i.e., you can’t help hearing appropriately structured sounds as language). Then a basic question is: What exactly does this device do? In particular, what are its input and output, and how does it get from one to the other? According to Fodor (1983) (at least in the book), the input is the transducers at the ear; the output is some “shallow” representation, perhaps a syntactic structure or a “logical form” [which in current Chomskyan generative theory (government-binding theory and/or the minimalist program) is a part of syntactic structure]. The output is definitely not a form of representation in which the internal semantic properties of words are available—what I would call “conceptual structure” and what Fodor would call “narrow content” or the syntax of the language of thought. For the moment let me use the term “propositional structure” as a relatively theory-neutral term for this level; the intent is that it is the level in terms of which rules of inference and/or meaning postulates (take your choice) are stated. An important part of Fodor’s (1983) agenda is that language perception is informationally encapsulated, so that nothing outside the language module can affect the operation of language perception. Much of the fuss about modularity (e.g., Crain & Steedman, 1985; Marslen-Wilson & Tyler, 1987) has been about the semantic end of the system—whether syntactic parsing is influenced by semantics or not. The usual argument goes: suchand-such a property of meaning or context has a measurable influence on syntactic parsing, therefore language perception is not informationally encapsulated, therefore modularity is wrong. The usual reply goes: If you look at the phenomena more subtly and do the following experiments, you find that in fact syntax is not influenced by semantics up to a certain point in time, therefore modularity is right. (Representative examples include Altmann, 1987; Clifton & Ferreira, 1987; Frazier, 1987; Fodor, Gar-
1.
Fodorian and Representational Modularity
5
rett, & Swinney, 1992.) But notice that the argument really concerns Fmodularity, not modularity in general. F-modularity might be wrong without requiring a reversion to indiscriminately general-purpose processing. It might be, rather, that semantic effects are indeed possible, but constrained by a different version of the modularity hypothesis. Before addressing the issue of the syntax–semantics connection, I want to soften you up by looking at the input end of the language perception module, pointing out four cases where the visual system has a robust effect on language perception. This will show either (a) that the language perception module is hardly as informationally encapsulated on the input end as Fodor suggests, or alternatively (b) that the language perception module, if informationally encapsulated, is a great deal more distended than ought to be appealing to advocates of F-modularity. These two alternatives will make more sense after the examples. 1. In his footnote 13, Fodor (1983) himself mentions the McGurk effect (McGurk & MacDonald, 1976), in which lipreading contributes to phonetic perception and can even override the acoustics of the speech signal in creating judgments about what the utterance sounded like. In the McGurk effect, subjects have no conscious awareness of the disparity between lip position and acoustic signal—they can’t help hearing the sound that would correspond to lip position. This is a classic sort of modularity effect. (Fodor’s footnote, significantly, is not in the section on informational encapsulation, where it would be most pertinent.) 2. Mentioned in Jackendoff (1987) is the case of reading, where visual inputs encode quasi-phonological information. This operation is fast, mandatory (try seeing writing as mere design!), and subject to domain-specific brain damage—again hallmarks of modular processes. However, suppose Fodor were right about the scope of the language perception F-module: its input is transducers at the ear, and its output is syntactic representation. The problem is that phonological structure is a level of representation intermediate between these two, hence by hypothesis informationally encapsulated. That means that the visual system should not be able to affect phonological structure and supplant the normal input of the module. In other words, if Fodor were right about the informational encapsulation of the language module, there could be no such thing as reading. Alternatively, Fodor (1983) might reply (and has done so) that we really don’t know the exact extent of the language module, and that a full characterization would include this second class of possible inputs. And certainly within this class of inputs there is domain specificity and informational encapsulation. It doesn’t change the phonological content of what you’re reading if you change the colors on your computer screen or
6
Ray Jackendoff
carve the letters in a tree or realize them in neon lights. That is, the full capabilities of the visual system are not being exploited. Yet—there is certainly interpenetration between the parts of vision involved in reading and in seeing cows. So the alternative of simply viewing reading as another part of the language module has to finesse a serious problem of how orthographic information is segregated and processed differently from the rest of visual input. A number of papers in Garfield (1987) mention reading, but curiously, none observes this profound challenge to Fodor’s position. For instance, Clifton and Ferreira (1987) speak of a “lexical-processing module (which processes both visual and auditory information)” (p. 279), the former of which sneaks in reading without further notice. Yet the visual information involved in reading is not raw visual input: identifying letters and their linear order would appear to require a level of visual processing at which size and shape constancy have been established, and at which shapes can be abstracted away from the peculiarities of typeface. Moreover, the linear order of written words can’t be detected by a purely “lexical” module. So much of the “vision module” must be invoked in reading—one input module coupled into the middle of another. Moreover, this chapter, as well as Carroll and Slowiaczek (1987) and Frazier (1987), discusses the effects of parsing difficulty on eye movements in reading, without noticing that this constitutes a severe violation of the tenets of F-modularity. Within Fodor’s conception, how can the inner workings of the domain-specific and informationally encapsulated language module interact with the inner workings of the domain-specific and informationally encapsulated vision module, such that, say, “the eyemovement control system interrupts the word-recognition processor and switches into reanalysis mode, under the control of the language processor” (Carroll & Slowiaczek, p. 234)? Unlike the other two papers, Carroll and Slowiaczek at least recognize vision and language as “two systems that are biologically and functionally distinct” (p. 235); but they neglect the obvious consequence for F-modularity. 3. Also mentioned in Jackendoff (1987) are ASL and other signed languages, in which representations derived from visually perceived gesture substitute in the language module for intimate details of the segmental phonology. Here informational encapsulation is even more compromised than in reading. In reading, visual representations are at least used to derive phonological representations that are part of the language module anyway. In ASL, a whole set of abstract gestural representations is substituted for phonetic content in encoding and accessing lexical items (as well as other linguistic units)—while preserving syntactic structure and rhythmic aspects of phonology. If Fodor were right about the inputs and out-
1.
Fodorian and Representational Modularity
7
puts of the (innate) language perception module, and about informational encapsulation, ASL simply couldn’t exist. Again, the alternative could be that there is still another subdivision of the language module that Fodor just neglected to mention. But when does the “additional subdivision” game stop? 4. A more modest case of leaking through the barriers of the language module has to do with the stress-timing of hand movements (beat gestures) signaling emphasis. Consider the sentence in (1), whose emphatic stress is indicated by capital letters. This sentence can be accompanied by the sequence of beat gestures shown in (a) below it (where X is the location of a beat gesture), but certainly not by the sequence in (b). In fact, it is virtually impossible to perform the latter version of the sentence; the reader is encouraged to try it. The overall generalization is that beat gestures may coincide only with emphatic stresses. (1)
This is the most riDICulous thing I’ve EVer heard. a. X X b. X X X
Of course, the beats need not be visual—they can be performed by pounding on the table (auditory) or poking the hearer (haptic). The point is that the temporal placement of gesture is determined by the interaction of the language module with a more general process of rhythmic perception. Rhythmic perception in turn crosses modalities and content domains, pertaining, for instance, to gesture and music as well as language. Thus speech rhythm, which is an important part of linguistic representation and linguistic processing, involving complex rules in the grammar and deeply embedded in the grammar’s inner workings, is neither entirely domain specific—since it has parallels in other domains—nor informationally encapsulated—since it interacts with rhythm in these other domains. Yet again, a proponent of F-modularity might say this is just another branch of the informationally encapsulated language module that Fodor (1983) didn’t happen to think of. But the point this time is that speech rhythm is not entirely domain specific: it bleeds off into various other domains. More generally, looking at all these exceptions at the input end alone, it is beginning to look like the language perception module not only takes input from the auditory system, but also gerrymanders the visual system in an altogether puzzling way, allowing it access to phonetic perception (through the McGurk effect), phonological encoding (through reading), lexical encoding (through ASL), and stress-timing (through beat gesture). This is far from the possibility of arbitrary visual and inferential influence on language perception that Fodor dreads, but it is
8
Ray Jackendoff
also far from the simple input system he lays out in the book. What does informational encapsulation amount to, if every time a putative counterexample appears, the language module is enlarged? Notice too that this move is only open on the input end of the language module. For if Fodor wants to preserve syntax from semantic influence, putative evidence for such influence cannot be countered by enlarging the language module to encompass those portions of semantics. That’s just what Fodor wants to avoid. (Curiously, to my knowledge none of the critics of F-modularity attend to potential counterexamples on the input end. For some reason, interaction of language perception with semantics, on the output end, is a much more fashionable issue.)
WHAT IS THE OUTPUT OF LANGUAGE PERCEPTION? Marslen-Wilson and Tyler (1987) argue (as I did also in Jackendoff, 1987) that the output of language perception is not “a shallow level of structure,” as Fodor (1983) claims. They say (p. 38), “. . . what is compelling about our real-time comprehension of language is not so much the immediacy with which linguistic form becomes available as the immediacy with which interpreted meaning becomes available. It is this that is the target of the core processes of language comprehension.” Why would Fodor object to this conclusion? Remember that the main contrast he is trying to make is between the input systems, which are fast, mandatory, domain specific, etc., and the central processes, which are supposed to be concerned with what he calls belief fixation. The latter are supposed to be relatively slow and nondomain specific, isotropic (i.e., potentially drawing on any kind of mental information), and Quinean (i.e., potentially resulting in the revision of any kind of mental information). Fodor’s chief example of belief fixation is scientific theorizing and discovery. Let’s grant that this is the character of belief fixation. Put in less grandiose terms, belief fixation is the process of deciding whether you think that what someone tells you is true or not. But what is the input to this process? According to Fodor (1983), the input to belief fixation has to be the output of the input system, namely, some sort of “shallow representation,” say a syntactic parse. It is not yet propositional structure. But in order to be fixed (i.e., for its truth value to be determined), a belief has to be formulated in terms of propositional structure. A syntactic representation is simply the wrong vehicle for belief fixation. Thus some process must convert syntac-
1.
Fodorian and Representational Modularity
9
tic structure into the propositional content of what the speaker said. Is this process slow, Quinean, and isotropic? Or is it part of the language perception processor? Fodor seems to think it is not part of language perception (or else fails to notice that it is necessary); Marslen-Wilson and Tyler and I think it is. This conversion certainly is fast and mandatory—one doesn’t have a choice whether to interpret potentially meaningful utterances as meaningful. Perhaps it lags a few milliseconds behind syntactic parsing. That’s no big surprise, since the proper relations among elements in propositional structure depend in part on syntactic structure, so they can’t be determined for sure until syntactic structure is in pretty good shape. But despite this lag, the character of this process is a lot more like parsing than like scientific theorizing. Let me suggest three examples that in different ways highlight the fast and mandatory nature of the conversion from syntactic structure to propositional structure. First, suppose I utter, “The guy on my right is a jerk.” I feel fairly confident that that the guy on my right will experience some immediate affective reaction, measurable by something like a GSR—and that everyone else will experience somewhat less of one. If my guess is right, this means that everybody is deriving the meaning of the sentence and interpreting it contextually to determine whether they are the person being insulted. I might expect a hearer to undergo some GSR just from hearing the lexical item jerk in this context, but considerably more if it is understood as being directed at his or her own person. (Has anyone actually done this experiment?) Now, detecting the meaning of this sentence may lead to a subsequent process of belief fixation that’s slow and Quinean: “Am I a jerk? What evidence in my past experience can I bring to bear? Gee, maybe I really am a jerk, contrary to all my past beliefs. . . .” But preceding that, I am pretty sure there will be a fast, mandatory, modulelike reaction to what is said. This reaction is not an issue of belief fixation—it’s evidently prior to belief fixation, and definitely not a central process in Fodor’s sense. Here is a more intricate example, bearing on the experiments discussed by Marslen-Wilson and Tyler (1987) and by Fodor et al. (1992) concerning the interpretation of the implicit subject (PRO) of a VP. Both papers discuss cross-sentential cases such as (2). (2) As John was leaving the theatre he saw Annie walking down the street. He wanted to talk with her. (PRO) following quickly after her, . . . . Fodor et al. argue from their experimental results that the assignment of PRO to John depends on contextual information and is not a function
10
Ray Jackendoff
of the modular language parser per se. However, consider well-known examples such as these, where PRO is interpreted intrasententially: (3) a. Bill asked Harry to leave. (Bill asked Harryi [PROi to leave], i.e., PRO = Harry) b.Bill asked Harry to be permitted to leave. (Billi asked Harry [PROi to be permitted PROi to leave]) What accounts for the difference in the interpretation of PRO? One might initially think it has to do simply with the change from active to passive in the subordinate clause. But the facts are actually more complex. Here are two examples with passive subordinate clauses in which the outcome is different. (4) a. Bill asked Harry to be examined by the doctor. (Bill asked Harryi [PROi to be examined by the doctor]) b.*Bill asked Harry to be forced to leave. (*Billi asked Harryj [PROi/j to be forced PROi/j to leave]) In (4a) PRO proves to be Harry, not Bill; (4b) proves to be ungrammatical. It turns out that the interpretation of PRO is a complex function of the semantics of the main and subordinate verbs—complex enough that there is no completely adequate linguistic theory of it yet (see Sag & Pollard, 1991, and Jackendoff, 1997, pp. 70–73, for recent attempts and literature surveys.) It cannot be determined from a syntactic parse alone. How does this bear on modularity? The interpretation of PRO and the grammaticality of sentences containing PRO depend on the meanings of two verbs and the combinatorial properties of their meanings, as determined only in part by the syntactic parse of the sentence. But certainly the interpretation of PRO precedes belief fixation: the interpretation of PRO as Harry in (4a) and as Bill in (4b) isn’t a belief—it’s simply part of what the sentence means. That is, the interpretation of PRO in “controlled” VPs, which has always taken to be within the language processor, involves the semantics of verbs and the combinatorics of propositional structure. It is not part of belief fixation. Incidentally, using the combinatorial properties of verb meanings to help interpret PRO also violates one of the basic tenets of Fodor’s theory of lexical meaning. According to Fodor (1975, 1998; Fodor, Garrett, Walker, & Parkes, 1980), a word meaning is an undecomposable monad in the language of thought. Although such a monad may be connected to others by meaning postulates, meaning postulates cannot compose in such a way as to account for the interactions among verb meanings necessary to compute the antecedent (or lack thereof) for PRO in (3)–(4).
1.
Fodorian and Representational Modularity
11
Rather, the verb meanings must have internal structure, possibly of the sort proposed by Sag and Pollard (1991). My final example concerns sentence (5). (5)
The light flashed until dawn.
The interpretation of this sentence requires that the light have flashed repeatedly. Yet there is no lexical item in the sentence that creates this aspect of the interpretation: in isolation, The light flashed implies that it flashed once, so the source of repetition is not flash; in John slept until dawn, John is not falling asleep repeatedly, so the source of repetition is not until. Rather, the source of repetition is the interaction of the meanings of these two words: until implies the bounding of a process extended in time, and flash alone cannot describe such a process. To remedy this incompatibility, the interpretation reconstrues (or coerces) flash as an extended process by taking it to imply a petition of punctual events. (Talmy, 1978, is an early citation of this phenomenon; see discussion and references in Jackendoff, 1997, pp. 51–53.) Here again the sense of repetition is not part of the syntactic structure of the sentence; rather it is part of the propositional content. Nevertheless, the mental process producing this aspect of the interpretation is not slow, optional, isotropic, or Quinean; it is fast and mandatory. And it must take place before belief fixation gets underway, since it creates the mental expression about which belief is to be fixed, and over which inferences are computed. Arriving at this interpretation has now been shown experimentally to entail some additional processing load; moreover this additional load can be sorted out from syntactic processing (Piñango, Zurif, & Jackendoff, 1999). However, it is emphatically not a matter of slow belief fixation. Overall then, the conclusion is that the output of language perception is not a syntactic structure but an expression in the Language of Thought/conceptual structure/narrow content.
REPRESENTATIONAL MODULARITY A lot of the confusion about modularity, I suspect, comes from failing to observe an important property of mental representations, considered as formal systems. This problem, hinted at in the previous section, is embodied in the common practice of drawing diagrams of processing along the lines of (6), (6) syntactic processor → semantic processor
12
Ray Jackendoff
and saying things like “the output of the syntactic processor is sent to the semantics.” Although this makes sense intuitively, it neglects an important step. A domain specific syntactic processor is concerned strictly with syntactic trees, made up of things like nouns (Ns), verb phrases (VPs), complementizers, and case-marking. If it constructs a parse of a sentence, the parse is still made out of syntactic units. Suppose it were to send this parse off to the semantics. What would the semantics do with it? The semantic processor doesn’t know anything about noun phrases (NPs) and case-marking; rather it knows about things like conceptualized objects and events, and about claims to truth and falsity. A syntactic parse simply contains no such information. Let me make this more concrete. The syntactic information that a certain NP is the object of a verb is on its own useless to the semantic processor. Rather, what the semantic processor needs to know is that the individual denoted by this NP plays a certain thematic role (say Patient) in the action denoted by this verb. Similarly, in a classic example of ambiguous PP attachment such as Fran painted the sign on the porch, the syntactic processor can say only that the final PP may be attached to either the NP object or the VP. But this is of no use to the semantic processor, which is concerned with whether the speaker is specifying the location of the sign or the location of the action of painting. In both these examples, it is necessary to correlate the syntactic structure with its semantic consequences. In other words, “sending” syntactic information to the semantic processor actually must entail a process of conversion from one format of encoding to another, a nontrivial process. It is not like sending a signal down a wire or a liquid down a pipe. It is rather a computation in its own right. But this is precisely the step that F-modularity is missing. Leaving the output of the language perception module at “shallow representations,” that is, at syntactic structure, leaves it formally unavailable for the process of belief fixation, which requires semantic information. The argument of the previous section amounts to the claim that the conversion of syntactic to semantic information must be part of language perception. The observation that syntactic and semantic information are formally incompatible provides a key to making sense of all the counterexamples of the preceding two sections, and of why they are problematic to Fodor (1983). We begin by making a distinction between three sorts of mental processes: 1. Processes that construct a full representation in a particular format, given a collection of fragmentary structures in that format. The classic example is a syntactic parser, which is given a sequence of lexical
1.
Fodorian and Representational Modularity
13
categories like (7a) and constructs a fully specified syntactic structure like (7b). (7) a. Determiner^Adjective^Noun^Auxiliary^Verb^Noun b. [S [NP Det [AP A] N] Aux [VP V [NP N]]] Such processes are called integrative processes in Jackendoff (1987). 2. Processes that convert one form of mental representation into another. The classic example is the conversion of acoustical information— frequency analysis of a continuously varying speech signal—into a discretely segmented phonetic representation. Another, as just observed, is needed to convert syntactic parses into specifications of semantic roles. Such processes are called translation processes in Jackendoff (1987) and interface processes in Jackendoff (1997). (The reason for change in terminology is that these conversions do not preserve “meaning.” As will be seen later, it is better to think of them as implementing partial homologies between two forms of representation.) 3. Processes that take full representations in a particular format and relate them to or construct new representations in the same format. Classic cases are rules of inference, which derive new propositional representations from existing ones, and principles of mental rotation, which derive new visual representations from existing ones. Another might be checking to see whether two words rhyme, which compares two phonological representations. I’ll call these inferential processes. Belief fixation in Fodor’s sense is an inferential process. I would like to suggest that the locus of modularity is not large-scale faculties such as language perception (Fodor’s view) but at the scale of individual integrative, interface, and inferential processors. That is, the informational encapsulation and domain specificity of a mental process has to do precisely with what representations it accesses and derives. I will call this approach representational modularity. Such a view has been espoused not only by Jackendoff (1987) but also by Arbib (1987), Levelt (1989), and Bever (1992) (among others). In order to flesh this out a bit, let’s consider the two forms of mental representation about which the most is known: phonological and syntactic structure. We can see that these are strongly domain specific and encapsulated from one another: notice that phonologists and syntacticians can (and do) go their merry way for years without talking to one another. This isn’t a joke—the need for intercommunication is rarely felt, even if less frequently acted on. For instance, the nature of stress rules and of principles of syllabification (phonology) has nothing whatsoever to do with the nature of case-marking or the formulation of the passive (syntax). On the
Ray Jackendoff
14
present view, then, the independence of phonological and syntactic representations means that the two representations have separate integrative processors. Let’s call them the phonology and syntax modules. Of course, phonological structure is neither auditory nor motor: it represents speech in terms of discrete linearly ordered speech sounds, whereas the auditory and motor signals are continuously varying. Hence there must be separate integrative processors for these latter representations as well (though virtually nothing is known about the formal nature of the representations). Just having integrative processors isn’t very useful. In speech perception, auditory signals have to be transformed or converted into phonological structure; in speech production, phonological representations have to be converted into motor signals. For these purposes we need interface modules. What about syntax? A moment’s thought suggests that syntactic information never has a direct bearing on auditory or motor representations. Rather, the effect of syntax on these latter two is always mediated by phonology. In other words, the architecture of the system is such that syntax is connected to phonology by an interface module, but there is no such module connecting syntax to auditory or motor information. Thus the layout of the modules so far looks like (8); where a diagram like (6) had a simple arrow we must insert an interface module.
→
↔
↔
(8) auditory information auditory-to→ phonology → interface phonological syntactic structure structure phonologyto-motor phonology– motor interface syntax information interface →
Mappings are needed in both directions between phonology and syntax, since they are related both in perception and production. Accordingly, I have placed a double-headed arrow between them, suggesting a bidirectional interface module. Alternatively, there might be separate processors for phonology-to-syntax and the reverse. I have no opinions either way, so in what follows, I will use the notation with double-headed arrows to stand for the disjunction of these two hypotheses. As already intimated, syntactic structure is a distinct level of representation from meaning (a.k.a. propositional structure, Language of Thought, conceptual structure, narrow content). In addition to the argument marshaled above, they must be distinct because (1) syntactic structure has
1.
Fodorian and Representational Modularity
15
word order, whereas there is no reason to believe thought does; (2) different syntactic structures are used in different languages to express the same thought. So meaning requires an integrative processor of its own. It is therefore necessary for linguistic inputs to be converted into the format of propositional structure before inferential processes can go to work. Hence the system requires an interface module that goes from syntactic structure to propositional structure. Likewise, in speech production, propositional information must give rise to syntactic structure, so that the words can be placed in proper hierarchical order. Incorporating these conversions into the system, we add (9) as a continuation of (8). (9) phonological syntactic propositional structure structure structure ↔ ↔ phonology– syntax– ↔ ↔ syntax propositional interface interface (9) is still an oversimplification. Phonological information having to do with stress and intonation contour interacts directly with the subsystem of meaning dealing with focus and presupposition, with no necessary syntactic intervention, for instance in the contrast between JOHN hit Bill, John HIT Bill, and John hit BILL. So there is another highly constrained interface module directly from phonology to meaning. (By the way, some theorists posit a level of “linguistic semantic structure” intervening between syntax and propositional structure. If so, this just adds an extra integrative module and an extra interface module to the picture in (9). It does not change the structure of the theory of representational modularity in any essential way.) The upshot is that when one perceives language, auditory information drives the modules in (8)–(9) successively from left to right; when one speaks, propositional information drives the modules successively from right to left. Thinking in terms of spreading activation, one could think of language perception as activating and clamping the auditory level, with activation spreading to propositional structure, and of language production as activating and clamping propositional information, with activation spreading to motor signals. But of course this is a relatively crude way of thinking about it; the interest lies in the connections made within modules and between them, such that activation can spread appropriately. Note in particular that it is not necessary for a level of representation to be completed by its integrative processor in order for the next interface module to start passing information up or down the line; any fragment of representation at one level is sufficient to call into action (or activate) any
Ray Jackendoff
16
modules that can make use of this information. That is, this architecture permits “opportunistic” or “incremental” processing [to use MarslenWilson & Tyler’s (1987) and Levelt’s (1989) terms, respectively]. In addition, since interface modules operate in both directions between phonology, syntax, and propositional structure, it is possible to use intermodular feedback in the course of processing. For instance, a phonetic input, bereft of word boundaries, might well have to wait for evidence of semantic plausibility to arise before word boundary decisions are settled for certain. (10) is a case in point: the existence of a boundary preceding the first contrastively stressed syllable depends on the semantics as well as the syntax of the following contrasting clause. (10) a. It’s only a PARent, not a TEACHer. b. It’s only apPARent, not REAL. Here the phonology module must potentially entertain alternative structures, both of which get passed on by the interface processors successively to syntax and semantics. When the semantics processor resolves the ambiguity, thus “clamping” the intended meaning, the interface processors pass down the inhibition of the rejected structure to syntax and phonology. We see a similar effect in the experimental work of Tanenhaus, SpiveyKnowlton, Eberhard, and Sedivy (1995), where eyetracking shows that visual input interacts with parsing of sentences like (11). (11) Put the apple on the towel in the cup. In order to be able to talk about what we see, interface modules must connect propositional structure to the visual system (Macnamara, 1978; Jackendoff, 1987; Landau & Jackendoff, 1993). To the extent that the visually presented environment affects the course of syntactic parsing, then, it is due to exactly the sort of feedback effect posited for (10), only this time between syntax and visual representations via propositional structure, rather than between phonology and propositional structure via syntax. Each module thus can be driven by representations derived by others. At the same time it must satisfy its own constraints as well. For instance, as is well known, the mapping from auditory information to phonology is nontrivial—that is, there is serious work for the interface processor to do. However, it is not the job of the auditory-to-phonology processor to supply word boundaries where there are no pauses in the auditory signal. In fact, as far as this processor is concerned, the first clauses in (10a, b) are indistinguishable. The possibility of a word boundary after a in (10a) must be supplied, rather, by the integrative processor for phonology. That is, part of the job of the phonology module in language perception is to find the word boundaries in the utterance, which are under-
1.
Fodorian and Representational Modularity
17
determined by the input. And sometimes, as seen in (10), feedback is needed via the phonology–syntax module and even beyond, in order to solve this problem. Moving up one level, the phonology-to-syntax interface module provides linear order but underdetermines constituency; so one of the main problems for the syntax integrator is working out a syntactic constituent structure consistent with its own constraints. This is of course the standard problem of parsing; often feedback from the semantics-to-syntax interface module will be needed to sort out the correct parse. Finally, syntactic structure often underdetermines propositional structure. For instance, syntax provides no evidence for the sense of repetition in (5). Rather, this must be provided either by the integrative module for propositional structure or by the syntax–propositional interface module. Notice how each module is thus domain specific—an integrative processor deals with only one kind of representation, an interface processor with two (we might therefore want to call it “bidomain specific”). Similarly, each module is informationally encapsulated—the only kind of information that can influence it is its designated input level. Through the chaining of integrative and interface processors, we achieve the overwhelmingly complex conversion of acoustic information into meaning. Notice also that if each processor is mandatory and fast, then the chain will be mandatory and (almost as) fast. That is, the effect of Fodor’s facultysized module is created by the chaining of a series of representationspecific modules.
DIFFERENCES BETWEEN F-MODULARITY AND REPRESENTATIONAL MODULARITY There are two important differences between this chain of smaller modules and Fodor’s (1983) faculty-sized language module. First, it closes a crucial gap in Fodor’s conception. As pointed out in the two previous sections, Fodor has no way to get from “shallow representations” (i.e., syntax) to the Language of Thought (i.e., propositional structure). In representational modularity, this function is fulfilled by an interface module that is altogether parallel to the one that mediates between phonology and syntax, which for Fodor falls within the (F-) module for language perception. A second difference between F-modularity and representational modularity is that the latter permits a single level of representation to be fed simultaneously by multiple interface processors, all of whose outputs are taken into account by the integrative processor in constructing a most
18
Ray Jackendoff
coherent representation at that level. A simple case would be the McGurk effect, where an interface processor with visual inputs (specifically mouth configuration) evidently provides information to phonological structure about the composition of phonetic segments. The phonology integrative processor doesn’t care which interface processor has provided the input; it just puts the conflicting fragments of information together as best it can. In other words, the phonology processor is still domain specific and informationally encapsulated; it just has a new source of phonological information. But in turn, this new source is an interface processor which (presumably) is likewise constrained; its input is only mouth configurations, not hair color or high-level beliefs, and its output is only fragments of phonological structure. (We return to the constraints on interface processors in the next section.) Quite a different case is presented by the sense of body orientation (Lackner, 1981, 1988). Here inputs converge from a wide variety of sensory systems: stretch receptors in the muscles, touch and pressure sensors in the skin, the semicircular canals and otolithic organs in the ear, and of course the visual system and auditory localization. The result is a unified sensation of body position and orientation, which in the present framework can be seen (for a first approximation) as the consequence of a single integrative processor reconciling the inputs from a large number of independent interface modules. Here, if anyplace, is a modular system in Fodor’s sense: it is certainly a fast, mandatory, and cognitively impenetrable input system—but with multiple sources of input. These are cases where multiple processors converge on a single integrative processor. The converse also exists: multiple interface processors that take aspects of the same level of representation as their input. A simple case is auditory representation. Alvin Liberman and colleagues (e.g., Liberman & Studdert-Kennedy, 1977) have argued at length that the auditory-to-phonetic interface processor is a specialized modular device that runs in parallel with ordinary auditory processing (i.e., church-bell and thunder perception), taking the same auditory input and doing different analyses of it. In fact, there appear to be other specialized devices that also use the auditory signal in different ways: voice recognition and auditory affect (tone of voice/emotion) perception. Each of these is known to be subject to differential dissociation due to brain damage (Etcoff, 1986, 1989). That is, the auditory-to-phonetic translation is only one of several specialized devices that diverge from auditory representation. Likewise, propositional structure is used as input for speaking (via the propositional structure-to-syntax interface), drawing inferences (a process creating new propositional structures from old), and acting (i.e., converting an intention—a kind of propositional structure—into a format of rep-
1.
Fodorian and Representational Modularity
19
resentation that can drive bodily motion such as navigating or reaching). Through the interface to the visual system, its representations can be used to check a verbal claim or an inference against a visual input. Thus propositional structure fulfills its central function by virtue of having many interfaces to different capacities. A different sort of interface configuration arises in reading aloud. Here visual input drives phonological representation through an interface, but then the information flow goes in both directions: toward pronunciation through the phonology-to-motor interface, but also toward understanding, through the phonology-to-syntax interface (except when reading aloud in a language one does not know!). Clifton and Ferreira (1987, p. 290) make the cautious hypothesis that “some representational vocabulary—for instance, thematic roles [in my approach an aspect of propositional structure—RJ]—may even be shared between the modules of the grammatical system and the general-purpose system for representing knowledge and beliefs.” What I hope to have shown is that it is necessary for modules to share information. Total informational encapsulation of a system makes the system totally useless to the rest of the brain. The interaction of integrative and interface modules in the present proposal makes the sharing of information altogether natural. What about the “slow” central processes posited by Fodor? To the extent that Fodor is correct about belief fixation being relatively slow, isotropic, and Quinean, these properties are now confined to inferential processors which compare newly constructed propositional structures to those present in long-term memory, and which construct further inferences and judgements on the basis of this comparison. If, as I believe, there is a further system of central representations(s) concerned with spatial understanding, this too may well support slow and isotropic inferential processes: consider spatial problems like optimally packing a carton, designing a better mousetrap, and deciding the best route to drive to the other side of the city. That is, inferential processes may well be modular too, though of a different sort than the integrative and interface modules.
THE BIDOMAIN SPECIFICITY OF INTERFACE MODULES So far I have spoken of an interface module as if it simply has access to its characteristic input representation and produces its characteristic output representation. But we can be more specific, just from looking a bit more closely at the examples presented so far. Consider the sharing of auditory information among four different interfaces. The auditory-to-phonetic module pays attention to timing and
20
Ray Jackendoff
distribution of formant transitions—the information it needs to derive phonetic information. But it is oblivious to the general amplitude and pitch envelope, which are cues to voice and affect recognition. Conversely, the modules for identifying voice and affect are oblivious to the distinctions that identify phonetic segments, but they do detect the cues relevant to their own domains. That is, each interface module makes use of only a subset of the distinctions available in its input representation. Conversely, an interface module need not provide all the sorts of information needed by its output representation: think again of how the auditory-to-phonetic module does not provide word boundaries. Thus, more generally, an interface module can be more domain specific than either its input or out- put representation, constrained to deal with only a subset of each. This characteristic is replicated in other interface modules. The phonology-to-syntax module correlates linear order in its input and output representations. But it knows nothing about aspects of phonological structure such as syllabification, which are relevant to pronunciation but not to syntax, and it knows little about embedding in syntactic structure, which is more relevant to semantics than to phonology. Similarly, the interface that relates propositional structure to visuospatial understanding knows nothing about such important aspects of propositional structure as scope of quantification, conditionals, illocutionary force, or value. It recognizes only those aspects of propositional structure that deal with objects, their parts, their location, their motion, and perhaps the forces among them. It also does not know everything there is to know about spatial representation: in particular, it probably does not have access to detailed analogue shape information about objects (Landau & Jackendoff, 1993). In every case, then, the bidomain specificity of the interface processor is limited precisely to those aspects of the two representations that can be directly correlated, while other aspects of each representation are treated as irrelevant, “invisible” to the interface. A crucial point emerges from this look at the interface modules: we should not think of the mind at all as translating or interpreting input signals into thoughts. There is no sense, for example, in which syntactic structure is a translation of phonological structure; rather, some aspects of it correlate with some aspects of phonological structure, and other aspects of each are independent. That is, we ought to think of the relation between representations created by an interface not as a translation but rather as a sort of partial homology. In this light, let’s consider again the counterexamples to Fodorian modularity mentioned earlier. As suggested in the previous section, the McGurk
1.
Fodorian and Representational Modularity
21
effect can now be attributed to an additional interface processor that contributes information to the integrative phonology module. But this interface can’t tell phonology about all aspects of phonological structure—only those distinctive features that can be detected by visual inspection (lip closure and rounding, and perhaps vowel height) plus perhaps some weak information about degree of stress. Similarly, its input is not all aspects of visual structure, but only those that pertain to the external appearance of the vocal tract. So it implements an extremely limited partial homology between the visual input and phonological structure. Reading requires a different visual-to-phonological interface. It provides phonology with predominantly segmental information; punctuation gives some information about prosodic bracketing that is also derivable from the auditory signal. However, unlike the auditory-tophonological interface, reading does not give information about stress (except through use of underlining or italics). And on the other hand, reading gives more reliable information about word boundaries than auditory input. So it too implements a partial homology between visual input and phonological structure—with a quite different selection of visual inputs than the McGurk effect, and with outputs partially overlapping with the auditory-to-phonological mapping. The timing of beat gestures with stress presents a different partial homology. Basically, beat gestures set up a metrical grid of their own; these must be placed in optimal coincidence with the metrical grid in phonological structure. Every other aspect of phonological structure is irrelevant. A similar though more complex problem occurs in setting texts to music: essentially the musical and linguistic stresses must coincide (within certain degrees of latitude). In each of these cases, all that is required is a highly bidomain specific interface processor that has access to only a subset of phonological information pertaining to rhythm. Sign language presents a more complex case. Phonological structure is now generally viewed as a collection of semi-independent tiers— subsystems that are themselves independent but connected by limited interfaces. These subsystems include segmental sequence, syllabification, morphophonological structure (division into stems, prefixes, and suffixes), stress, and intonation. That is, phonology itself is a microcosm of the larger system. (The same is likely true of propositional structure, where, for instance, information about topic and focus is largely orthogonal to information about thematic roles.) Sign language evidently replaces one of these subsystems—the segmental structure—with a new one linked to visual input, whose distinctive features are things like hand shape and hand movement. But the rest of phonological organization— syllabic, prosodic, and morphological structure, is preserved. Again, it
22
Ray Jackendoff
appears that the replacement parts in the system are still highly domain specific and encapsulated. What begins to bother me here is the proliferation of special-purpose interface processors. The situation is better than an indiscriminate generalpurpose processor, because each processor is so limited in what it can do. On the other hand, where do they come from? Do all these pathways of information flow, complete with all their constraints, have to be innate? I see a couple of possibilities. My inclination is to think that all the levels of representation—the integrative processors—are innate, and that some of the interface processors are innate but others are not. The process of reading, for instance, acts like a module in an accomplished reader, but it requires intensive training for most people in a way that the phonologyto-syntax module does not. At this point I don’t fully understand the logical and neurological issues involved in making a claim that an interface module is learnable, so I will have to leave it at that. (A possibly wild suggestion: Suppose there are brain areas for whose use more than one integrative module competes, and which shift allegiances depending on attention. Such an area would be in a position to detect and record fortuitous coincidences in pattern between two modules that are capable of using it. It might therefore be possible for it to develop into an interface processor without having any special use for that purpose wired into it from the start.)
THE RELATION OF PROCESSING TO THE LINGUIST'S GRAMMAR The standard model of generative grammar, handed down in its essentials from Chomsky (1965) and continuing into Government-Binding Theory (GB) (Chomsky, 1981) and the the Minimalist Program (MP) (Chomsky, 1995), conceives of grammar as syntactocentric and derivational. The central generative capacity of language is concentrated in syntax. An initial stage of syntactic derivation (D-structure in GB, the operation Merge in MP) builds structure, complete with lexical items; then various rules of movement and deletion produce levels of syntactic structure that are subjected to phonological and semantic interpretation. Phonological and semantic structures are conceived of as outputs of a syntactic derivation, with no significant generative capacities of their own. As has been remarked for decades, this framework of grammatical theory cannot serve directly as a model of processing. We were all taught as graduate students that one should not think of a syntactic derivation as modeling the course of processing: one doesn’t think of the initial symbol
1.
Fodorian and Representational Modularity
23
S first, then gradually expand it till one chooses the words, then push the pieces around until one finally decides what the sentence means and how to pronounce it. So the notion of derivation has been distanced from processing by calling it “metaphorical”; this term has been applied especially to processes of syntactic “movement.” It seems to me in retrospect that Chomskyan generative grammar could well have seen fit to abandon the syntactocentric assumption in the middle 1970s, but the opportunity was resolutely rejected. At this time phonological theory began to develop autonomously generated tiers of prosodic structure, syllabic structure, and intonation contour, linked by “association lines,” and it became clear that these were by no means “continuations” of low-level syntactic derivations. But the autonomy of phonology never came to be recognized in Chomskyan syntax, and today the Minimalist Program (Chomsky, 1995) speaks of “phonological Spell-Out” in terms roughly comparable to those in early generative syntax. Likewise, by the middle 1970s the extended standard theory had beaten off the challenge from Generative Semantics, which claimed that syntax is just a “continuation” of a derivation whose generative component is made up of semantic elements. Much of this argumentation (Chomsky, 1972; Jackendoff, 1972) concerned the autonomy of syntax from semantics. Yet the next major move in the theory was to institute a syntactic level of Logical Form (LF) (Chomsky, 1979; May, 1985), derived by “covert movement,” whose purpose was to make syntax and semantics less autonomous from each other. Before long Chomsky would say (Chomsky, 1986, p. 68) “PF [phonetic form] and LF constitute the ‘interface’ between language and other cognitive systems, yielding direct representations of sound on the one hand and meaning on the other. . . .” In this statement, Chomsky (1986) elides the fact on which so much has hinged in the present chapter: an interface has to implement a relationship between two distinct representations. Hence it cannot be derivational in the standard sense, changing one structure into another built out of the same primitives. That is, the interface between syntax and meaning—not to mention that between syntax and phonology— cannot be like moving elements around in a tree, interior to a syntactic derivation. Nor can the systems dealing with meaning and phonology make any sense out of a syntactic tree. So a grammar by necessity needs components that establish relationships among these disparate formats. A syntactic level alone cannot “yield a direct representation of meaning.” (Incidentally, I must say that it is hard to believe that Chomsky could be guilty of such an elementary formal error. But, despite a fairly extensive correspondence with him about this issue, I sadly see no alternative.)
24
Ray Jackendoff
Several independent factions in generative grammar have been attracted to a different conception of grammar, a constraint-based theory, in which the notion of derivation is replaced by a set of “declarative” constraints on wellformedness of grammatical structures, and which characterize relatedness between constructions by means other than syntactic movement. Constraint-based frameworks include Head-driven Phrase Structure Grammar (Pollard & Sag, 1994), Lexical-functional Grammar (Bresnan, 1982), Construction Grammar (Fillmore & Kay, 1993; Goldberg, 1995), Role and Reference Grammar (Van Valin & LaPolla, 1997), Autolexical Syntax (Sadock, 1991), Multidimensional Categorial Grammar (Bach, 1983), Autosegmental Phonology (Goldsmith, 1976), and Optimality Theory (Prince & Smolensky, 1993; Grimshaw, 1997); a similar architecture is proposed for musical structure (though involving quite different representations) by Lerdahl and Jackendoff (1983). An important characteristic of many of these approaches is that they admit multiple sources of generative capacity—multiple autonomous levels of structure, each characterized by its own set of constraints, plus sets of linking (or interface) constraints that establish the connections among the levels. Jackendoff (1997) proposes a general form for such architectures that incorporates the basic insights of these approaches. In such frameworks, a well-formed sentence must have well-formed structures in each component, connected in a well-formed fashion by linking constraints. In other words, the syntactocentric conception of grammar has been abandoned in favor of one in which syntax is but one among several generative components. The interesting feature of these architectures in the present context is how nicely they map into models of processing. One can see the constraints specific to a particular level as principles applied by the integrative processor for that level; likewise, the linking constraints are the business of the interface processors. The constraints within a level are nondirectional, so that one can use them to build (or activate) structures from the bottom up or from the top down. The interface constraints are likewise nondirectional, so that one can, for instance, use them to build (or activate) partial syntactic structures based on phonological input— or equally well vice versa. Moreover, the interface constraints can be used directly to generate feedback, so that for instance, in perception, semanticto-syntactic and syntactic-to-phonological constraints can together serve to constrain phonological analysis, as in (10) above. In addition, the role of the lexicon becomes clearer. In the standard architecture, lexical items are inserted into initial syntactic derivations, then carried through the derivation until they are “interpreted” semantically and phonologically. This of course bears no relation to a theory of processing.
1.
Fodorian and Representational Modularity
25
In a constraint-based theory, the account is quite different. A lexical item is a long-term memory association of a phonological structure, a collection of syntactic features, and a meaning (propositional structure). It can therefore be viewed as part of the linking constraints: if, for instance, the sound /kæt/ is present in phonological structure, this licenses a linked singular count noun in syntactic structure and a linked meaning [CAT] in conceptual structure. In other words, a lexical item is a small-scale linking rule, and the lexicon in general is to be thought of as part of the interface components from phonology to syntax and from syntax to semantics. This said, it is easy to see the role of lexical access in processing. Consider first perception. The auditory-to-phonetic interface dumps a raw sequence of phonetic segments in the lap of the phonology integrative module. In order to figure out where to place the word boundaries, the phonology module sends out a call to the lexicon: “Does anybody out there sound like this?” (i.e., does that phonological structure of any item match this raw input?) And various lexical items pipe up, “Me!” (i.e., become activated). But by virtue of becoming activated, a lexical item doesn’t just activate its phonology: it also activates its syntax and semantics, and thus establishes partial structures in those domains and partial linking to the phonology. Similarly in production: suppose one has a complex conceptual structure one wants to express. Then the semantics-to-syntax processor sends a call to the lexicon: “Does anybody out there mean this?” And various candidates raise their hands: “Me!” and thereby become activated. But by virtue of becoming activated, they also activate their syntax and phonology, and thus establish partial structures in those domains and partial linking to the intended message. I don’t think there is anything new about this idea in processing: it is just that it corresponds so nicely to the view of lexical items that falls out of the constraint-based approach to grammar. The upshot, then, is that representational modularity is a variant of modularity that, unlike F-modularity, can be brought into a very close and appealing relation to a variant of generative grammar that is independently of interest—though a relation to standard generative grammar is as distant as ever.
DEGREES OF MODULARITY In talking with people about modularity, I often get the impression that they think of a modular capacity as entirely independent of the rest of the mind. And that is indeed what domain specificity and informational
26
Ray Jackendoff
encapsulation seem to imply—just the opposite of isotropic and Quinean (i.e., maximally connected). But, as stressed several times already, an entirely domain-specific and informationally encapsulated module would be functionally disconnected from the rest of the mind, and could therefore serve no purpose in the larger goal of helping the organism perceive and behave. The view of modularity developed here invites a middle road, because an important part of the hypothesis is how modules communicate with each other, namely, through interface modules. In particular, the means by which the language faculty communicates with central cognition is of precisely the same essential character as the means by which the various subparts of the language faculty communicate with each other. So no module is entirely informationally encapsulated. To make clearer how Fodor’s (1983) original agenda now plays itself out, it is worthwhile reviewing what domain specificity and informational encapsulation mean in the present framework. The domain specificity of an integrative module has to do with the range of distinctions encoded by the level of representation with which this module deals. The phonology module deals exclusively with phonological distinctions—and its subcomponents are even more domain specific: the stress tier deals only with stress grids, for instance. The propositional structure module deals with propositional organization of thought; the spatial structure module deals with the shape, location, and motion of physical objects (and like phonology it may well divide into subcomponents). The domain specificity of an interface module is better termed “bidomain specificity”: it has to do with the range of distinctions that the module is sensitive to in both its input and output representations. As we have seen, an interface module may be more domain specific than the integrative modules it connects; the “McGurk” module is an especially good example of a very narrowly specific interface module. Informational encapsulation is a bit more complex. It seems to me that we can’t talk properly about the informational encapsulation of a module. Rather, we have to talk about it in relational terms. The original intuition is that the speaker’s hair color can’t affect phonetic perception—a relational statement. The more general question is: to what degree does a distinction D1 in one representation R1 have a bearing on a distinction D2 in representation R2? For instance, word boundaries in phonology have a direct reflection in syntactic structure; so syntax is not at all encapsulated with respect to this aspect of phonology. Similarly, embedding in syntactic structure is closely related to embedding in corresponding propositional structure, so there is little encapsulation here.
1.
Fodorian and Representational Modularity
27
For a more distant case, motion detected in the visual field (or detection of someone’s hair color, for that matter) can affect propositional structure through the spatial-to-propositional interface, and, if expressed in language, have an indirect effect on produced phonological structure through the chain of interfaces. Even the output of the semicircular canals may have an effect on phonological structure (the speaker may say “I’m dizzy!”), but the effect is distant, mediated by a long chain of intermediate representations and interfaces. On the other hand, the motion of the speaker’s lips may have a direct effect on phonological structure, through the “McGurk” interface. So here, in a very small domain, gerrymandered in both input and output components, we have more information flow, and more direct information flow, than in random aspects of vision and phonology. In other words, the presence of an interface between two representations is what makes them not informationally encapsulated from each other. The extent of the information flow depends on the degree to which the interface module is more domain specific than the representations it connects. The phonology-to-syntax interface is relatively rich; the “McGurk” interface is not. In principle, an interface module might be precisely as domain specific as its input and output representations. This would mean that every distinction in the input representation could make a difference in the output representation, and every distinction in the output representation could be affected by something in the input representation. In such a case, though, it would hardly make sense to speak of two distinct “modules” any more: their interactions would be more or less unconstrained. We might as well see the union of their domains as a single larger domain, and we might as well see the interface module just as a part of the principles involved in integrating this larger domain. It therefore begins to make sense to speak of “degrees of modularity” rather than absolute modularity. Two domains connected by a narrow “information bottleneck” will be relatively modular: not very many parts of each domain can affect the other. As the interface becomes richer, more parts can interact. If communication between the two domains is wide open, it is impossible to say where one leaves off and the other begins. Given the gradual nature of this transition from relatively modular to nonmodular, it is impossible to say exactly where modularity leaves off. A cynic might say therefore that the issue of modularity is dissolved. I would disagree. Rather it develops into a more nuanced set of questions: What families of distinctions form richly interconnected and wellintegrated domains, and where are there more restricted “informational bottlenecks”? The correlates in processing are as they always have been:
28
Ray Jackendoff
Can one identify particular stages of informational integration, enhanced by priming or disrupted by concurrent load in the same domain? Can these stages be identified with discrete levels of representation in the formal theory? To what extent can these stages be temporally distinguished in the course of processing and/or spatially distinguished by brain imaging? To what extent can they be differentially disrupted by brain damage? It seems to me that this slightly more nuanced viewpoint on modularity leaves unaffected all the standard issues in processing. Perhaps it can spare us continuation of some of the needless debate of the 15 years.
ACKNOWLEDGMENTS It is a pleasure to be able to honor Edgar Zurif by contributing to the present volume. I’m not sure how to translate my admiration of Edgar into the medium of cold public print. Suffice it to say that, although he has elevated whining into an artform (and is happy to acknowledge it), you don’t have to go very deep before you encounter a thoroughgoing mensch of the greatest professional and personal integrity. I have been honored to have him as a colleague and friend for these last 15 years. This paper originated in remarks on Fodorian modularity presented at an MIT Cognitive Science colloquium in fall 1991, which explains the datedness of many of the references. Nevertheless, I believe that many of the misunderstandings enumerated here still persist, so that despite the passage of time the discussion is still pertinent. In particular, almost everything said here about F-modularity applies as well to the version of modularity in Fodor (2000).
REFERENCES Altmann, G. (1987). Modularity and interaction in sentence processing. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing (pp. 249–258). Cambridge, MA: MIT Press. Arbib, M. (1987). Modularity and interaction of brain regions underlying visuomotor coordination. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing (pp. 333–364). Cambridge, MA: MIT press. Bach, E. (1983). On the relation between word-grammar and phrase-grammar. Natural Language and Linguistic Theory, 1, 65–90. Bever, T. (1992). The logical and extrinsic sources of modularity. In M. Gunnar & M. Maratsos (Eds.), Modularity and constraints in language and cognition (Vol. 25 of the Minnesota Symposia on Child Psychology, pp. 179–212). Hillsdale, NJ: Erlbaum. Bresnan, J. (Ed.). (1982). The mental representation of grammatical relations. Cambridge, MA: MIT Press. Carroll, P. J., & Slowiaczek, M. L. (1987). Modes and modules: Multiple paths to the language processor. In J. Garfield (Ed.), Modularity in knowledge representation and naturallanguage processing (pp. 221–248). Cambridge, MA: MIT Press. Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: MIT Press. Chomsky, N. (1972) Studies on semantics in generative grammar. The Hague: Mouton.
1.
Fodorian and Representational Modularity
29
Chomsky, N. (1979). Language and responsibility. New York: Pantheon. Chomsky, N. (1981) Lectures on government and binding. Dordrecht: Foris. Chomsky, N. (1986) Knowledge of language. New York: Praeger. Chomsky, N. (1995) The Minimalist Program. Cambridge, MA: MIT Press. Clifton, C. & Ferreira, F. (1987). Modularity in sentence comprehension. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing (pp. 277–290). Cambridge, MA: MIT Press. Crain, S. & Steedman, M. (1985). On not being led up the garden path. In D. Dowty, L. Kartunnen, & A. M. Zwicky (Eds.), Natural language parsing: Psycholinquistic, computational, and theoretical perspectives. Cambridge: Cambridge University Press. Etcoff, N. L. (1986). The neuropsychology of emotional expression. In G. Goldstein & R. E. Tarter (Eds.), Advances in clinical neuropsychology (vol. 3). New York: Plenum. Etcoff, N. L. (1989). Asymmetries in recognition of emotion. In F. Boller & J. Grafman (Eds.), Handbook of neuropsychology (vol. 3, pp. 363–382). New York: Elsevier Science Publishers. Fillmore, C., & Kay, P. (1993). Construction grammar coursebook. Berkeley, CA: University of California, Copy Central. Fodor, J. (1975). The language of thought. Cambridge, MA: Harvard University Press. Fodor, J. (1983). The modularity of mind. Cambridge, MA: MIT Press. Fodor, J. (2000). The mind doesn’t work that way: The scope and limits of computational psychology. In press. Fodor, J., & Lepore, E. (1998). The emptiness of the lexicon: Reflections on James Pustejovsky’s the generative lexicon. Linguistic Inquiry, 29, 269–288. Fodor J., Garrett, M., Walker E., & Parkes, C. (1980). Against definitions. Cognition, 8, 263–367. Fodor, J., Garrett, M., & Swinney, D. (1992). A modular effect in parsing. Unpublished manuscript. Frazier, L. (1987). Theories of sentence processing. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing (pp. 291–308). Cambridge, MA: MIT Press. Garfield, J. (1987). Modularity in knowledge representation and natural-language processing. Cambridge, MA: MIT Press. Goldberg, A. (1995). Constructions: A Construction Grammar approach to argument structure. Chicago: University of Chicago Press. Goldsmith, J. (1976). Autosegmental phonology (Doctoral dissertation, MIT) Indiana University Linguistics Club, Bloomington. Grimshaw, J. (1997). Projection, heads and optimality. Linguistic Inquiry, 28, 373–422. Jackendoff, R. (1972). Semantic interpretation in generative grammar. Cambridge, MA: MIT Press. Jackendoff, R. (1987). Consciousness and the computational mind. Cambridge, MA: MIT Press. Jackendoff, R. (1997). The architecture of the language faculty. Cambridge, MA: MIT Press. Lackner, J. (1981). Some contributions of touch, pressure, and kinesthesis to human spatial orientation and oculomotor control. Acta Astronautica, 8, 825–830. Lackner, J. (1988). Some proprioceptive influences on the perceptual representation of body shape and orientation. Brain, 111, 281-297. Landau, B. & Jackendoff, R. (1993). “What” and “where” in spatial language and spatial cognition. Behavioral and Brain Sciences, 16, 217–238. Lerdahl, F., & Jackendoff, R. (1983). A generative theory of tonal music. Cambridge, MA: MIT Press. Levelt, W. J. M. (1989). Speaking. Cambridge, MA: MIT Press. Liberman, A., & Studdert-Kennedy, M. (1977). Phonetic perception. In R. Held, H. Leibowitz, & H. L. Teuber (Eds.), Handbook of sensory physiology: Vol 8; Perception. Heidelberg: Springer-Verlag. Macnamara, J. (1978). How do we talk about what we see? Manuscript, McGill University.
30
Ray Jackendoff
Marslen-Wilson, W., & Tyler, L.K. (1987). Against modularity. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing (pp. 37–62). Cambridge, MA: MIT Press. May R. (1985). Logical form: Its structure and derivation. Cambridge, MA: MIT Press. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature (London), 264, 746–748. Piñango, M. M., Zurif, E., & Jackendoff, R. (1999). Real-time processing implications of enriched composition at the syntax-semantics interface. Journal of Psycholinguistic Research, 28, 395–414. Pollard, C., & Sag, I. (1994). Head-driven Phrase Structure Grammar. Chicago: University of Chicago Press. Prince, A., & Smolensky, P. (1993). Optimality Theory: Constraint interaction in generative grammar. Piscataway, NJ: Rutgers University Center for Cognitive Science. Sadock, J. M. (1991). Autolexical syntax. Chicago: University of Chicago Press. Sag, I., & Pollard, C. (1991). An integrated theory of complement control. Language, 67, 63–113. Talmy, L. (1978). The relation of grammar to cognition—A synopsis. In D. Waltz (Ed.), Theoretical issues in natural language processing 2 (pp. 14–24. New York: Association for Computing Machinery. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634. Van Valin, R. D., & LaPolla, R. J. (1997). Syntax: Structure, meaning, and function. Cambridge: Cambridge University Press.
Chapter
2
Remarks on the Architecture of Language Processing Systems Merrill Garrett
THE GOALS OF THE DISCUSSION I will note some of the recurrent conflicts among experimental outcomes that rely on behavioral indices of parsing (various response time and error measures). These controversies may be understood in terms of two additional data classes: a recent series of eye-movement studies that supports strong interactive claims re parsing, and an equally recent series of event related potential (ERP) studies that supports modular systems. I will argue that the various conflicts between empirical findings require a resolution in terms of a filtering model, and sketch the outline of such a proposal. The essential feature of the resolution lies in using the language production system as the source of the apparent conceptual and discourse level constraints on parsing. Further, because the evidence from language production itself indicates a modular architecture, this approach is one that embeds the full range of interactive processing claims in an architecture that preserves the essential correspondence between the informational structure in formal grammar and the processing types in real-time language performance systems.
A BACKGROUND ASSUMPTION When we contemplate language processing questions, there are multiple sources that we draw on to constrain theoretical claims. These begin with formal grammars, and should in a quite real sense, end with formal
Language and the Brain
31
Copyright © 2000 by Academic Press. All rights of reproduction in any form reserved.
32
Merrill Garrett
grammars. When we ask how words are learned, recognized, pronounced, integrated in utterance, or interpreted in their discourse ensembles, we pose those questions in terms of structural features that grammars associate with words as a function of their embedding in grammatical rule systems. We know at the outset that the classes of structure that capture the major distributional features of language expression will govern our attempts to discover real-time information processing accounts of language comprehension, production, acquisition, evolution, and dissolution. The vocabulary of processing descriptions incorporates phonetic, phonological, morphological, lexical, syntactic, and semantic variables. Formal grammars array these facets of language structure in somewhat variable ways, modulo some variation in the goals of linguistic theory embraced by different theoretical camps (see Newmeyer, 1998, for an enlightening historical reprise of the contemporary scene in linguistics). But, withal, the major threads are woven in impressively similar design: aspects of form and content lead intertwined but distinct lives. Grammatical structure is diverse across languages in this regard, but it is not haphazard. The theories of real-time language use that our research generates must afford an answer to the question of how languages could display the structure they do, given the processing account we offer. Processing accounts that fail such a test require amendment, extension, or abandonment.
SOME EXPERIMENTAL EXAMPLES AND PERSPECTIVES FROM PARSING STUDIES Research in comprehension has played out against a background of claims for the centrality of syntax in the architecture of processing systems. Modular systems theory has assigned a central role to the construction of syntactic representations for sentences during the comprehension process (e.g., Forster, 1979; Foder, 1983; Frazier, 1990; Gorrell, 1995; Crocker, 1996). By contrast, some accounts that emphasize interactive processing perspectives eschew specifically syntactic representation in favor of a direct mapping from lexical inputs to a discourse model via a mixed computation that combines syntactic, semantic, and pragmatic constraints (e.g., Marslen-Wilson & Tyler, 1987). Interaction can be described in several different ways, and not all interactive proposals treat relations among the potential information types in the same ways—see, for example, Altman and Steedman (1988) for a discussion that highlights important distinctions. More recently, some varieties of constraint based parsing rely on an ensemble of statistically based conditions on the use of lexical
2.
The Architecture of Language Processing Systems
33
items that includes a range of linguistic and nonlinguistic properties, without principled distinctions among the types (McClelland, St. John, & Taraban, 1989; MacDonald, Pearlmutter, & Seidenberg, 1994; Spivey & Tannenhaus, 1998). Interpretation is based on the lexical string and the bundle of interacting constraints associated with each element; interpretation is direct and without an identifiable mediating syntactic representation. There is, of course, theme and variation among these positions, with varying takes on the relevance of syntactic representations and on the relation of the processing claims to grammatical theory. I mean here only to swing a broad brush. The first thing to note is that all theoretical camps have their ammunition. We do not lack for observations that testify to seemingly systematic aspects of the language comprehension profile. They do not line up in sterling orderly rows, however. On the one hand, numerous studies report findings that parsing performance is unresponsive to clearly available semantic bias: research with rapid serial visual presentation (RSVP), self-paced reading, cross-modal priming, naming and lexical decision latency, and several cousins and variants of these provide examples of processing circumstances in which plausible construals of sentence contents would dictate analyses other than the one listeners or readers appear to (momentarily at least) embrace. Frazier (1999) in a recent book remarks on the import of the intuitions we have about our processing of this example: a gift for a boy in a box Intuition testifies what systematic experiment affirms: the boy is initially in the box. Why indeed is the plausible attribution not the governing one? Evidently because the processes of phrasal assignment and interpretation of the consequent structure are “faster” than the inference that gifts are more likely to be in boxes than are boys. Various answers can and have been offered, and we will touch on some in later sections. But, examples like this one show that rather general terms (boy) linked to structurally versatile elements (in) may resist the influence of specific and available commonsense relations (gift in box) while accepting uncommon ones (boy in box). The message that this conveys to many observers is that interpretive constraints on the processing of sentence form are distinct from syntactically driven ones. The experimental literature provides a variety of illustrations implicating a wide range of structures [see references cited above; see also Frazier & Clifton (1996) for pertinent further discussions]. The example cited above implicates prepositional phrase attachment—a favorite stalking horse for debates about syntactic and semantic/pragmatic interaction to which we will return. I will say little
Merrill Garrett
34
more about indicators that available semantic and conceptual constraint have limited impact on parsing decisions and will spend more ink on evidence for the contrary position for reasons that will become obvious. So, on the other hand, we may likewise cite a variety of rather similar studies reporting results supporting the claim that lexical recognition and parsing procedures are intrinsically subject to semantic bias. To illustrate briefly, a long-running experimental attack on the modularity question is represented in the work of Marslen-Wilson and Tyler (their 1987 paper reviews several relevant findings). A representative example (Tyler & Marslen-Wilson, 1977) illustrates an important point about the sort of contextual constraint they tested. It focused on the likelihood of interpreting an ambiguous phrase like “landing planes” as a plural noun phrase (NP) (“Landing planes are very noisy near an airport”) or a singular gerundive nominal (“Landing planes is dangerous in a storm”). The experimental effect of an immediately prior context (“If you are standing near the runway . . . ” vs. “If you are not a trained pilot . . .”) was evaluated by reaction time for choice between the variants of the linking verb “is/are” following the ambiguous phrase. Reaction time (RT) was faster for targets compatible with the prior context sentence. The significant point is that the interpretive force of the context was available within the few hundred millisecond time frame required for a lexical choice that fit one of two competing syntactic analyses of the ambiguous phrase. The contextual constraint is at the sentence/discourse level. Work by Crain and Steedman (1985) and others also implicates interpretation of discourse structure in parsing. That influential work indicates that the likelihood of interpreting a verb phrase as a relative clause modifier rather than as a main verb is influenced by the discourse setting: contrasts between single protagonist settings versus dual protagonist settings (for NP interpretation) shows the latter promotes relative clause analyses [i.e., fitting with continuation (a) rather than (b) in the example]. the teachers taught by the Berlitz method . . . (a) . . . were very successful
/
(b) . . . but couldn’t get jobs anyway
The significant observation is that the tendency to analyze the material following the initial NP (e.g., “the teachers”) as a relative modifier is enhanced by appositeness to the discourse (i.e., when a relative is needed to distinguish among potential referents in the discourse). This work is one of several to address the analysis of garden paths in reduced relatives of the venerable “horse raced past the barn” ilk (Bever, 1970). Other work by Trueswell and Tannenhaus (1994) with the same structure reports that
2.
The Architecture of Language Processing Systems
35
an inanimate NP eliminates the garden path effect so generally observed; for example: the fossil examined . . . versus the archeologist examined . . . Here the focus is on lexical effects rather than the broader scale of the discourse. Of particular interest to the discussion here are studies of lexical constraints that evaluate argument structure effects (see Carlson & Tanenhaus, 1988). These implicate a combination of subcategorization frames and thematic structure, namely, the structural configurations that can be associated with a lexical item and the semantic/conceptual roles (agent, theme, . . .) occupied by the NPs in the structure. With respect to the first variable, subcategorization frames, questions may be posed both regarding the extent to which detailed subcategorization information is immediately recruited and, to the extent that it is, how the information is deployed. There is evidence for effects that seem to ignore subcategorization (e.g., postulation of a trace following an intransitive verb; see, for example, Mitchell, 1994, among others), and evidence that supports early availability and application of such information. These latter are combined with assessment of whether different verbs have different preference orderings for the class of structural options that can be associated with them. So, for example, a comparison may involve verbs that prefer a direct object versus those that prefer a sentence complement (from Trueswell & Kim, 1998): The man accepted the prize. The man accepted (that) the prize was not going to him. For the example pair, the direct object analysis with prize as theme is preferred over the sentence complement analysis. By contrast, for a verb like “expected”, the preference would be shifted more into balance between the two options, and for a verb like “realize,” the sentence complement analysis would be strongly dominant. Self-paced reading tasks and eyemovement studies show reprocessing effects for the complement versions that are interpretable as reflections of these differences (e.g., reprocessing in the disambiguating regions following verbs like “accept” and “expect,” but not verbs like “realize“). One may, as Trueswell and Tanenhaus (1994) and several others have done, profitably combine subcategorization bias with thematic role assignment, for the latter reflect plausibility factors: the readiness with which available NPs for argument slots suit their thematic roles affects processing time. The demonstration of lexical biases
36
Merrill Garrett
and of the interaction of plausibility with those biases is a matter to which we will return. A considerable body of work on parsing examines the analysis of prepositional phrases, namely, the structure with which we began this brief excursion, and which, by the observation made there, clearly may run counter to locally available practical knowledge derivable from the lexical elements in the target sentence. But, work by various investigators (see, e.g., McClelland et al., 1989) argues that the likelihood of interpreting a prepositional phrase as an instrumental adverbial phrase (“they saw the man with the binoculars but couldn’t recognize him”) or as a reduced relative clause modifier (“they saw the man with the binoculars had a red hat on”) can be influenced by the relative plausibility of the relations among the lexical constituents. Clearly, the matters of timing and measurement, plus the nature of the constraint type will bulk large in sorting through the experimental space. In these and other examples, we see different kinds of nonsyntactic interpretive constraint at work. Note the very important differences between the application of constraints that inhere in the meaning of individual lexical items and those that depend on contingent facts merely associated with the interpretation of a word, or that cannot be determined from an individual lexical entry at all but instead arise via the interpretation of phrase, sentence, and discourse. One will see the import of a processing effect that relies, for example, on the fact that dogs are animate physical objects in a different light than one that relies on our knowledge that they may tend to bite, or do not read newspapers, or that one may be alleged to have only three legs. This is true both in terms of practical matters such as the likely time course of access to different informational classes and in terms of the import of observed effects of the various constraints vis-à-vis the rationalization of relations between grammars and parsers. A particularly salient example of this is the following: recent studies of eye movement suggest that mere observation of the layout of a visual array can directly affect the attachment preferences for prepositional phrases. Using eye-tracking measures, Tanenhaus et al. (1995) report that attachment follows the implied contrasts in a visual array: “put the apple on the napkin” is variously interpreted—and immediately so, re the eye pointing measure—depending on whether an unambiguous locative analysis or a relative modifier is apposite. To observe that the mere experience of a visual scene directly translates to a linguistic bias seems the very epiphany of nonmodularity. Were it so in just those terms, what would be left of the notion of informational encapsulation of linguistic representation? A less tendentious way to put the matter is that some mechanism brings the conceptual force of the visual information to bear on language with great rapidity. Is it a parsing influence? I set this aside for
2.
The Architecture of Language Processing Systems
37
the moment, but note that such a case highlights the need for an account of why constraints are variously effective. If the informational representations that drive language processing were, in fact, genuinely integrated as some interpretations of the eye-movement data would have it, what stops the seamless workings of this in other experimental situations? Overall, outcomes are mixed and the picture is one that defies easy summary. Multiple structural types have been evaluated. Some appear to be more susceptible to interpretive manipulation than others. So, for example, prepositional phrase attachments have come in for a great deal of attention—in part because of the substantial variation one can induce in attachment preference as a function of the interpretation of the sentence. By contrast, the effects of semantic constraint on direct object/sentence complement ambiguities (as in “the child knew the answer to the problem was in the book”) are murkier. Experiments that focus on implicit structural elements (e.g., empty categories or structural traces of varying sort) present still another variable scene (see, e.g., Nicol & Swinney, 1989, for some general commentary). Given the state of affairs just described, one might suppose that we are not in a position to draw confident conclusions about how modular or nonmodular syntactic processes in language comprehension may be. Why does the biasing work sometimes and not others? For some tasks, but not others? And what does it mean vis-à-vis the general question of the underlying architecture if we accept that there are significant materials and task specificity to these effects? Lots of folk would argue they can provide the answers to such questions strictly within the domain of design and paradigm. But I doubt that the answer lies in systematic incompetence on either side. At the center of this is the question whether the primary determinants of the variability are principled in terms of processing types, or merely adventitious—accidents of strength and timing. A paper by Bever, Sanz, and Townsend (1998) puts the matter in terms of a set of propositions of the debate (here, slightly rephrased): If syntax is modular, nonsyntactic information can’t affect it. Syntax processing is absolute rather than probabilistic. Sentence comprehension could rest solely on distributed cues of diverse type, without recourse to sentence level syntax. If syntax is necessary for comprehension, syntactic assignment must precede interpretation. Syntactic properties of sentences are not affected by conceptual beliefs. All are denied under their analysis for various reasons—some of which I will take note of here. The resolution of the contrasting views as suggested by Bever et al. incorporates the constraint satisfaction approach
38
Merrill Garrett
into a system of strategies that is applied for initial analysis. But, the proposal preserves syntactic processing as a backup and confirmatory system, and as a system that provides deeper and more sophisticated analysis of sentence form. Detailed syntax is a late processing phenomenon on this view. This constitutes at bottom an analysis-by-synthesis (ABS) procedure that embeds the grammar as a generative source (see, e.g., Fodor, Bever, & Garrett, 1974, for an early but still useful exposition). The solution that is pursued structures the architecture of the comprehension system so that it accommodates the interactive observations, while preserving a role for specifically syntactic representation in the ABS system. This proposal seeks to avoid the “mystery” of why we need a grammar in the context of comprehension mechanisms that ignore the typology of information that grammars robustly display. It does so by postulating that grammars are necessary psychological components in two senses: First, they are essential to account for language acquisition. Bever et al. (1998) note with some justice that development of a distributed network representation of grammatical targets requires targets to train on. There is no bootstrap to be fished out of the constraint sea for this purpose. And, second, they afford a principled resource for evaluation and revision of analyses delivered by the constraint satisfaction comprehension system. This seems cogent, but I am inclined to be even more exigent. Aspects of this approach are quite attractive as a synthetic effort. But, there are some features of the proposal that I have reservations about, and some empirical observations that suggest the need for specifically syntactic processes that are earlier than a late stage cleanup and elaboration function. Various experimental findings suggest that basic syntactic structure is assigned very rapidly (e.g., differences in the recovery of sentence content for syntactically well formed and ill formed sentences presented in RSVP and same–different matching modes; see Forster, 1979, for an instructive analysis of some rate issues). It may be that here, as has from time to time been suggested, one should distinguish between early stage parsing that is of rather limited scope and later stage parsing that has deeper structural issues to resolve. The Bever et al. treatment has that logical character, but treats the early stage as genuinely different in kind from the later stage, namely, an “approximate, strategic mode”—done in whole or in part by constraint satisfaction methods versus a “rule mode” of symbolic calculation. Though I think the behavioral evidence for early stage, specifically syntactic operations, is quite relevant, I will not pursue it here, in part because there is lots of discussion available without my two cents worth, and in part because I want to focus on evidence of similar import that derives from electrophysiological studies of language processing. The con-
2.
The Architecture of Language Processing Systems
39
vergence of such data with the behavioral patterns that show modular architecture is quite forceful.
THE ELECTROPHYSIOLOGICAL RECORD Recording systems using scalp electrodes for detection of electrical activity in the brain have been with us for some decades (eeg systems). But the most recent decade has seen the culmination of many years of patient effort by a relative handful of very able investigators who were seeking to develop the necessary technology and methodology to examine cognitive processes with this tool. Their work has combined with the advent of new microelectronic and computational tools to make the approach much more generally available to the scientific community. Much of the resulting research of relevance to our purpose here uses event related potential (ERP) techniques to look for characteristic patterns of brain activity elicited by informationally (e.g., linguistically) defined stimulus classes. Electrical records are time-locked to stimulus onset, and the records for many instances of a given stimulus type are averaged across trials to extract the regularities of response to the type. Evidence from that work and from related studies using brain imaging technology indicates the existence of brain systems responsive to specifically syntactic properties of sentence structure. These syntactically driven ERP patterns can and have been quite systematically compared and contrasted with electrophysiological complexes that show selective response to interpretive aspects of sentence structure. The structural properties that give rise to the former are readily associated with conventional grammatical distinctions. The central presumption for interpretation of these patterns is straightforward: If one presumes that the linguistic analyses we work with in language are partly or largely artifacts of linguistic analysis and not representative of the computational structures that are responsible for language use, then we would not expect to find uniform brain consequences of the distinctions drawn by such artifice. Several reviews provide excellent introductory treatments of the methodology for language studies and summary descriptions of recent findings. I, therefore, provide little such assistance here; see, for example, Coulson, King, and Kutas (1998); Friederici (1998a); Hagoort, Brown, and Osterhout (1999). Some salient points are cited below for brief illustration of the range of observation in play for purposes of the argument underway in this essay; reference to “N400,” “P600,” and “LAN” follows current labeling practices for the major response types of interest.
40
Merrill Garrett
ERP responses to semantic congruity were established by Kutas and Hilyard (1980), namely, variation in the N400 response to “goodness of fit” of a lexical item to its sentential context. And from the outset, and in many subsequent studies, N400 was shown to be indifferent to syntactic well-formedness. Neville et al. (1991) reported syntactically specific effects in a direct comparison of semantic and syntactic violations in closely matched sentence pairs visually presented. They reported an early negative polarity complex largely focused on left anterior recording sites and emerging around 250–300 ms poststimulus (i.e., timed from the point of first logically possible detection of the deviance in the string), with maximum amplitude around 400 ms (a pattern now generally referred to as an LAN response for left anterior negativity). It was elicited by major category violations and by a wh-movement violation. That pattern is similar in timing and configuration to the N400 response elicited by semantically implausible lexical targets, but it appears at quite different recording locations from those associated with the N400. In fact, when the two violation types were contrasted in the same sentence frames (as Neville et al. did), the two responses were complementarily distributed vis-à-vis recording sites: for semantic violations, primarily posterior sites were activated, with greater prominence on the right, but for syntactic violations, sites were strongly left anterior. On that evidence, distinct patterns of neural activity arise from the two types of violations. Osterhout and colleagues in several publications have reported a strong and sustained positive polarity shift observable bilaterally and at most recording sites starting around 500 ms and peaking at 600–700 ms after the point of a syntactic violation. A group at the Max Planck Institute in Nijmegen led by Hagoort and Brown has also provided extensive analysis of this ERP complex for Dutch language materials. This response (P600) is observed for agreement violations and subcategorization violations as well as for phrasal violations (Hagoort, Brown, & Groothusen, 1993; Osterhout & Holcomb, 1992; Osterhout, Holcomb, & Swinney, 1994); this prominent positive shift response was also observed for Neville et al.’s (1991) phrase structure violations. Though it is rather later than either the LAN or the N400, it is well to keep in mind that the temporal indicators with this methodology, while powerful and affording considerable precision, have some limitations. In particular, we are not entitled to assume that the processing events that cause the ERP pattern did not arise until the point of our ability to detect it with our measure. We can be confident they arose no later than that point but the possibility exists that a chain of events is set in motion by the stimulus features at some earlier point (but does not immediately “surface” for our observation), and is detectable for us only after some delay. Having said this, I will neverthe-
2.
The Architecture of Language Processing Systems
41
less at a later point entertain some intriguing speculations that use the observed temporal order of the various semantic and syntactic responses as their base. Two additional features of work by the Nijmegen group on P600 patterns for Dutch language materials are of special interest here. One is their observation of an N400 complex that often appeared at ends of syntactically deviant sentences. They suggested this might reflect a kind of interference (arising from the earlier occurring syntactic analysis problems) with closure or interpretive “wrap-up” processes of semantic integration for the sentence as whole. On that understanding, this effect is an indication of some contingent relations among the hypothesized processing types. Yet another demonstration of this from their work uses “syntactic prose” (much in the manner of earlier word monitoring experiments by Marslen-Wilson & Tyler, 1980) in which the test “sentences” mirrored the syntactic organization of their initial experimentation (including all the violation types) but lacked semantic coherence. In such sentences, structure is signaled only by sequence of grammatical class and inflection because the major content items are randomly substituted for (with preservation of grammatical class), and hence no sensible interpretation is supportable for the successive constituent groups of the structure—nor, of course, for the string as a whole. Under these circumstances, they observed the same P600 patterns as for the matched normal and syntactically deviant but interpretable stimuli, but did not observe the N400 wrap-up response (Hagoort & Brown, 1994; see also Hagoort, Brown, Vonks, & Hoeks, forthcoming). This reinforces the claim for a syntactically driven P600 and the general impression of linked but distinct processing types. Friederici and collaborators have reported several similar outcomes in studies of German that have looked at phrasal violations, subcategorization violations, and gender effects among others. But, that work has focused greater attention on the earlier language components and indeed some of their work has indicated both an LAN pattern at around 400 ms and an LAN pattern for some violations arising with somewhat shorter latency—an “early, left, anterior negativity” (ELAN) (Friederici, Hahne, & Mecklinger, 1996). An early left negativity has also been remarked by Neville in her work (see below). I will not further pursue the ELAN/LAN distinction here, but it is one of substantial interest. A revealing feature in some of these studies in German is the comparison of E/LAN and P600 responses to variations in strategic responses by subjects (viz., as effected by instruction to attend to or ignore sentence interpretations, or by changes in the relative proportions of the violation types). In a number of instances, P600 responses were modified by such manipulations, indicating some greater sensitivity to the comprehender’s processing priorities and/or
42
Merrill Garrett
attentional commitments, but the E/LAN responses were not so affected, indicating no such dispositional capacity (Hahne & Friederici, in press). Further, when violations were constructed at a given point in the test string such that there were either semantic only, syntactic only, or both semantic and syntactic violations, interactions were observed. When occurring separately, each elicited its characteristic wave form (LAN and P600 for the syntactic violations, and N400 for the semantic violation), but when combined, only the LAN appeared (Friederici, Hahne, & von Cramon, 1998). Again, as in several other result patterns already mentioned, there is an indication of contingent relations among the processing types. Exactly how one should seek to reconstruct these might vary, but the message is that the observational base has the potential to tell us about this kind of issue, and these types of investigations hold the prospect of significantly finer discriminations among parsing processes with this methodology than we might have anticipated. See Friederici (1998b) for a proposal that associates the E/LAN responses with automatic syntactic analysis and the P600 with processes of reanalysis and repair of syntactic structure (e.g., temporary ambiguity and garden path effects). There are many other quite fascinating clusters of observations available and/or rapidly developing with ERP measurements. For example, Coulson et al. (1998) review evidence for the view that there are multiple LAN effects, some of the sort I have noted above, but others better understood in terms of working memory mechanisms that are involved in dealing with discontinuous dependencies in sentence processing. Recent observation also suggests that the P600 may reflect multiple aspects of sentence processing that can be distinguished in terms of responsible recording sites (Hagoort & Brown, in press). In another vein, reports by Neville, Mills, and Lawson (1992) on ERP tests of deaf readers of English reveal some systematic differences in ERP responses to vocabulary classes that are correlated with major contrasts in syntactic function (“closed class” and “open class” elements). Neville’s work also combines this observation with functional magnetic resonance imaging (fMRI) studies that bear on issues of localization, and these outcomes are congruent with both ERP and imaging based claims for localization in classic areas of language specialization. Work by a number of the cited investigators and others likewise pursues convergence of ERP and imaging results as well as extensions of these techniques to the study of patients with brain injury who display aphasic patterns of selective loss of grammatical capacity (see, e.g., Brown, Wassenaar, & Hagoort, 1999; Friederici, von Cramon, & Kotz, 1999). There are, in short, many more aspects of the electrophysiological and imaging findings that can be brought to bear on the effort to identify the contributing processes to real-time language use. So, I em-
2.
The Architecture of Language Processing Systems
43
phasize that the sketch that I have provided here is just a sketch, and is to be understood as a (representative) sample taken to illustrate the basis for the argument I am pursuing in this essay, not as a systematic representation of the field of enquiry being drawn on. Observations based on correlations of brain response and language processing performance provide persuasive evidence for distinct brain systems associated with several different structural domains, and these domains have a natural interpretation within the framework of grammatical theory: phrase structure, inflection, movement and binding, and lexical semantics. And, there is good reason to consider some of the processing evidence derived from such study as indicating strategic and dispositional responses to both semantic and syntactic task priorities. These latter may be distinguished from the former in terms of both electrode location (and plausible construal of that data in terms of brain generator location), and in terms of time course of processing. So, one may remark a system response to syntactic configuration within 200 ms or less, and reflections of the integrative effects that unite syntactic and semantic factors following closely, but with variable relation to the initial syntactic response. Some salient conclusions that seem indicated by data of the type just reviewed are as follow: Semantic integration of lexical elements into their context is distinct from syntactic integration of lexical elements into their structural frame. More than a single syntactic signature is demonstrable and they span a substantial time range—approximately 150 ms to 700+ ms. Early responses to syntax are inflexible and are localized to a substantial degree in classic language processing areas associated with syntactic processing (viz., based both on language impairment data and brain imaging data). Later responses to syntax show greater sensitivity to interpretive factors than do the early responses and are much more broadly distributed. Though the dissociation of the processing types cannot tell us how syntactic processing fits into the overall language processor, it does rather convincingly indicate that the identification of such a structural type is not an artifact of linguistic analysis.
LANGUAGE PRODUCTION STUDY Any general evaluation of the architecture of language processing systems must address not only the character of language comprehension, but also that of language generation. Several issues immediately present
Merrill Garrett
44
themselves, not the least of which are important differences between production and comprehension regarding the intrinsic relations between conceptual, semantic, syntactic, and phonological processes. And, as I will momentarily review, given such differences in the way the two classes of processing tasks relate to conceptual and semantic systems, the production evidence shows a surprising degree of modular structure. This is central to the current discussion, for I wish to argue that linkages between production and comprehension systems may rationalize some conflicting results outlined in the discussion of parsing studies, and thus may lend support to a somewhat more uniform treatment of the processing relations between syntactic structure and various classes of interpretive constraint. Language production models must account for the real-time integration of utterance form in the service of the specific force of the meaning that a speaker wishes to convey on a given occasion of utterance. So, language production models are driven in the first instance by an interpretation of conceptual content that gives rise to the communicative intent of a speaker. Language comprehension models, by contrast, are driven in the first instance by the acoustic or orthographic input that is the target of interpretation. Given these differences, one might, with no other facts available, expect the primary processes of sentence analysis/construction, and particularly those for phrasal structures, to directly and pervasively reflect the nature of the controlling inputs in the two cases. But the apparent case is somewhat different.
Figure 1
A model of production processes, adapted from Levelt (1989).
2.
The Architecture of Language Processing Systems
45
Diverse strands of investigation underlie claims for models of the language production system. Levelt (1989) provided an admirable summary of the evidence then available as well as a comprehensive interpretation and elaboration of language production theory. Research in the ensuing decade has extended the observational base and tested the general architectural claims he laid out in a variety of ways. Figure 1 provides a summary picture of the organization of the production system. Modular processing in both lexical and phrasal domains is represented. My own work and that of a number of others examining the distributional character of spontaneously occurring speech errors indicates such a multistage process. It is one in which abstract lexical representations (i.e., phonologically uninterpreted objects, referred to as lemmas in Figure 1) are integrated into a syntactic representation, followed by processes in which the phonological form of utterance components and associated prosodic structures is developed. Semantic control is exercised in the initial stage of lexical and phrasal selection, but is not directly implicated in the mechanisms of phrasal integration or phonological interpretation. Data used to motivate this picture include patterns of word substitution errors, as well as various movement errors of sound and word exchange or anticipation or shift in location that allow one to infer what the computationally effective descriptions sentence elements may be at different points in the encoding sequence. Many detailed constraints of syntax, morphology, phonology, and prosody may be remarked in the error patterns (Fromkin, 1971), but semantic constraints on the interaction of sentence elements at syntactic and phonological levels are not readily discerned (Bierwisch, 1982). Certainly, one may see powerful semantic influences on certain word substitution errors, but these appear to be best characterized as errors in the initial selection of lexical content, that is, errors prior to the operation of the mechanisms that take semantically selected elements and incorporate them into syntactic and prosodic structures (see, e.g., Garrett, 1980, 1993a, for a summary review of these arguments). Experimental work by Levelt and colleagues (see Levelt, Roelofs, & Meyer, 1999, for a review and citation of many relevant research reports) has addressed a variety of issues in this area, with special force in the domain of lexical retrieval for syntactic integration. So, for example, a series of experimental investigations with picture–word interference tasks has explored the interfaces between the conceptual/semantic levels, lemma levels, and word form levels depicted in Figure 1. That work has provided the basis for quite detailed accounts of the time course of activation of the different classes of structure associated with lexical targets, with rather compelling indications for staged recovery of semantic and phonological description. A related line of reasoning applies to the study of tip of the
46
Merrill Garrett
tongue (TOT) states, in which one sees a prima facie sharp dissociation between semantic and phonological information. Recent experimental work with TOT states has indicated that more precise boundaries may be drawn that correspond to the experimentally certified structure based on the picture–word interference paradigm and the spontaneous word substitution patterns noted above: in TOT states, distinguishable access to semantic, syntactic, and phonological representations may be identified (see, e.g., Vigliocco, Antonini, & Garrett, 1999). To this list, one may add a convergence of the reaction time and error profile studies with ERP investigations. In picture naming tasks, it has been demonstrated that syntactically constrained responses precede phonologically constrained responses as evidenced by differences in timing of motor readiness potentials (van Turennout, Hagoort, & Brown, 1998). I should, however, not leave this brief discusion without emphasizing that I have set aside many interesting and important issues regarding the internal structure of the processes just outlined, and in particular issues concerning the degree and role of feedback relations between phonological, syntactic, and semantic processing stages. The general architectural contrasts under discussion are well attested; the detailed relations among the component systems are the focus of considerable debate. For example, a paper by Dell, Schwartz, Martin, Saffran, and Gagnon (1997) reports some combined modeling and experimental work that is important to our understanding of the conditions that underlie potential interactions between levels. Against that background, they provide an insightful perspective on the general issues of modular systems in networks. There is also in the earlier cited Levelt et al. (1999) review a carefully considered assessment of various experimental problems as they relate to the picture–word interference paradigm and the dynamics of lexical recognition and retrieval. Work by Kathryn Bock and colleagues is particularly revealing with respect to ways in which interpretive and lexical constraints interact with sentence construction (see Bock, 1990; Bock & Levelt, 1994, for reviews). Several aspects of this program of study are relevant, including work on memory (e.g., Bock & Warren, 1985), agreement processes (e.g., Bock & Eberhard, 1993), and work on “syntactic priming” (Bock, 1986), and my brief remarks will not do justice to the breadth and subtlty of the inquiry. I will focus for illustration on aspects of the syntactic priming work. In such experiments one can affect the relative likelihood of producing a given syntactic type in a picture description episode (e.g., active vs. passive, or double object constructions vs. to-datives) by manipulating the syntactic form of a sentence generated on the just preceding trial; for example, passives are more likely to be used for the picture description task if preceded by a passive prime than an active prime. N.B.: The prime sen-
2.
The Architecture of Language Processing Systems
47
tences and the picture description targets are unrelated in content vis-àvis objects referred to and the cover task used in the experiments obscures the experimenter’s interest in the successive trial relations. This work provides a clear demonstration of a processing stage/representation that is sensitive to phrasal configuration but not to lexical content, and more particularly, not to the semantic force of the lexical content. So, for example, work by Bock and Loebell (1990) showed that syntactic priming effects treat as equivalent objects the by-phrases in sentences like these: the plane landing by the control tower versus the plane landed by the pilot Thematic roles differ but syntactic configuration is preserved. By contrast, when configuration changes, though prosody and superficial sequencing of words is preserved, priming is lost (as for the pair: “Susan took a book to read”; “Susan took a book to class”). Note that these findings do not arise via the use of techniques that are not demonstrably sensitive to influence by interpretive variables. So, for example, animacy and thematic role have been examined in similar tasks (Bock, Loebell, & Morey, 1992) and shown to have clear effects on the relative likelihood that a given NP will occupy a particular argument slot, but such effects do not interact with the priming of phrasal configurations. This is a kind of outcome that is similar in spirit to the idea derivable from speech error distributions that there is some degree of computational separation between the selection of lexical elements that are embedded in a sentence structure and the integration of the structure itself. Note that this claim does not run counter to lexically driven encoding schemes, but it does not fit versions that do not countenance a distinguishable syntactic representation. Quite the contrary, as the priming patterns provide a prima facie case for the existence of an abstract structure that encodes syntactic configuration but not lexical content. These several kinds of findings indicate a significant degree of modular structure in the language generation system, with global outline as in Figure 1. With these observations about language production in hand, let us return to the problem we began with—that of how one should deal with the contrasting patterns of evidence for interaction and modularity in the domain of comprehension as discussed in the previous two sections.
ANOTHER PATH: LANGUAGE PRODUCTION AS A COMPREHENSION FILTER It surely looks as though one should grapple with the idea that in comprehension studies there is simultaneous good reason to aver both
48
Merrill Garrett
modular and nonmodular processing profiles. If one agrees, there are several pretty puzzles that present themselves in trying to do this. So, if one were to accept at face value the claims for constraint based processing that is genuinely nonmodular (in the relevant informational structure sense), one must find some way to account for the experimental circumstances in which semantic, conceptual, and idiosyncratic distributional factors seem to have little impact. The constraint system is well suited to generate interactions. But, its capacity to account for circumstances in which such are not forthcoming is less obvious. Similarly, the emerging electrophysiological and brain imaging profile of language processing, as well as long-standing observations of language impairment that show selective losses vis-à-vis information type must find an explanation that does not rely on a comprehension model in which informational sources are combined statistically and without regard to type. The line taken by Bever et al. (1998) addresses some of the experimental studies of parsing that show interaction and brings them into an accord with some of the evidence for abstract syntactic representations as relevant to processing. But that avenue does not in my judgment as readily deal with instances in which interaction does not manifest itself in early stage parsing, nor with the electrophysiology patterns, or the structure of language impairments. My inclination in these circumstances is to pursue an accommodation that is similar in general character to Bever et al.’s proposal, but without the assumptions of centrality for constraint based sentence analysis and “slow syntax,” and with a different source for the interpretive constraints that appear in parsing performances. That accommodation calls for an integration of the language comprehension and language production systems in ways that preserve the identity of both, but assigns production processes a prominent role in the routine function of the comprehension system. We are quite accustomed to agree that language generation and language comprehension systems are intimately linked via monitoring functions of the latter on the former. That is, we are quite accustomed to think of the comprehension system as providing a continual error control mechanism via monitoring of production processes (see Levelt, 1989, for a review). It has occasionally been briefly noted (Forster, 1979; Jackendoff, 1989; Garrett, 1993b) that production systems should be viewed as having a functionally similar role with respect to comprehension, namely, that they provide a continual error control mechanism via the production monitoring of partial products of the recognition system. A somewhat broader way of putting this point is to think of the production system as a filter on the generation of multiple analyses by the recognition system. There is considerable reason to take seriously the view that recognition
2.
The Architecture of Language Processing Systems
49
systems respond to local ambiguity in the analysis of input strings by maintaining, at least temporarily, multiple analysis paths that are pruned in terms of information gleaned either from posterior context or in terms of higher order interpretive constraints. The idea that I wish to pursue is that the production system should be understood as the major factor in the application of such interpretive constraints in sentence comprehension. It is, in fact, quite plausible to assume that the language production system can rapidly incorporate the initial lexical elements and elementary phrasal structures identified by the recognition system with existing discourse constraints to give rise to candidate sentence structures. On the production hypotheses sketched in the previous section, lexically driven sentence encoding under message control is what the production machinery is specifically designed to do. What reasons, apart from mere possibility, do we have to pursue this line? Here are some straws in this theoretical breeze: • The lexically based character of production routines matches many features of current parsing study (e.g., subcategorization and thematic structure; computational efforts at merging parsing and generation systems). • The speed of lexical recognition and structure projection is compatible with quite early context effects via production. • Experimental demonstrations of conceptual and discourse projection are compatible with a production base. • Production performance profiles can be used to predict comprehension performance (agreement error and lexical biases used in comprehension tests are usually indexed by production bias). • The electrophysiological patterns for comprehension have a plausible fit to a production based processing component. • Language disorders show interesting evidence for the articulation and codependence of comprehension and production systems. • Language acquisition profiles apparently differ for production and comprehension.
Lexically Driven Encoding Lexically driven encoding in the modeling of production processes is quite in keeping with a number of current “lexically driven” approaches to parsing. In thinking about this, it is well to keep in mind that “lexically driven parsing” comes in several varieties. To embrace such a view is not thereby to embrace a view of parsing as interactive in the relevant sense when one is speaking of comprehension any more than it requires such
50
Merrill Garrett
for production. One may pursue a parser in this mode with as much or as little reliance as one chooses on the lexical semantic, conceptual, or other information associated with a given lexical target. And, one may, of course, exploit in similarly varying ways the linkage between lexically driven parsing and “lexical preferences” as a principle of choice dictating parsing options—as contrasted with general principles of configural economy or structurally driven choice (minimal attachment, late closure, etc.). This latter move is sometimes taken as the opening wedge in a general frequency of usage argument that leads one down the path to claims that processing is fundamentally a matter of the statistical likelihoods of co-occurrence relations between successive words and groups of words. Whether one takes these kinds of statistical properties of words to be an overlay on a syntactically specific parsing architecture or as a central determinant of processing in a system that does not single out grammatical category as a special type is an entirely open matter. The use of probability in parsing can be implemented in ways that preserves the separability of information types or ways that do not. In this same general vein, one may remark on the place of argument structures associated with lexical elements—the subcategorization frames and thematic structures discussed in the section entitled Some Experimental Examples and Perspectives from Parsing Studies. A very productive line of investigation of the role of semantic variables in sentence comprehension has been the examination of ways in which the thematic roles associated with the structure of verbs interact with subcategorization biases in the structure of verbs. In language production modeling, the lemma driven systems that have been proposed incorporate argument structures (e.g., see Levelt, 1989, chapter 2) in ways quite in keeping with the appeal to argument structures that one finds in parsing studies. From a related but different perspective, one might cite a discussion by Kim, Srinivas, and Trueswell (1998), though in so doing, I’m turning it a bit on its head since it is meant to advance the case for constraint based parsing—which it ably does. That discussion is focused on development of parsing systems based on lexicalized tree adjoining grammar (LTAG: Joshi & Schabes, 1997), and it lends itself to my current point simply because the elementary trees in the LTAG representations encode the syntactic information in ways reminiscent of the properties of lemmas and their invocation. Indeed, these observations simply emphasize the potential for overlap in some machine parsing proposals and systems for generation. The similarity of the compositional operations required of a parser and those required for the lexically driven sentence encoding model has not escaped attention. Computational efforts to develop systems that will support both analysis and generation are thus of particular interest. So, for example,
2.
The Architecture of Language Processing Systems
51
work on production by Stone and Doran (1997) with tree adjoining grammars is illustrative vis-à-vis the general classes of representation alluded to by Kim et al. in the parsing context above. And, work by Vosse and Kempen (1999) and by Kempen (1999) likewise lays out ways to exploit common devices for parsing and production. (I choose these as recent interesting references that bear on the point pursued here, but several useful discussions of this general issue are to be found in the computational literature. It is a matter of long-standing interest.) Note, however, that though the operations of phrasal composition may be very similar, the controlling inputs to the processors may be treated in distinct ways, particularly as one thinks in terms of the nature of the human production and comprehension systems rather than machine systems. In the case of normal production exercise, the lexical nomination is via message level constraint. In the case of normal first stage comprehension, it is via the outputs of the lexical recognition system. If the parser is to be driven primarily by lexical recognition outputs, without immediate appeal to higher order constraint, then it is here in particular that one can see the useful force of a production architecture that separates the semantic control process from direct involvement in the phrase building operations. That move in the production domain enables phrasal composition systems that can operate with the more “limited” inputs that a recognition parser might deal in. This connects to two additional observations. First, one may remark on the fit this provides to the empirical warrant of the production data discussed in the previous section. For example, in speech error data one finds indication that phrase building errors do not arise out of the kind of semantic similarity that, for example, gives rise to meaning based lexical selection errors. So, if one examines word exchange errors (e.g., “Cats are a good place for barns”; “This idea had the guy . . . ”), one finds the meaning similarity of the interchanged elements is not comparable to that holding between the target and intrusion in a meaning based word substitution error (e.g., “He shot him through with a sword”/ arrow; “You can just go jump in the river!”/lake). This and other evidence noted in the previous section invites a view of semantic control of phrasal integration for production as indirect in ways that may facilitate the intersection with comprehension systems. The other observation turns on whether we should interpret this design feature as enabling the phrase building system of the producer to be directly engaged by lexical inputs either from conceptual/semantic or form driven systems. This way of talking would suggest that it is literally the same machinery that computes the phrasal structures for both production and comprehension. There are at least two reasons to be cautious about accepting that notion. First, it would defeat the purpose of an error
52
Merrill Garrett
checking system. If a benefit to be got from the interaction of production and comprehension is their mutual error monitoring functions, then independent sources for the compared signals is rather a necessity. So, the systems of construction might be identical—but there better be two of them, and with different drivers. Second, there is some evidence from language pathologies indicating the selective preservation of comprehension when production is compromised and vice versa. The detail of structural compromise in each of these complementary cases is, however, not sufficient for a fully confident application of that dissociation to the present question. And, it should be noted that such a dissociation is both a virtue (on the error checking mode) and a potential problem for the proposal I’m advancing since it requires that we look for dependencies between the use of production resources and success in comprehension and vice versa. I will return to this point. The speed of lexical recognition and structural projection is compatible with support for production based processing as a contributor to comprehension. The proposal under evaluation requires early and accurate identification of lexical targets and the recovery of their structural burden within a window that is plausibly supportive of recognition and comprehension performance time constants. These processes are the fodder for the production system as well as the comprehension system and must work in a timely way if any of this is to succeed. We can begin by noting a phenomenon that might be taken as a kind of approximate “existence proof” for such capacity—though it has not been recruited quite to the purposes I intend here. Here I have in mind the “close shadowing” performances documented by Marslen-Wilson (1973), in which some few (perhaps 1 in 5) persons show the ability to sustain repetition latencies for normal connected prose of 250–300 ms with quite good accuracy. What levels of representation are recovered may be not entirely clear, but the reported work certainly showed some clear engagement of the syntactic and semantic force of the shadowed materials. Other examples, of many that might be cited, will remind readers of well-known related evidence. Long since, Marslen-Wilson and Tyler (1987, for review) have argued for a very rapid process in the recognition of lexical targets in the spoken language domain and for the assimilation of the information associated with lexical entries to the comprehension process. How fast? Very. Work by Zwitserlood (1989) is instructive. Using a cross-modal priming paradigm, she provided evidence for the multiple activation of words compatible with the initial phonetic segments of test words and an effective linking of lexical targets to sentence environments at a moment contemporaneous with the occurrence of the terminal phonetic segments of the test words. A related phenomenon is lexical ambiguity resolution. In a variety
2.
The Architecture of Language Processing Systems
53
of experimental studies, multiple activation and rapid contextually appropriate resolution is observed (Swinney, Zurif, Prather, & Love, 1996, for review). Recent ERP results reported by Van Petten et al. (1999) show N400 responses to sentence incongruous words arising prior to the point in the acoustic input at which the word becomes phonetically identifiable. So, lexical information on which encoding as well as decoding could rely is indeed rapidly accessible. Particularly interesting in this regard is an experiment by Trueswell and Kim (1998) using a “fast priming” technique that indicates the subcategorization features of a verb are rather immediately available for the constraint of syntactic analysis. In this procedure, prime words are presented at rates too fast for conscious report. The prime words contrasted those with a direct object bias (e.g., “obtain”) and those with a sentence complement bias (e.g. “realize”). Sentences appeared with and without an overt complementizer signal [the man accepted (that) . . .], followed by an implausible target NP (“the fire”), followed by the words of the verb phrase of the complement sentence. The man accepted (that) the fire could not have started in the attic. Results showed interference from primes with structures incompatible with the required sentence analysis (i.g., direct object structure primes induced a slow down at the NP and at the first element of the verb phrase), and possible facilitation for compatible primes. In production terms, access to the representation for, for example, “obtain,” if implemented at the lemma level as a procedural call, would activate a package with direct object subcategorization and patient thematic role. This package is compatible with the verb “accept” but in fact would lead to the wrong analysis for the target sentence. That analysis is, on the Trueswell and Kim (1998) results, implemented and then abandoned only at the locus of the verb phrase in spite of the information contained in the immediately preceding NP which constitutes an implausible filler for the theme role in the direct object structure for “accept”. Interpreting the pattern of effects in light of a production application is not difficult, but it is not decisive either. But, at very least, it suggests that the projection of a lexically based syntactic frame as a constraint on analysis of immediately following input is very rapid and may proceed before thematic appropriateness of the NP can be evaluated (see also McElree & Griffith, 1995, 1998). Context, Discourse, and Plausibility Effects The kind of information available and the time needed to extract it are sharply in focus throughout. The just prior discussion focused on lexical recovery questions. Here we look at more global sorts of information for
54
Merrill Garrett
which impact on comprehension is reported. Ideally, the production system will have some sort of scenario to work from—and isolated sentences with no context are not the best candidates for the operation of production based constraint mechanisms. It is, however, a rather striking fact that a major methodological feature of experimentation designed to evaluate the contribution of various interpretive and discourse factors on comprehension is the use of more elaborated contextual presentations for target sentences. So, in variations of, for example, the Marslen-Wilson and Tyler (1987) and Crain and Steedman (1985) studies adverted to earlier (and many similar not cited), a consistent feature of experimentation has been the use of minidiscourse settings as context for the test sentences. Such a circumstance is necessary to the manipulation of discourse factors and is, therefore, also quite plausible for the application of production based constraints on the analysis of target structures. Sentences in isolation are less readily accommodated by the production hypothesis except insofar as the relevant structure is immediately projectable from lexical entries (e.g., in the manner discussed earlier). From this, one may derive the prediction that well established discourse based influences on parsing may be among the fastest acting such sources of constraint, and in particular, faster than inferentially driven constraints based on the contextually derived meaning of the lexical content of sentences. The eyepointing studies noted in the earlier section in which I discussed experimental examples from parsing studies (e.g., Tanenhaus et al., 1995) may likewise be plausibly construed from this production perspective. Indeed, in that case, perhaps an even stronger claim might be worth consideration, for we might suppose that the eye movement data arise in significant measure from the primary production system, situated as it is in a highly structured environment and dealing in commentary of a structurally regular range. So, in that task, the visual input is prime production fodder and the mission to match auditory input and potential description high on the priority queue. However one might sort out such a possibility, the rapid assimilation of a visual input to language performance may fall more naturally in the performance domain of production than of comprehension mechanisms. Work by Forster and colleagues (Forster, 1979) established a prima facie case for an impact of what he dubbed “plausibility” factors on comprehension in a variety of processing tasks—acceptability, RSVP, and same–difference matching. Those materials were not presented in context, but Forster characterized the nature of the plausibility variation as reflecting the relative ease with which a scenario apposite to a test sentence could be constructed. Consider examples like the following:
2.
The Architecture of Language Processing Systems
55
The pretty minister cried while cannons roared. Sweating lightly, he walked under the snake. The effect (controlled for a variety of nuisance factors) is robust under conditions of repetition and instruction. That is, it does not appear to be the case that the consciously available strategic resources of listeners or readers are particularly useful in resolving the computational difficulty posed by sentences like those in the examples. Forster’s work generally did not reveal an interaction between syntactic complexity and plausibility variation (Forster, 1979; but see also Forster, 1987). The import of this is fairly clear for the position I’m advancing. If the locus of plausibility effects were in primary parsing operations, one would be forced to affirm what many experimental outcomes do not support, namely, that very general knowledge based constraints are involved in the immediate projection of phrasal geometry during sentence comprehension. If, on the other hand, one attributes the effects quite literally to the ease of generation of a possible construal of the relations implied by the sentence, the production based description is a natural one. This is an area in which some empirical leverage on the hypothesis might be profitably looked for. Production Profiles Predict Comprehension Performance It is worth noting that in many experimental Comprehension investigations that look for effects of frequency of structural configuration or other collocational factors, test material has been constructed using procedures that use sentence completion tasks as a basis for estimation of preferences—here I cite no references since the practice is so commonplace. In other instances, the appeal is to statistical measures of relative frequency of occurrence in large corpora. In both cases, the conflation of production and comprehension proclivities is clear. Thus, there is, other things being equal, no necessary problem in the application of a production based parsing filter to existing accounts of putatively comprehension based performance reflecting preference data. There is an interesting caveat, however. It should be borne in mind that in the case of production, the preferences for one or another structural path will have the full conceptual resources associated with lexical content, but a strictly bottom-up driven parser that operates off the phonological form and grammatical category information may not—to the extent that such cooccurences in generation may reflect conceptual contingencies not available to the recognition parser. That is, the frequency effects may be different in the two domains if the picture I am suggesting
56
Merrill Garrett
is correct: effective recognition frequencies might collapse across factors that affect production. The use of production data to test recognition parsing in paradigms that do not segregate production and recognition influences on task performance may this obscure a contrast of interest. Questions about which class of statistical constraints to apply in parsing systems have been a recent topic of interest (e.g., Mitchell, 1994; Gibson & Tunstall, 1999; Kim & Trueswell, 1998) and there may be scope for pursuit of a production/comprehension based analysis couched in terms of availability of the different types of information in the two systems. Another, and more detailed example of the convergence of performance profiles in the two domains is provided by work based on studies of agreement error patterns in speaking. Work by Bock and Miller (1991) and several followup studies demonstrated some stable patterns in the breakdown of mechanisms for number agreement in English (since extended with some interesting modifications to several other languages; see Bock, Nicol, & Cutting, 1999; Vigliocco & Zilli, 1999, for more recent treatments). For our purposes, the significant facts are these: number mismatch between a head noun and a local noun, for example, as italicized in the example The baby on the blankets . . . (is/*are playing with a puppy.) enhances the likelihood of number error in a sentence completion task compared to a control with no mismatch. Further, the error rate is substantially greater when the mismatch is one in which the local “distractor” is plural and the head singular—the reverse mismatch is often not significantly different from baseline—though this varies a bit. The same effect can be observed in a “recognition” task. Nicol, Forster, and Veres (1997) used a timed reading task to evaluate the same contrasts. A particularly significant aspect of their approach is that they did not use test sentences with errors in them. Rather, they used fully grammatical sentences, but with NPs located in ways that mirrored the position and structural roles of interfering NPs in the production task. They found elevated reading times for the sentences with interfering NPs compared to those without NPs mismatching number; moreover, these effects were limited to mismatches involving a plural interfering NP—just as the case for production performance. Evidently, some processing difficulty in the coordination of number marking arises in production and in comprehension that is of rather striking similarity. Profile of Electrophysiological Responses to Sentence Processing The profile of electrophysiological responses to sentence processing presents another area in which one might speculate about the production hypothesis. The ERP components focused on in the section in which I discuss
2.
The Architecture of Language Processing Systems
57
electrophysiological evidence may be fitted to this picture, though at the price of taking seriously a feature of the profiles that I have already noted is at least questionable, namely, the observed time frames for the various syntactic and semantic responses so indexed. The various patterns (E/LAN, P600, N400) do not, we noted, all relate to syntax and interpretation in the same way, and we can ask whether these differences in structural sensitivity are commensurate with assumptions one might make about the drivers for primary syntactic processing and/or production based filtering or adjustment. For good or for ill, the obvious move here is to examine the differences between E/LAN and P600, as both of these have been linked to syntactic processing as primary factors in their elicitation. If the P600 is, as has been suggested by several observers, the reflection of processing that is aimed at “fixing” the deviant structure, that is, it is at least in part the “patchup/backup” system that we are seeing, and if the production system is taken to be the source of the structures used for that function, then we are commited to the view that the P600 is, in part at least, a production reflex. The further observation that the E/LAN is focused on “language specific” areas whereas P600 reflects a much wider distribution of activity fits this general line of speculation, as does the further observation that P600 is responsive to strategic control and to certain semantic interactions. On the production construal, P600 should be responsive to semantic constraint in a way that the ELAN and LAN are not—if the former reflects production system control and the latter does not. And, indeed, that appears to be supportable on current observation. From this, and from observations of how the production system responds to semantic control, one might essay the “prediction” that there is structure in the P600, that is to say, differences in structural sensitivity reflected in its distribution or time course. In particular, one might expect to distinguish responses that are primarily syntactic from those that reflect semantic constraint. Recall from the discussion of the production system that there is a stage of the production process for which it appears that semantic influence is only indirectly manifest and configurational syntax is a major factor. I wish that I could claim to have had such a thought before knowing that such claims have begun to appear, but, alas, I cannot. So call it a “hybrid postdiction.” In fact, we do not know how this expectation might fare. Hagoort et al. (1999) do note some indication of a difference in the P600 response to outright syntactic violation (a more posterior distribution) and the recovery from misanalysis traceable to syntactic bias (more equal or more frontal distribution), but it is not obvious exactly how, if at all, that might relate to the production structure noted. What is clear, is that one may bias parsing preferences with semantic constraint and that this is measurable with P600 (e.g., Hagoort, Brown, Vonk, & Hocks, forthcoming).
58
Merrill Garrett
It would be worth knowing with some precision how similar constraint affects E/LAN (if at all) and whether syntactic bias and semantic bias can be distinguished in the topography of the P600. Language Disorders One of the parade cases of dissociation of language production and language comprehension comes in the study of agrammatism, seen (typically) in Broca’s aphasia. Clinically, agrammatism is an expressive disorder and is often remarked in conjunction with what seemed to be clinically fairly well preserved comprehension capacity. In an important paper, Caramazza and Zurif (1976) reminded people that this was indeed a matter of relative success in test circumstances not precisely focused on the detail of syntactic factors supporting comprehension, and then provided several experimental demonstrations that such patients did in fact suffer from apparently syntactic limitations (see Caplan, 1987, for review). This was variously interpreted as an indication of a compromise of grammatical knowledge, specifically the syntactic component of the processing/ knowledge system. Preserved lexical semantic capacity, and inference based on such, was offered as the basis for the apparent preservation of comprehension performance in the face of syntactic impairment. Much more has been written about this aspect of agrammatism and its reconstruction (see, e.g., papers in Fromkin, 1995, for wide ranging discussions) and it is a rich area of study. For present purposes, the first important feature lies in the observation of preserved inferential capacity vis-à-vis the lexical semantic base and a significant syntactic impairment that in many patients manifests itself in both structurally compromised production and syntactically limited comprehension. Such a confluence of limitations/capacities is unproblematic for the production hypothesis under examination here. But, the matter is, of course, by no means so simple. There are a variety of complications, only two of which I will comment on here. First, some agrammatic producer/comprehenders are capable of syntactic processing that is sufficient to support substantial success in well-formedness judgment tasks (Linebarger, Schwartz, & Saffran, 1983); that is, they have a “paradoxical syntactic capacity” if one takes their comprehension problems as syntactically based. Second, there are attested cases of patients who display agrammatic speech but do not display a comprehension limitation; that is, they are “paradoxical comprehenders” if one takes their production as the index of their potential. A brief look at each of these circumstances will have to suffice here.
2.
The Architecture of Language Processing Systems
59
The ability to make judgments that successfully distinguish syntactically well-formed from ill-formed sequences has been observed in agrammatics who fail diagnostic comprehension tests (e.g., a limited understanding of passives or center branching relatives) and this implies what I wish to affirm: the input parser may deliver a wff product that is disconnected from reliable semantic interpretation. But, the failure of such patients could take several underlying forms given the framework being considered here. If the mechanism for integrating/comparing a recognition based representation with a production based representation is impaired, an erroneous semantically driven production target may be accepted even if the input representation is accurate (i.e., an analysis driven by lexical semantic constraints will have no “recognition brakes”). That circumstance may or may not be combined with a failure in the production machinery itself that uses semantic constraint to engage phrasal construction mechanisms. This is a mechanism that, once engaged, may operate without immediate semantic control (viz., as the production evidence reviewed in the previous section suggests). The picture invited is of a breakdown in the mechanisms that insure a link between syntax and interpretation, not of a breakdown in the mechanisms of phrasal construction—which are by the assumptions of the production model, capable of some independent (of semantic direction) operation and that operation might be exercised in either comprehension system, production system, or both. In brief, there is not much help here. The main message is that the association of meaning with syntax is potentially loose at both ends, and that a syntactic representation could conceivably be available from either source that might mediate successful well-formedness judgments. We do have some evidence that the application of interpretive constraints in ambiguity resolution (Swinney, Zurif, & Nicol, 1989) and in binding (Zurif, Swinney, Prather, Solomon, & Bushnell, 1993; Swiney et al., 1996) is compromised in agrammatics. See also Nicol and Love (Ch. 5, this volume) for an analysis of certain comprehension failures in agrammatism that are traceable to production limitations. That is, we can assert that the loss of productive capacity is sometimes linked with the loss of timely access to integration of interpretive constraints on sentence analysis (see Swinney and Zurif, 1995, for one statement of this perspective). And we have reason to believe that the semantic capacities are intact, at least based on success in tasks that depend on lexical semantic inference. The production hypothesis is compatible with some dissociation of production and comprehension capacities, but is committed to the view that compromise of basic production capacity should be accompanied by reduced ability to use interpretive constraints to correctly constrain syntactic
60
Merrill Garrett
analysis. This idea is most directly challenged by those reported agrammatics (see, e.g., Miceli, Mazzucchi, Menn, & Goodglass, 1983; Kolk, Van Grunsven, & Keyser, 1985) who display no comprehension deficit, that is, they show evidence of production compromise without an apparent concomitant loss of comprehension function. But, the kinds of information about compromise in those patients that would bear on the production hypothesis is not readily available. Production driven syntax is assumed to play a significant role in the normal functions, but purely form driven syntactic representations, if constructed by the “data driven” parser are nevertheless interpretable. Thus, it would be instructive to determine whether the successful performance of such patients is responsive to the same range of contextual constraints and to the same course of processing as normal listeners. In this context, we may consider an observation from ERP experimentation with agrammatic patients by Brown et al (1999). These patients were all Broca’s aphasics as indexed by the Aachen Aphasia test and all were characterized by effortful production and ommisions of morphosyntactic elements in their output—so, broadly speaking, all were agrammatic producers. These were divided into a group of five patients with chance performance on syntactically demanding comprehension tests and a group of five who, though somewhat impaired, scored above chance on the test, thus indicating significant syntactically based comprehension capacity. All 10 patients displayed a normal semantic integration effect (N400) to semantically implausible targets in test sentences, but the five patients with poor comprehension performance showed no P600 ERP signature to syntactically deviant sentences; by contrast the patients with above chance comprehension scores did show a reduced but unmistakable P600 response to syntactic deviance (agreement violations and word order violations). Interestingly, the poor comprehenders who showed no P600 did show an N400 for the word order violations—that is, absent the P600 response, they nevertheless showed sensitivity to the semantic oddity produced by the word inversion. We cannot further interpret these interesting findings for the issues here addressed beyond saying they give additional evidence that significant comprehension capacity, is, as the earlier case studies indicated, observable in presence of impaired production, and, additionally, based on the ERP evidence, that this capacity includes the processes that drive the late syntactic component. No E/LAN results were observed in this experiment in either normal controls or patients. This kind of study, however, represents the kind of inquiry that will be needed in order to evaluate the import of aphasic compromise for the production hypothesis. So, for example, would such patients show the same delay in on line ambiguity processing or in binding tests for reactivation of potential antecedents
2.
The Architecture of Language Processing Systems
61
that the larger class of agrammatics displays based on the evidence cited above? And can the performance of the patient groups be distinguished in terms of responsiveness to early and late syntactic components? Quite apart from these matters, it would be important to discover in some detail whether performance compromise in the production domain is the same for agrammatics with relatively preserved comprehension as for those whose syntactically mediated comprehension is deficient. “Agrammatism” is a term with considerable latitude of application and there is significant variation in the character of such compromise across patients. Add to this the certain fact that we do not know just how any given agrammatic output symptom relates to an underlying production limitation, and it should be obvious that the existence of the “paradoxical comprehenders” is the call to a research program rather than a prima facie disconfirmation of the role of production in comprehension processing. Acquisition Profiles It’s pretty clear on the basis of contemporary studies of language acquisition that kids understand a lot that is syntactically quite sophisticated before they talk much and before their own spontaneous speech displays a similar degree of syntactic sophistication (see McKee, MacDaniel, & Snedeker, 1998, and references therein). Indeed, when they do talk at early stages, many investigators have taken children’s utterances to be quite aberrant re adult syntactic targets. Such a condition, taken at face value poses something like the problem of the agrammatic aphasics noted in the preceding section: they are individuals (children or aphasic) whose production output is “impoverished” or “deviant,” but whose understanding seems to draw on significantly greater resources of the syntax. If production plays the role I have ascribed to it in the proposal for an integrated production and comprehension system, how can such a circumstance exist? Several observations and ideas suggest themselves. First, with respect to children, one may question the extent to which their underlying production system is in fact lacking in resource or discriminative capacity. We can note (McKee & Emiliani, 1992) that children’s productions are more complex than superficial appearance would suggest, and that elicited production tasks can bring to light evidence of adult grammatical models driving the child’s production at the stage when only two or three words are being generated. It may be quite feasible to defend the view that the observed production profile is reduced in range not because the child’s ability to bring conceptual content to bear on linguistic representation is limited, and in particular not because of limited access to abstract syntactic devices of the language, but rather because of
62
Merrill Garrett
fairly late stage processes in the encoding cycle. Moreover, there is a complementary point: very young children may not appreciate or control discourse structure and some of the associated implicational devices required to fully exploit it. If so, the apparent dissociation of production and comprehension capacity may have roots other than in the components needed to exercise the production hypothesis I have been pursuing. Overall, it does seem clear that examination of relations between children’s developing production and comprehension capacities has the potential to contribute significantly to an evaluation of dependencies between the two systems and their bearing on the production/comprehension hypothesis.
SUMMARY REMARKS This is in several respects a potentially frustrating proposal for those who like their argument space to draw nice sharp distinctions—as most of us do. And from time to time I think of myself that way, so the production hypothesis makes me a little nervous. For, one might wish to argue that all this conversation about production is merely a funny way to talk about comprehension processes since the assimilation of lexical and phrasal inputs into discourse formats in order to drive production might just be the very process that Marslen-Wilson and Tyler (1987) wished to invoke when they averred that mapping is “direct” from speech input to mental models, and encompassing no intervening representation of syntactic form. Syntax on that view is just another information class embedded in the mapping procedure. That embedding might be done in various ways. How human heads do it remains to be determined. For the hypothesis I have in mind, a lot will turn on whether and how it works out that a partial analysis of the input delivered primarily by data driven processes can be integrated with a preexisting discourse framework, and thus give the production machine a suitably constrained input so as to deliver a useful matching and control signal for integration with the later products of the syntax recognition device. How detailed and accurate can that data driven analysis be if it has no semantic crutches? Accurate enough that the production machine can achieve the necessary convergence with the recognition system within a useful time frame given richer lexical information and an existing discourse frame? I don’t know, but I think it’s a pretty good bet—certainly one worth close inspection. If such is the case, the picture of the recognition/comprehension component would remain dominantly form driven from beginning to “end” (viz., at match/merge of the top-down production structure with the final
2.
The Architecture of Language Processing Systems
63
bottom-up parsing product). Of course, here’s the other end of the possible spectrum: it may be that the devil in these details could turn out to be something like a constraint satisfaction machine of the sort that Bever et al. (1998) adopted as a front end to the ABS system in their proposal. If so, the differences between that proposal and my own would diminish. But, I think there would remain some useful leverage in the production hypothesis even in that scenario; I note some that obtain on either outcome. It is well to remember another point in this context, namely, that the perspective I’m pursing holds that the recognition based parsing system and the production based phrase building systems are complementary— and the recognition device in particular must be independently capable of achieving richly elaborated structure if it is to serve in its turn as an error correction device and source of constraint on the production system. The production hypothesis offers distinct drivers for the determination of sentence form under comparable time constraints that fits naturally into a scheme of error control requirements on each of the major performance systems. Beyond that, we do know that the production system must be designed to start from conceptual inputs that will drive linguistic outputs. And it looks like a part of the way the human production system works is to put a little space in between the semantic and syntactic engines in the process of doing that job. This is a point worth dwelling on for a moment. We expect production processes to have conceptual/ semantic drivers. That is the nature of the input to the system. Nevertheless, the production data indicates that this is compatible with a significant degree of separation in the system for semantic, syntactic, and phonological processing—the system does not manifest itself as a set of “mixed representations.” Why aren’t we just a little amazed by this? If there is computational advantage to mixed mode processing, why not here if anywhere? It’s an interesting thought to ponder: if the direct use of semantic guidance for phrasal construction is essential to real-time success in the compositional operations of language processing, one might reasonably expect to see this strongly manifest in the organization of the phrasal representations computed for language production. The relevant information is indisputably activated therein. But, instead, one sees considerable evidence indicating components of the production system that involve grammatical category and phonological representation but that do not display a similarly striking dependence on the semantic force of the lexical elements (see references in the section entitled Language Production Study). Various reasons might be offered for such a condition. One might even speculate that a motivation for this condition in production arises out the utility of interactions with a system (recognition) that does its processing work on a more limited information base.
64
Merrill Garrett
The incorporation of the range of interpretive constraints typically appealed to in strongly interactive treatments of parsing for comprehension can be accomplished without abandoning architectures with significant modularity. Production architectures are demonstration cases of that possibility. But, given that assumption, the question I posed for an integrated constraint satisfaction mode of comprehension is fair to pose here for the production proposal. Namely, why on the production view is the effect of interpretation variable across experiments? I have suggested a couple of reasons in preceding remarks vis-à-vis differences in the availability of the necessary lexical and contextual information as it arises in a given communicative situation or from the interaction of production and comprehension processes. The production architecture account must appeal to variations in informational availability just as all interactive accounts must do. The principal added ingredient is provided by the need for production structure itself. The account is explicit about assigning the source of conceptual, semantic, and discourse factors to a language processing system for which interpretive factors are expected—indeed, necessary—components. A variety of factors may be brought to bear: expectations and intentions that can control production decision should be present and determining for comprehension; discourse structure when present should be determining, and may well be so with greater immediacy than virtually any other constraint type; whatever aspects of lexically coded information are not recoverable from first pass recognition will be effective, but only within the time constraints required for application of the constraints via production mechanisms. For this latter point, the relation between subcategorization and thematic role information will be instructive. But, the lexical structure issues will be irritatingly malleable until such time as arguments about the relative availability of the different classes of information about lexical elements are better fleshed out. There are further arguments to be made along these lines, I think—some of them certainly already underway in the context of the picture–word interference task alluded to in the section entitled Language Production Study where some interesting efforts to link word recognition and word production processes seem the best way to deal with conflicting experimental outcomes in that paradigm (see, e.g., Roelofs, forthcoming). The treatment of the overall comprehension process as involving an “interaction” between production system syntax and data driven recognition devices has some appeal in terms both of the task of linking processors to grammars and the general interest of identifying and motivating commonalities in the two major performance modes. The virtues of this move as a way to rationalize incompatible aspects of the experimental landscape are threefold: it simplifies the comprehension
2.
The Architecture of Language Processing Systems
65
scheme by eliminating the need to try to enfold a layer of semantically and pragmatically driven mechanisms; it advances an interesting set of claims about relations between the two major performance systems; and it makes it easier to preserve a picture of recognition processing that corresponds to the informational structure of grammars. In short, because the architecture of the production system is arguably one in which the conceptual, discourse, and semantic factors are also broadly distinguishable from the mechanisms of phrasal construction and phonological interpretation, the overall picture of processing is one in which one may more readily see both a general fit of production and comprehension processes to each other and of each of these to the character of grammatical systems.
ACKNOWLEDGMENTS This work was supported in part by National Multipurpose Research and Training Center Grant DC-01409 from the National Institute on Deafness and Other Communication Disorders. I wish also to express appreciation for comments from audiences at UCLA Linguistics, at the University of Pennsylvania I.R.C.S., and at a colloquium, MPI for Pyscholinguistics in Nijmegen, all of which helped me in thinking about the matters presented here.
REFERENCES Altman, G., & Steedman, M. (1988). Interaction with context during human sentence processing. Cognition, 30, 191–238. Bever, T. G. (1970). The cognitive basis of linguistics structures. In J. R. Hayes (Ed.), Cognition and the development of language. New York: Wiley. Bever, T. G., Sanz, M., & Townsend, D. (1998). The emperor’s psycholinguistics. Journal of Psycholinguistic Research. Bierwisch, M. (1982). Linguistics and language error. In A. Cutler (Ed.), Slips of the tongue (pp. 29–72). Amsterdam: Mouton. Bock, J. K. (1986). Syntactic persistence in language production. Cognitive Psychology, 18, 355–387. Bock, J. K. (1990). Structure in language: Creating form in talk. American Psychologist, 45, 1221–1236. Bock, J. K., & Eberhard, E. (1993). Meaning, sound, and syntax in English number agreement. Language and Cognitive Processes, 8, 57–99. Bock, J. K., & Levelt, W. J. M. (1994). Language production: Grammatical encoding. In M. Gernsbacher (Ed.), Handbook of psycholinguistics. San Diego: Academic Press. Bock J. K., & Loebell, H. (1990). Framing sentences. Cognition, 35, 1–39. Bock, J. K., Loebell, H., & Morey, R. (1992). From conceptual roles to structural relations: Bridging the syntactic cleft. Psychological Review, 99, 150–171.
66
Merrill Garrett
Bock, J. K., & Miller, C.A. (1991) Broken agreement. Cognitive Psychology, 23, 45–93. Bock, J. K., Nicol, J., & Cutting, J. (1999). The ties that bind: Creating number agreement in speech. Journal of Memory and Language, 40, 330–346. Bock, J. K., & Warren, R. (1985). Conceptual accessibility and syntactic structure in sentence formulations. Cognition, 21, 47–67. Brown, C., Wassenaar, M., and Hagoort, P. (1999). Functional plasticity in aphasic patients with agrammatic comprehension. Manuscript, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands. Caramazza, A., and Zurif, E. (1976). Dissociation of algorithimic and associative processes in language comprehension: Evidence form aphasia. Brain and Language, 3, 552–582. Caplan, D. (1987). Neurolinguistics and linguistic aphasiology: An introduction. Cambridge: Cambridge University Press. Carlson, G., & Tanenhaus, M. (1988). Thematic roles and language comprehension. In W. Wilkens (Ed.), Thematic relations syntax, and semantic (vol. 21, pp. 263–300). New York: Academic Press. Crain, S., & Steedman, M. (1985). On not being led up the garden path: The use of context by the psychological processor. In D. Dowty, L. Kartutunen, and A. Zwicky (Eds), Natural Language Parsing, Cambridge University Press. Coulson, S., King, M., & Kutas, M. (1998). Expect the unexpected: Event related response to morphosyntactic violations. Language and Cognitive Processes, 13, 21–58. Crocker, M. (1996). Computational Psycholinguistics. Dordrecht: Kluwer. Dell, G., Schwartz, M., Martin, N., Saffran, E., & Gagnon, D. (1997). Lexical access in aphasic and nonaphasic speakers. Psychological Review, 104, 801–838. Fodor, J. A. (1983). The modularity of mind, Cambridge, MA: MIT Press. Fodor, J. A., Bever, T. G., & Garrett, M. (1974). The psychology of language: An introduction to psycholinguistics and generative transformational grammar. New York: McGraw-Hill. Forster, K. I. (1979). Levels of processing and the structure of the language processor. In W. Cooper and E. C. T. Walker (Eds.), Sentence processing. Englewood, NJ: Erlbaum. Forster, K. I. (1987). Binding, plausibility, and modularity. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language understanding. Cambridge, MA: MIT Press. Frazier, L. (1987). Theories of sentence processing. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language processing. Cambridge, MA: MIT Press. Frazier, L. (1999). On sentence interpretation. Dortdrecht: Kluwer. Frazier, L., & Clifton, C. (1996). Construal. Cambridge, MA: MIT Press. Friederici, A. (1998a). The neurobiology of language comprehension. In A. Friederici (Ed.), Language Comprehension: A Biological Perspective, (pp. 263–301). Berlin/Heidelberg/New York: Springer. Friederici, A. (1998b). Diagnosis and reanalysis. In J. D. Fodor & F. Ferreira (Eds.), Reanalysis in sentence processing. (pp. 177–200). Dordrecht: Kluwer. Friederici, A., von Cramon, Y., & Kotz, S. (1999). fMRI studies of patients with frototemporal lesions. Manuscript, Max Planck Institute for Cognitive Neuroscience, Leipsiz, Germany. Friederici, A., Hahne, A., & von Cramon, Y. (1998). First pass vs second pass parsing processes in a Wernicke’s and a Broca’s aphasic: Electrophysiological evidence for a double dissociation. Brain and Language, 62, 311–341. Freiderici, A., Hahne, A., & Mecklinger, A. (1996). Temporal structure of syntactic parsing: Early and late Event Related Potential effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1219–1248. Fromkin, V. (1971). The non-anomalous nature of anlomalous utterances. Language, 47, 27–52. Fromkin, V. (Ed.) (1995). Special Volume: Linguistic Representational and Processing Analysis of Agrammatism. Brain and Language, 50.
2.
The Architecture of Language Processing Systems
67
Garrett, M. (1980). Levels of processing in sentence production. In B. Butterworth (Ed.), Language production: Vol 1. Speech and talk (pp. 177–220). London: Academic Press. Garrett, M. (1993a). Errors and their relevance for theories of language production. In G. Blanken, J. Dittmann, H. Grimm, J. Marshall, & C. Wallesch, (Eds.), Linguistic disorders and pathologies: An international handbook. Berlin: de Gruyter. Garrett, M. (1993b). The structure of language processing: Neurophsychological evidence. In M. Gazaniga (Ed.), Cognitive neuroscience, pp. 881–889. Cambridge, MA: MIT Press. Gibson, E., & Tunstall, S. (1999). The parser need not keep track of contingent frequencies: A lexical account of the English determiner/complementizer ambiguity. Manuscript Department of Brain and Cognitive Sciences, Massachuseets Institute of Technology. Poster: 12th annual CUNY Conference on Sentence Processing, March 18–20, New York: CUNY Graduate School. Gorrell, P. (1995). Syntax and parsing. Cambridge: Cambridge University Press. Hagoort, P., and Brown, C. (1994). Brain responses to lexical ambiguity resolution and parsing. In C. Clifton, L. Frazier, & K. Rayner (Eds.), Perspectives on Sentence Processing, (pp. 45–80). Erlbaum: Hillsdale, NJ. Hagoort, B., & Brown, C. (in press). Semantic and syntactic ERP effects of listening to speech compared to reading. Neuropsychologia. Hagoort, P., Brown, C., & Groothusen, J. (1993). The syntactic postive shift as an ERP measure of syntactic processing. Language and Cognitive Processes, 8, 439–484. Hagoort, P., Brown, C., & Osterhout, L. (in press), The neurocognition of syntactic processing. In C. Brown & P. Hagoort, (Eds.), The neurocognition of language. Oxford: Oxford University Press. Hagoort, P., and Brown, C., Vonk, W., & Hoeks. J. (forthcoming). Syntactic ambiguity effects in coordination structures; ERP evidence. Manuscript, Max Planck Institute for Psycholoinguistics, Nijmegen. Hahne, A., and Friederici, A. (in press). Language comprehension: The brain dissociates early automatic and late controlled processes. Journal of Cognitive Neuroscience. Jackendoff, R. (1989). Consciousness and the computational mind. Cambridge, MA: MIT Press. Joshi, A., and Schabes, Y. (1997). Tree adjoining grammars. In G. Rozenburg & A. Salomma (Eds.), Handook of formal languages and automata: Vol. 3, Beyond words. Berlin: SpringerVerlag. Kempen, G. (1999). Human grammatical coding. (forthcoming). Cambridge: Cambridge University Press. Kim, A., Srinivas, B., & Trueswell, J. (1998). The convergence of lexicalist perspectives in psycholinguistics and computational, linguistics. Kolk, H., Van Grunsven, J., & Keyser, A. (1985). On parallelism between production and comprehension in agrammatism. In M. L. Lean (Ed.), Agrammatism. New York: Academic Press. Kutas, M., & Hilyard, S. (1980). Reading between the lines: Event related potentials during sentence processing. Brain and Language, 11, 354–373. Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press. Levelt, W., Roelofs, A., and Meyer, A., (1999). A theory of lexical access in speech production. Behavioral and Brain Science, 22, 1–75. Linebarger, M., Schwartz, M., & Saffran, E. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13, 361–392. MacDonald, M., Pearlmutter, N., & Seidenberg, M. (1994). Lexical nature of syntactic ambiguity resolution. Psychological Review, 101, 676–703. Marslen-Wilson, W. (1973). Linguistic structure and speech shadowing at very short latencies. Nature (London) 244, 522–523.
68
Merrill Garrett
Marslen-Wilson, W., & Tyler, L. (1980). The temporal structure of spoken language understanding. Cognition, 8, 1–71. Marslen-Wilson, W., & Tyler, L. (1987). Against modularity. In J. Garfield (Ed.), Modularity in knowledge representation and natural-language understanding. (pp. 37–62). Cambridge, MA: MIT Press. McClelland, J., St. John, M., & Taraban, R. (1989). Sentence comprehension: A parallel distributed approach. Langauge and Cognitive Processes, 4, 287–335. McElree, B., & Griffith, T. (1995). Syntactic and semantic processing in sentence comprehension: Evidence for a temporal dissociation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 134–157. McElree, B., & Griffith, T. (1998). Structural and lexical constraints on filling gaps during sentence comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 432–460. McKee, C., McDaniel, D., & Snedeker, J. (1998). Relatives children say. Journal of Psycholinguistic Research 27, 573–596. McKee, D., & Emiliani, M. (1992). Il Clitico: Cé ma non si vede. Natural Language and Linguistic Theory, 10, 415–437. Miceli, G., Mazzucchi, A., Menn, L., & Goodglass, H. (1983). Contrasting cases of Italian agrammatic aphasia without comprehension disorder. Brain and Language, 10, 65–97. Mitchell, D. (1994). Sentence parsing. In M. Gernsbacher (Ed.), Handbook of Psycholinguistics. San Diego: Academic Press. Neville, H., Nicol, J., Barss, A., Forster, K., & Garrett, M. (1991). Syntactically based sentence processing classes: Evidence from event related potentials. Journal of Cognitive Neuroscience, 3, 151–165. Neville, H., Mills, D. & Lawson, D. (1992). Fractionating language: Different neural subsytems with different sensitive periods. Cerebral Cortex, 2, 244–258. Newmeyer, F. (1998). Language form and language function. Cambridge, MA: MIT Press. Nicol, J., and Love, T. (2000). Overarching agrammatism: When comprehension involves production. In Y. Grodzinsky, L. Shapiro, & D. Swinney (Eds.), Language and the Brain, pp. – . Academic Press: San Diego. Nicol, J., & Swinney, D. (1989). The role of structure in coreference assignment during sentence processing. Journal of Psycholinguistic Research: Special Issue on Sentence Processing, 18, 5–24. Nicol, J., Forster, K., & Veres, C. (1997). Subject-verb agreement processes in comprehension. Journal of Memory and Language, 36, 569–587. Osterhout, L., & Holcomb, P. (1992). Event related brain potentials elicited by syntactic anomaly. Journal of Memory and Language, 31, 785–806. Osterhout, L., Holcomb, P., & Swinney, D. (1994). Brain potentials elicited by garden-path sentences: Evidence of the application of verb information during parsing. Journal of Experimental Psychology: Learning, Memory and Cognition, 20, 786–803. Roelofs, A. (forthcoming). Speaking while hearing or seeing words. Unpublished manuscript, University of Exeter, England. Spivey, M., & Tannenhous, M. (1998). Syntactic ambiguity resolution in discourse: modeling the effects of referential context and lexical frequency. Journal of Experimental Psychology: Language, Memory and Cognition, 24, 1521–1543. Stone, M., & Doran, C. (1997). Sentence planning as description using tree-adjoining grammar. ACL, p. 198–205. Swinney, D., & Zurif, E. (1995). Syntactic processing in aphasia. In V. Fromkin (Ed.), Special Volume: Linguistic Representational and Processing Analyses of Agrammatism. Brain and Language, 50.
2.
The Architecture of Language Processing Systems
69
Swinney, D., Zurif, E., E Nicol, J. (1989). The effects of focal brain damage on sentence processing: An examination of the neural organization of a mental module. Journal of Cognitive Neuroscience, 1, 25–37. Swinney, D., Zurif, E., Prather, P., & Love, T. (1996) Neurological distributuion of processing operations underlying language comprehension. Journal of Cognitive Neuroscience, 8, 174–184. Tannenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (1995). Interpretation of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634. Trueswell, J., and Kim, A. (1998). How to prune a garden-path by nipping it in the bud: Fast priming of verb argument structure. Journal of Memory and Language, 39, 102–123. Trueswell, J., & Tannenhaus, M. (1994). Toward a lexicalist framework for constraint based syntactic ambiguity resolution. In C. Clifton, K. Rayner, & L. Frazier (eds.), Perspectives in sentence processing. Hillsdale, NJ: Erlbaum. Tyler, L, & Marslen-Wilson, W. (1977). The on-line effects of semantic context on syntactic processing. Journal of Verbal Learning and Verbal Behavior, 16, 683–692. Van Petten, C., Coulson, S. Rubin, S. Plante, E., & Parks, M. (1999). Timecourse of word identification and semantic integration in spoken language. Journal of Experimental Psychology: Language, Memory and Cognition 25, 394– 471. Van Turennout, M., Haggort, P., & Brown, C. (1998). Brain acitivity during speaking; From syntax to phonology in 40 msec. Science, 280, 572–574. Vigliocco, G., Antonini, T., & Garrett, M. (1997). Grammatical gender is on the tip of Italian tongues. Psychological Science, 8, 314–317. Vigliocco, G., and Zilli, M. (1999). Syntactic accuracy in sentence production: Gender agreement in Italian and Spanish. In T. Jacobsen, A. Friederici, & M. Garrett (Eds.), Special Issue: Journal of Psycholinguistic Research. Vosse, T., & Kempen, G. (1999). Syntactic stucture assembly in human parsing: A computational model based on competitive inhibition and a lexicalist grammar. Manuscript, Leiden Univeristy, The Netherlands. Zurif, E., Swinney, D., Prather, P., Solomon, J., & Bushnell, C. (1993). An on-line analysis of syntactic processing in Broca’s and Wernicke’s aphasia. Brain and Language, 45, 448-464. Zwitserlood, P., (1989). The locus of effects of sentential-semantic context in spoken word processing. Cognition, 32, 25–64.
This . Page Intentionally Left Blank
Part
The Comprehension– Production Interface
II
This . Page Intentionally Left Blank
Chapter
3
Overarching Agrammatism Yosef Grodzinsky
If there is a single neuropsychological term that is most closely associated with Edgar Zurif (in terms of both content and style), it is “overarching agrammatism.” Zurif was the first to document the comprehension deficit in Broca’s aphasia systematically (cf. Zurif, Caramazza, & Myerson, 1972; Zurif, Caramazza, Myerson, & Galvin, 1974; Zurif & Caramazza, 1976), and show that this deficit has a syntactic character. These pioneering studies launched a new era in the neuropsychology of language, as they shifted the focus of research both to syntax, and to comprehension—two areas that are intensely investigated today, with non-negligible success, one might add. Indeed, a significant body of valuable data, as well as a corresponding mass of theoretical work, was initiated by these early discoveries made by Zurif and colleagues. Conceiving of agrammatism as an “overarching” deficit was the next step. The discovery of syntactic deficits in Broca’s aphasia led to the (then unconventional) belief that the deficit in Broca’s aphasia was not linked just to speech production, but rather, was generalized, spanning all communicative activities or channels. Moreover, the new findings indicated that the correct descriptive vocabulary for the deficit was linguistic. Thus, if previous accounts of Broca’s aphasia viewed it as a channel-specific disturbance (cf. Geschwind, 1970; Goodglass & Kaplan, 1972; Goodglass, 1976), the emerging position was that this syndrome manifests as a severe loss of syntax that cuts across all linguistic activities. This claim has an important consequence: it suggests that the neural tissue implicated in the deficit—Broca’s area and its vicinity, were the neural substrate for human syntactic capacity. The underlying reasoning was not unsound: some had already believed that speech production in Broca’s aphasia was syntactically impaired; now a syntactic impairment was discovered in comprehension as well. It thus followed, apparently, that the right unit of analysis for neuropsychologists of language was linguistic. On this view
Language and the Brain
73
Copyright © 2000 by Academic Press. All rights of reproduction in any form reserved.
74
Yosef Grodzinsky
(argued for vigorously in Zurif, 1978, and Zurif, 1980), one must “redefine” the cerebral language centers, reject the activity-based account of the nineteenth century, according to which Broca’s area subtends speech production; instead, a linguistic account is proposed, in which Broca’s area is the home of syntactic resources, recruited for both production and comprehension. The linguistic deficiency in Broca’s aphasia was, from now on, overarching agrammatism (OAA). Overarching agrammatism was an innovative and interesting idea. Its appeal, moreover, was in its simplicity. New data from structured experiments that began coming in at the time (mostly from the work of Zurif and colleagues), indicated that the modality-based account was not entirely true, and that receptive aspects of language were also implicated in Broca’s aphasia. These data flew in the face of the traditional account, as they demonstrated that the functional deficits subsequent to damage to Broca’s area is not restricted to productive aspects of language. A modality-based theory of the language centers had to give way to another, which was a proposal to view language as neurally represented, rather than according to linguistic levels. Syntax, on this view, was anterior, whereas the semantics was taken to be in and around Wernicke’s area. A move toward “redefining” the cerebral language centers had begun (Zurif, 1980). Yet there were problems with the new OAA account. First, apparent counterexamples were soon presented. Certain cases were reported, in which patients disruptions did not span all modalities. There were studies in which Broca’s patients scored “poorly” on production tests, yet thrived when their receptive skills were examined (Miceli, Mazzucchi, Menn, & Goodglass, 1983; Kolk & van Grunsven, 1985). And though at least some of these were later discredited (Zurif, 1996), they seemed (at least initially) to indicate that the OAA could not cover all cases. Further, they served as a basis for those who later attacked the concept of syndrome–complex in neuropsychology, and argued that the patient category Broca’s aphasia is incoherent (e.g., Caramazza, 1986), partially because of these apparent inconsistencies in the data. Still, problems with OAA actually seemed to run deeper. To see that, we need to consider what it takes to be convinced of the validity of such an idea. Recall that by overarching agrammatism one meant a specific disruption in language processing, that did not just span all modalities; the pattern of selectivity in the disruption was expected to exhibit crossmodal uniformity. OAA, then, was taken to be roughly the following: (1) Overarching agrammatism (OAA) The language disruption in agrammatism (a) spans all modalities, and (b) the selective pattern of impairment and sparing is identical cross-modally.
3.
Overarching Agrammatism
75
Such a version of OAA underlies the psycholinguistic account of agrammatism of Bradley, Garrett, and Zurif (1980). They proposed a comprehension/production parallelism, in that the deficit to both modalities involved closed versus open class vocabulary items. Their account promoted a particular view of the mental lexicon, whose disruption manifests in OAA. Although this proposal was later abandoned for various reasons, it did meet the requirements posited by the OAA hypothesis: it purported to provide uniform cross-modal evidence regarding the selectivity of the aphasic deficit. Bradley, Garrett, and Zurif’s (1980) account was based on meager empirical evidence: all they had available were spontaneous speech data in English, and one set of results from reaction time (RT) experiments that they ran with normal and aphasic subjects. Further, the linguistic basis of their account was rather shaky. Linguistic analyses of agrammatism that were proposed later, taken in conjunction with the massive body of experimental data that has since accumulated, enable us to evaluate the OAA account more seriously than ever before. Yet the type of data they considered underscored the existence of a receptive deficit in Broca’s aphasia, and thus from that moment on, OAA was to be true or false not depending on comprehension deficit per se, but rather, on the degree of similarity between it and the deficit in production. Based on the current state of the evidence, I propose to do such an evaluation below. My examination will consist of several steps: first, I will go over the current state of the art in studies of agrammatic speech production. Second, I will summarize the situation in the comprehension domain. Next, differences and similarities between the two will be examined, to evaluate the current validity of the OAA. Finally, I will try to look at points of connection. PRODUCTION: A TREE-PRUNING HYPOTHESIS Consider the structural properties of the agrammatic speech production deficit: Recent investigations indicate that, contrary to traditional views, inflectional categories are not all equally impaired. Rather, it turns out that inflectional elements are impaired or preserved depending on their structural position on the syntactic tree. The first piece of evidence for this claim came about through a Hebrew speaking patient studied by Friedmann (1994), who was selectively impaired in the production of inflection: she had problems with tense, but not agreement. This finding runs contrary to common belief, according to which agrammatic aphasics have equal problems with all functional categories. Friedmann also showed, through a retrospective literature review, that cross-linguistic evidence goes in the same direction: a significant group of patients
Yosef Grodzinsky
76
reported in the literature also showed impairment in tense but not agreement (Nespoulous et al., 1988; Miceli, 1989; Silveri, Romani, & Caramazza, Saffran, Schwartz, & Marin, 1980) yet the opposite (impaired agreement but not tense) is never found: (2) Speaking English: The kiss . . . the lady kissed . . . the lady is . . . the lady and man and the lady . . . kissing. (3) Reading French aloud: Target: Bonjour, grand-mere, je vous ai apporte’ good morning, grandma, I to-you have bring (pres.-perf.) Read: Bonjour, grand-mere, je portrai euh je /pu/ /zeda/ a-aporte’ bring (future) Seeking to obtain a detailed error analysis, Friedmann then created a set of tests to track the exact nature of the impairment in tense versus agreement in speech production. The distinction made by the patients was especially important in light of developments in linguistic theory: the split inflection hypothesis (Pollock, 1989) proposes structural differences between tense and agreement, and argues that they each form a distinct functional category. This hypothesis not only provides a powerful and precise descriptive tool, but also presents a host of related issues to be examined. So, the tests were first conducted on one patient (Friedmann, 1994; Friedmann & Grodzinsky, 1997) and then extended to a group of 13 Hebrew and Arabic speaking patients (Friedmann, 1998; Friedmann & Grodzinsky, in press): Yesterday the boy walked; →
→
(4)
Tomorrow the boy ________. Tense condition
Yesterday the boys ________. Agreement conditon
The results were remarkable: While agreement was normal, tense was severely impaired, even though the patient’s perception of time, as well as comprehension of temporal adverbs, was shown to be intact. Tense errors were mostly substitutions of inflection (with no prefered “unmarked” form, observed in repetition (5), and in completion (6) tasks. In (7) a numerical representation of error rates is presented: (5) Target: ha’anashim yixtevu mixtav la-bank the-people write-future-3-m-pl letter to-the-bank Repeated: ha-anashim katvu mixtav la-bank the-people write-past-3-m-pl letter to-the-bank
3.
Overarching Agrammatism
77
(6) Target: axshav ata holex. etmol ’ata ____ (expected: halaxta) now you go-pres-2-m-sg Yesterday you ___ (go-past-2-m-sg) completed: axshav ata holex. etmol ata telex now you go-pres-2-m-sg. Yesterday you go-future-2-m-sg. (7) Agreement errors 4%
(28/642)
Tense errors 44%
(353/805)
This dissociation suggests a deficit that implicates tense, but not agreement features. This, in itself, is new, for agrammatic aphasia has always been thought to implicated all functional elements equally, and the striking asymmetries we observe appear to have been overlooked. Further, the impairment touches on a cluster of syntactic properties related to the tense node (according the split-inflection hypothesis), which are also disrupted: Observed are subject omissions, difficulties with copulas, and specific word order problems that pertain to nodes in the syntactic tree that are beyond the tense node, but nothing below this node is impaired. Moreover, the impairment is associated with problems in yet higher parts of the tree (CP). As a result, questions and embedded clauses are nonexistent or completely ill formed. By contrast, other properties that are related to agreement, and lower parts of the tree, are left intact. The distinction that linguists have posited receives direct neurological support. Moreover, the disruption affects the tree from the tense node and above, and leaves whatever is below it intact. This rather rich cluster of cross-linguistic facts has led to a description of agrammatic speech production that is stated over trees, not elements. That is, unlike every previous statement, which looked at functional elements regardless of their position in the sentence, the currently available data lead to the view that agrammatic aphasic patients produce trees that are intact up to the tense node and “pruned” from this node and up (Friedmann, 1994; Friedmann & Grodzinsky, 1997): (8) Tree-Pruning Hypothesis (TPH, Friedmann & Grodzinsky, 1997, simplified): Broca’s aphasics cannot represent T; higher branches of the tree are pruned (Figure 1). Interestingly, this claim receives empirical support from yet another direction: there is a salient cross-linguistic difference in the production of verbs by Broca’s aphasics. In English, the speech output of Broca’s aphasics contains verbs that are bare stems, yet these are in their proper position in the sentence — always after the subject. In verb-second (V2) languages (e.g., Dutch, German), however, where inflected verbs undergo
Yosef Grodzinsky
78
Figure 1
Tree-pruning hypothesis.
movement, the situation is different. In these languages verbs start out in sentence-final position, and must raise to pick up its tense features, and the result is subject-verb-object (SVO) order. A nonfinite verb in a main clause (e.g., in a clause that contains an inflected auxiliary), will remain in final position, and its finite counterpart will be in second position. In a patient whose syntactic tree is pruned, verbs will fail to raise, and the result will be as is observed in Dutch: in aphasic speech verbs in main clauses not only appear uninflected, but also, are in sentence final position, resulting in ungrammatical strings (Bastiaanse & Van Zonnenfeld, 1998; Friedmann, 1998). Dutch agrammatics make no errors on infinitives in subordinate clauses, but have major difficulties with inflecting main verbs, which they mostly produce not only as infinitives, but, critically, in final position. So, in sum, the deficit in speech production in Broca’s aphasia is strongly linked to structural factors: they fail to represent parts of the syntactic tree from a certain node up, and given that these underspecified nodes usually house inflectional categories, it was believed, until more recently, that all inflections are impaired; the current cross-linguistic evidence, however, suggests a partial deficit as described above. Next, we will examine comprehension patterns in Broca’s aphasia, and see whether these parallel those in production. As we have seen, for the OAA to be true, such a parallelism is a must.
3.
Overarching Agrammatism
79
RECEPTIVE ABILITIES: A RESTRICTIVE TRACE-DELETION HYPOTHESIS A syntactic deficit to receptive abilities on Broca’s aphasia has been shown time and again over the past three decades. That there is selectively within the syntax is a more recent claim. Thus Caramazza and Zurif (1976) attempted to ascribe “asyntactic comprehension” to these patients, which implied a total loss of syntax. This suggestion, which gave rise to OAA, was clearly too strong. It was only later experimentation that indicated that the impairment was not as widespread: certain syntactic structures gave rise to normal comprehension, whereas others were impaired. The issue has been the stability of these patterns, and their precise description. As matters currently stand, the comprehension patterns of Broca’s aphasics appear stable; moreover, it may very well be that cases whose comprehension had previously been thought to be deviant (like the apparent counterexamples to OAA that are mentioned above) in fact conform to the pattern, in that they fall within the allowable variation (cf. Grodzinsky, Pinango, Zurif, & Drai, 1998). As this pattern is currently described, the deficit mainly affects syntactic movement. That is, patients mostly fail (a) to comprehend movementderived structures (Grodzinsky, 1986, 1990, 1995); (b) to detect violations of grammaticality when movement rules are involved (Grodzinsky & Finkel, 1998); and (c) to properly process movement derived structures on-line (Zurif, Swinney, Prather, Solomon, & Bushell 1993; Swinney & Zurif, 1995). In (9), some examples of the comprehension performance of these patients are presented, according to level of performance for the group (cf. Grodzinsky, et al., 1998, for discussion of this issue): Construction type (9) a. The girl pushed the boy b. The girl who pushed the boy was tall c. Show me the girl who pushed the boy d. It is the girl who pushed the boy e. The boy was interested in the girl f. The woman was uninspired by the man (10) a. The boy was pushed by the girl b. The boy who the girl pushed was tall c. Show me the boy who the girl pushed d. It is the boy who the girl pushed e. The woman was unmasked by the man
Performance level above chance above chance above chance above chance above chance above chance chance chance chance chance chance
Yosef Grodzinsky
80
These experimental results (pooled from a large body of studies) speak against the “asyntactic comprehension” account. They suggest, rather, that while a syntactic deficit exists, it is partial. For details, see Grodzinsky (1995). Here, just a summary statement will be presented: given that the constructions in (10) are derived by a movement rule, the following account has been proposed (a revision of the trace-deletion hypothesis, TDH, Grodzinsky, 1986, 1990): (11) Trace Based Account (TBA, Grodzinsky, 1995): a. Trace deletion: Traces in -positions are deleted from agrammatic representation. b. R(eferential)-strategy: Assign a referential NP a role by its linear position if it has no -role. This proposal is corroborated by several types of results. Consider, first, comparisons with scores obtained from Broca’s aphasics in a language with a different phrasal geometry. If the account would hold there, its generality would be significantly greater. Japanese is one such test case, as it has a different basic word order when compared to English (SOV vs. SVO). Further, its interaction with transformational movement is orthogonal to the active/passive distinction: it has two types of active sentences and two types of passives — each with a movement-derived version [nonscrambled for active (12a), direct for passive (13a)], and a version that is not [scrambled (12b) for active and indirect (13b) for passive]. Hagiwara (Hagiwara, 1993; Hagiwara and Caplan, 1990) has documented the comprehension skills of these patients systematically, and showed that movement is the decisive factor determining their comprehension (cf. Grodzinsky, 1998, for analysis). The Japanese data are summarized in (12)–(13): (12) Japanese Active: a. Nonscrambled (basic): Taro-ga Hanako-ni nagutta -NOM -ACC hit Taro hit Hanako b. scrambled (movement-derived): Hanakoi-o Taro -ga ti nagutta (13) Japanese Passive: a. Direct (movement-derived): Taroi-ga Hanako-ni ti nagu-rare-ta -NOM -ACC hit-PASS-PAST Taro was hit by Hanako
above chance
chance
chance
3.
Overarching Agrammatism
81
b. Indirect (not derived): Okaasan-ga musuku-ni kaze-o hik-are-ta above chance mother-NOM a son-by a cold-ACC catch-PASS-PAST Mother had (her) son catch a cold on her These results provide an important cross-linguistic angle to the account, and strengthen it significantly. Next, our cross-linguistic account is further fortified with evidence that shows its intertask generality: in Grodzinsky and Finkel (1998) we tested some predictions of the TBA to grammaticality judgment. The leading idea of this study was that, if traces of movement are deleted from syntactic representations, then any task that critically relies on these constructs will reveal the deficit. Thus, we presented the patients with judgments such as those in (14)–(15). In order to be able to see that the strings in (14) are ungrammatical, a subject must know that the relationship between the (bolded) antecedent and the trace(t) is illicit. But for that, the trace must be represented. Our patients failed at this task. By contrast, they were successful in detecting ungrammaticality in other, analogous cases, in which traces were not involved, such as those in (15). (14) a. *John seems that it is likely t to win b. *Which woman did David think that t saw John? c. *I don’t know what who saw t (15) a. *Who did John see Joe? b. *Who John saw Joe? c. *The children sang the football over the fence d. *The children threw These results provide an important angle on the TBA, corroborating the claim that trace deletion is a proper characterization of Broca’s receptive skills through a study of grammaticality judgment. Next, a real-time perspective is in order. It is by now well established that normal language users demonstrate trace-antecedent relations in real-time tasks (e.g., Stowe, 1986; Swinney, Ford, Frauenfelder, & Bresnan, 1988; Bever & McElree, 1988; McDonald, 1989; Swinney & Zurif, 1995). The typical experiment exploits priming effects to uncover antecedent reactivation. The leading idea is that the link between a trace and its antecedent causes antecedent reactivation at the trace in the course of comprehension. Thus in (16), the drink will be active when heard (namely, at point 1), will then decay (2), but will get reactivated following the verb (3), due to its link to the trace.
Yosef Grodzinsky
82
(16) The priest enjoyed the drink1 that the caterer was2 serving t3 to the guests This is precisely what on-line experiments on normal language users have discovered. Experiments typically take the drink as prime, and while the sentence unfolds auditorily, a target is flashed on a screen at points (1)–(3). The expected finding, then, is that if a target word, say, juice, is presented visually to subjects at points (1), (2), and (3) when they are listening to the sentence, and the subjects have to make a lexical decision on it, priming effects would be documented at (1) and (3), but not at (2). This is found: priming effects are obtained only in (1) and (3). Consider, now, the TBA and the expected real-time behavior of Broca’s aphasics in such tasks. Deleted traces mean no reactivation at the trace. This means that only in point (1) would a priming effect be obtained. Decay would explain the lack of an effect in (2), and the correlate to tracedeletion would be a lack of priming in (3). Conducting such experiments is quite difficult, yet this is precisely the result of a series of carefully controlled studies of of both subject and object relative clauses (Zurif et al., 1993; Swinney & Zurif, 1995). Importantly, Broca’s aphasics do prime, even if not in a fully normal fashion (e.g., Swinney, Zurif, & Nicol, 1989; Shapiro & Levine, 1990; Shapiro, Gordon, Hack, & Killackey, 1993). Yet, when faced with a task that involves priming within a movement-derived construction, they are seriously impaired. Finally, this failure is not characteristic of all aphasics, nor is it necessarily related to general comprehension skills: Wernicke’s aphasics with posterior perisylvian lesions perform normally on this task, even though their comprehension abilities are severely compromised. In sum, then, the claim that trace deletion characterizes the comprehension deficit in Broca’s aphasia is reasonably corroborated by the empirical record. The next step in an assessment of OAA is to compare the linguistic accounts of receptive and productive skills in Broca’s aphasia. If they are equivalent, the OAA will be supported; otherwise, its validity will, at the very least, be in serious doubt. OVERARCHING AGRAMMATISM EXAMINED We must now compare the two accounts, that are repeated below. (17) PRODUCTION Tree-Pruning Hypothesis (TPH, Friedmann & Grodzinsky, 1997, simplified): Broca’s aphasics cannot represent T; higher branches of the tree are pruned.
3.
Overarching Agrammatism
83
(18) RECEPTION Trace Based Account (TBA, Grodzinsky, 1995): a. Trace deletion: Traces in -positions are deleted from agrammatic representation. b. R(eferential)-strategy: Assign a referential NP a role by its linear position if it has no -role. The question is whether the TPH is derivable from the TBA, or vice versa, or whether we can generalize over both. It is unclear how the pruning of subtrees can be made to follow from trace deletion; thus, (18) does not seem to follow from (17). The opposite, however, is not an impossibility. In fact, it has been proposed by Friedmann (1998), who seeks to derive the receptive deficit from its productive counterpart. The intuition behind her proposal is that in all cases for which there are comprehension results, the moved antecedents of the deleted traces are in subject positions which, by the TPH, are pruned off. If this account will hold, then OAA is vindicated. The notion is that any node beyond T cannot be constructed, and as a result, a constituent that is moved to such a position has no place on the tree, hence no thematic role. Since movement is to subject position, the aphasics’ failure stems from the fact that moved constituents have no position, hence, presumably, no thematic role, and their comprehension failure follows. Technical details aside, for the data above, this idea may work. Yet there are data that remain problematic for such an account, at least as stated. These are comprehension asymmetries, as well as direct contrasts between production and reception of language, mostly in the domain of verb movement. Consider the data in (19), obtained in a comprehension experiment by Hickok and Avrutin (1995), and (20) from Saddy (1995) and Balogh and Grodzinsky (1996): (19) a. b. c. d.
Who did the girl push t? Who pushed the girl? Which boy did the girl push t? Which boy pushed the girl?
above chance above chance chance above chance
(20) a. The man is pushed by the boy b. Every man is pushed by a boy
chance above chance
It is clear that just position on the tree constituent cannot determine the outcome. The contrast between (19a) and (19c) — a who object question and a which object question — demonstrated that. Both question expressions are in the same position, and still, a comprehension contrast is documented. Similarly, the contrast between the quantified and referential subjects in (20) determines comprehension, although they are clearly at
84
Yosef Grodzinsky
the same syntactic position. So, it would appear that for OAA to hold here, more must be put in the account. Moreover, one would like to demonstrate a direct prediction of OAA, Friedmann style: that movement into lower positions in the tree is intact. Finally, it seems that there is a direct comprehension/production contrast, in the domain of verb movement. While the production of moved verbs is severely impaired, as Friedmann has shown, all available data suggest that this is not the case for comprehension (cf. Grodzinsky & Finkel, 1998, for discussion). If these problems are resolved, then overarching agrammatism will be vindicated. Zurif’s original idea — that a supramodal account of the deficit is possible, and that as a consequence, Broca’s area is home for certain syntactic abilities for all modalities — will, potentially, be correct, with the necessary refinements.
A TRIBUTE TO EDGAR ZURIF A good friend of mine, Gedon Medini, once told a story of Duke Ellington who, in a TV interview with him late in his life, “was asked what he would like to say about the influences and events that shaped his starry life. He said something like this: ‘Ya know what it’s like runnin’ around life, like a maze, and you comes to a corner and ya don’t know which way to go, and then some guy is standing there and you asks him, “Which way?” and he points and says, “That way,” and ya run and run until you come to another fellow and you ask him, and he points and says, “That way,” and ya run and run until you come to another fellow and you ask him, and he points and says, “That way” — What I want to do today . . . is to thank all those guys who was standin’ on these corners at the right time and pointed me off in the right way.’” (Medini, 1988). Edgar’s role in the international quest for understanding brain/ language relations has been, in the past three decades, very central. As is clearly apparent from most contributions to this volume, and as an even cursory look at the current neuropsychological literature will reveal, he has been to many (certainly to me) that guy around the corner; and he did, it seems, point us off in the right way. ACKNOWLEDGMENTS The preparation of this paper was made possible by NIH Grant 00081 and DC 02984 to the Aphasia Research Center, Boston University School of Medicine, and Israel–U.S. Bi-national Science Foundation Grant 97–00451 to Tel Aviv University.
3.
Overarching Agrammatism
85
REFERENCES Balogh, J. & Grodzinsky, Y. (1996). Varieties of passive in agrammatism. Paper presented at the Academy of Aphasia, London. Bastiaanse, R., & Van Zonnenfeld, R. (1998). On the relation between verb position in Dutch agrammatic aphasics. Brain and Language 64, 165–181. Bever, T. G., & McElree, B. (1988). Empty categories access their antecedents during comprehension. Linguistis Inquiry 19(1), 35–45. Bradley, D. C., Garrett, M. F., & Zurif, E. B. (1980). Syntactic deficits in Broca’s aphasia. In D. Caplan (Ed.), Biological studies of mental processes. Cambridge, MA: MIT Press. Caramazza, A. (1986). On drawing inferences about the structure of normal cognitive processes from patterns of impaired performance: The case for single-patient studies. Brain and Cognition, 5, 41–66. Caramazza, A. & Zurif, E. B. (1976). Dissociation of algorithmic and heuristic processes in sentence comprehension: Evidence from aphasia. Brain and Language, 3, 572–582. Friedmann, N. (1994). Tense and agreement in an agrammatic patient. Masters thesis, Tel Aviv University. Friedmann, N. (1998). Functional categories in agrammatism. Doctoral dissertation, Tel Aviv University. Friedmann, N., & Grodzinsky, Y. (1997). Tense and agreement in agrammatic production: Pruning the syntactic tree. Brain and Language, 56, 397–425. Friedmann, N., & Grodzinsky, Y. (1999). Neurolinguistic evidence for split inflection. In M. A. Friedmann & L. Rizzi, (Eds.) The acquisition of syntax. London: Blackwell. Geschwind, N. (1970). The organization of language and the brain. Science, 170, 940–944. Goodglass, H. (1976). Agrammatism. In H. Whitaker & H. H. Whitaker (Eds.), Studies in neurolinguistics, Vol. 2. New York: Academic Press. Goodglass, H., & Kaplan, E. (1972). The assessment of aphasia and related disorders. Philadelphia, PA: Lea & Febiger. Grodzinsky, Y. (1986). Language deficits and the theory of syntax. Brain and Language, 27, 135–159. Grodzinsky, Y. (1990). Theoretical perspectives on language deficits. Cambridge, MA: MIT Press. Grodzinsky, Y. (1995). A restrictive theory of trace deletion in agrammatism. Brain and Language, 51, 26–51. Grodzinsky, Y. (1998). Comparative aphasiology: Some preliminary notes. In R. Bastiaanse & E. Visch-Brink (Eds.,) Levels of representation in aphasia. San Diego: Singular Press. Grodzinsky, Y., & Finkel, L. (1998). The neurology of empty categories: Aphasics’ failure to detect ungrammaticality. Journal of Cognitive Neuroscience, 10, 281–292. Grodzinsky, Y., Pinango, M. M., Zurif, E., & Drai, D. (1999). The critical role of group studies in neuropsychology: Comprehension regularities in Broca’s aphasia. Brain and Language, 67, 134–147. Hagiwara, H. (1993). The breakdown of Japanese passives and -role assignment principle by Broca’s aphasics. Brain and Language, 45, 318–339. Hagiwara, H. & Caplan, D. (1990). Syntactic comprehension in Japanese aphasics: Effects of category and thematic role order. Brain and Language, 38, 159–170. Hickok, G., & Avrutin, S. (1995). Comprehension of Wh-questions by two agrammatic Broca’ aphasics. Brain and Language, 51, 10–26. Kolk, H., van Grunsven, M. (1985). On parallelism in agrammatism. In M.-L. Kean (Ed.), Agrammatism. New York: Academic Press. McDonald, M. C. (1989). Priming effects from gaps to antecedents. Language and Cognitive Processes 4(1), 35–56.
86
Yosef Grodzinsky
Medini, G. (1988). Clinical psychology in Israel: Identity and directions. Israeli Journal of Psychiatry, 25, 157–166. Miceli, G., Mazzucchi, A., Menn, L., & Goodglass, H. (1983). Contrasting cases of English and Italian agrammatic aphasics. Brain and Language, 19, 65–97. Miceli, G., Silveri, M., Romani, C., & Caramazza, A. (1989). Variation in the pattern of omissions and substitutions of grammatical morphemes in the spontaneous speech of socalled agrammatic patients. Brain and Language, 36, 447–492. Nespoulous, J.-L., Dordain, M., Perron, Ska, B., Bub, D., Caplan, D., Mehler, J., & Lecours, A. R. (1988). Agrammatism in sentence production without comprehension deficits: Reduced availability of syntactic structures and/or of grammatical morphemes? A case study. Brain and Language, 33, 273–295. Pollock, J.-Y. (1989). Verb movement, universal grammar and the structure of IP. Linguistic Inquiry, 20, 365–424. Saddy, D. (1995). Variables and events in the syntax of agrammatic speech. Brain and Language, 50, 135–150. Saffran, E., Schwartz, M., & Marin, O. (1980). The word-order problem in agrammatism: II. Production. Brain and Language, 10, 263–280. Shapiro, L. P., Gordon, B., Hack, N., & Killackey, J. (1993). Verb–argument structure processing in complex sentences in Broca’s and Wernicke’s aphasia. Brain and Language, 45, 423–447. Shapiro, L. P., & Levin, B. A. (1990). Verb processing during sentence comprehension in aphasia. Brain and Language, 38, 21–47. Stowe, L. A. (1986). Parsing WH-constructions: Evidence for on-line gap location. Language and Cognitive Processes, 1, 227–245. Swinney, D., Ford, M., Fraunfelder, U., & Bresnan, J. (1988). On the temporal course of gapfilling and antecedent assignment during sentence comprehension. In B. Grosz, R. Kaplan, M. Macken, & I. Sag (Eds.), Language structure and processing, Stanford, CA: CSLI. Swinney, D. & Zurif, E. (1995). Syntactic processing in aphasia. Brain and Language, 50, 225–239. Swinney, D., Zurif, E. B., & Nicol, J. (1989). The effects of focal brain damage on sentence processing. An examination of the neurological organization of a mental module. Journal of Cognitive Neuroscience, 1, 25–37. Zurif, E. B. (1978). Language and the brain. In M. Halle, J. Bresnan, & G. Miller (Eds.), Linguistic theory and psychological reality. Cambridge, MA: MIT Press. Zurif, E. B. (1980). Language mechanisms: A neuropsychological perspective. American Scientist, May. Zurif, E. B. (1996). Grammatical theory and the study of sentence comprehension in aphasia: Comments on Druks and Marshall (1995). Cognition, 58, 271–279. Zuriff, E. B., & Caramazza, A. (1976). Linguistic structures in aphasia: Studies in syntax and semantics. In H. Whitaker & H. H. Whitaker (Eds.), Studies in neurolinguistics (Vol. 2). New York: Academic Press. Zurif, E. B., Caramazza, A. & Myerson, R. (1972). Grammatical judgments of agrammatic aphasics. Neuropsychologia, 10, 405–417. Zurif, E. B., Caramazza, Myerson, R., & Galvin, J. (1974). Semantic feature representations in normal and aphasic language. Brain and Language, 1, 167–187. Zurif, E. B., Swinney, D., Prather, P., Solomon, J., & Bushell, C. (1993). An on-line analysis of syntactic processing in Broca’s and Wernicke’s aphasia. Brain and Language, 45, 448–464.
Chapter
4
Speech Perception, Conduction Aphasia, and the Functional Neuroanatomy of Language Gregory Hickok
INTRODUCTION The classic functional–anatomic model of language holds that Broca’s and Wernicke’s aphasia are the only syndromes that result from damage to an actual language processing “center.” All the rest result from a disconnection of one computational system from another. In the case of conduction aphasia, a central focus of this chapter, the disconnection supposedly involves the pathway between the two primary language centers themselves, Broca’s area and Wernicke’s area. In this chapter, I will argue that at least one type of conduction aphasia is not a disconnection syndrome at all, but a central language disorder in its own right, one which impairs phonological systems that normally take part not only in speech production, as has been claimed previously, but also in speech perception. Because Wernicke’s model is frequently misunderstood, it’s worth starting by reviewing his original 1874 proposal. The Classic Model of the Functional Neuroanatomy of Language Wernicke (1874/1977) proposed that language processing is supported by two centers, one in auditory cortex which houses sound-based representation of words, and the other in motor cortex which houses motorbased representations of the articulatory gestures required to produce words. According to Wernicke, these two language centers are connected to one another directly, and also to a highly distributed network involved
Language and the Brain
87
Copyright © 2000 by Academic Press. All rights of reproduction in any form reserved.
88
Gregory Hickok
Figure 1 The present author’s representation of Wernicke’s model of the functional neuroanatomy of language. Directional arrows indicate sensory and motor input/output to the system. Nodes A and M are centers for representing the sensory and motor traces of linquistic events: A, center for auditory word representation; M, center for motor word representation. The top row of unlabeled nodes denotes a distributed cortical network for conceptual knowledge representation.
in the representation of conceptual knowledge. The connections between the language centers and the distributed conceptual representation system were assumed to be bidirectional, at least for the connection between auditory cortex and conceptual representation. Figure 1 summarizes Wernicke’s model graphically. This representation of the model more accurately reflects Wernicke’s ideas than does the oft used Lichtheim (1885) diagram, which is misleading with its unidirectional connections.1 Wernicke assumed that in normal language comprehension acoustic input activated stored auditory representations associated with the sound pattern of words, these auditory word representations then activated their associated conceptual representations. Volitional production, according to Wernicke, started with the activation of conceptual representations which in turn activated both their associated auditory and motor representations in parallel. The activation of auditory word representations served to constrain the selection of the appropriate (i.e., intended) motor pattern via the direct auditory– motor pathway: . . . the actual sensory image [concept] of an object is . . . able to activate the motor image directly . . . Observations of daily 1 This is no fault of Lichtheim’s. The diagram from Lichtheim’s paper (1885) which is so often used to represent the Wernicke–Lichtheim model was not Lichtheim’s conception of how the system works, but rather a hypothesis which he later rejects. Although not diagramming it fully, the version he adopts is one in which the connection between the auditory word center and the conceptual “center” is bidirectional, as Wernicke proposed.
4.
Functional Neuroanatomy of Language
89
speech usage and the process of speech development indicates the presence of an unconscious, repeated activation and simultaneous mental reverberation of the acoustic image which exercises a continuous monitoring of the motor images. (p. 106) Therefore, the sum total of [the object concept and the acoustic word image] always functions in harmony with the appropriate intensity necessary for correct selection of the word. (p. 107) Aphasic syndromes could be accounted for within this framework. Damage to the auditory word center leads to impaired comprehension because the sound pattern associated with the individual words can not be activated; production is fluent because motor memories are preserved and because they can be activated via the direct concept-to-motor pathway, but speech errors (paraphasias) are evident because the modulatory influence of auditory word representations on motor selection is lost. Damage to the motor word center yields impaired (i.e., nonfluent) production while leaving comprehension intact. And damage to the direct connection between auditory and motor systems leads to paraphasic output similar to that found with damage to the auditory word center because the lesion interrupts the pathway by which sensory–motor interaction takes place, but with spared comprehension because the auditory word center is intact. These symptom patterns correspond to Wernicke’s, Broca’s and conduction aphasia, respectively. Lichtheim later elaborated on the predictions of the model showing how the transcortical aphasias follow from damage to the connections between the language centers and (distributed) conceptual representations. A Hypothesis What I would like to suggest here is that Wernicke was essentially correct in hypothesizing (i) that auditory cortex participates in speech production and (ii) that the production deficit in conduction aphasia can be explained in terms of a loss of the modulatory influence of auditory-based representations on articulatory planning. Wernicke was wrong, however, in assuming that conduction aphasia is a disconnection syndrome. Based on a range of evidence I will argue that at least one type of conduction aphasia results from damage to phonological processing systems in auditory cortex which participate both in speech perception and in speech production. I will argue further that the asymmetry in conduction aphasia between production (impaired) and comprehension (spared) can be explained in terms of different degrees of lateralization in the computational systems that support (or can support) these two functions.
Gregory Hickok
90
The hypothesis put forth here is motivated primarily by two observations. The first is that there is mounting evidence that auditory cortex in the posterior supratemporal plane (pSTP; see Figure 2), is the primary substrate for speech perception (Hickok & Poeppel, to appear ).2 The second observation is that conduction aphasia can result from damage to exactly this cortical region in the left hemisphere (Damasio & Damasio, 1980).3 These observations lead to an apparent paradox: how is it that damage to speech perception systems can be associated with a syndrome in which auditory language comprehension is preserved and in which speech production is impaired? This paradox can be resolved given the following two assumptions: (i) that speech perception is mediated by the pSTP bilaterally (Boatman et al., 1998; Hickok & Poeppel, to appear), and (ii) that auditory cortex in the left pSTP participates in speech production, as Wernicke originally claimed. On these assumptions, one can explain the relatively spared auditory comprehension ability in conduction aphasia in terms of the right hemisphere’s capacity for speech perception, and the impaired production in terms of damage to those left auditory cortex systems that play an important role in speech production. Given the post hoc nature of these two assumptions, it is important to determine if there is any additional evidence in support of these claims. We turn to that evidence now.
EVIDENCE FOR BILATERAL ORGANIZATION OF SPEECH PERCEPTION SYSTEMS Profound Deficits in Auditory Comprehension Are Associated with Bilateral Posterior Supratemporal Plane Lesions If it is the case that relatively preserved auditory comprehension in conduction aphasia can be attributed to the speech perception abilities of the
2
I will follow Hickok and Poeppel’s (to appear) definition of speech perception, which is “the process by which acoustic input is coded into neural representations suitable for making contact with the mental lexicon.” This definition refers explicitly to those processes involved in normal language understanding, and not necessarily to those processes involved in performing laboratory speech-sound identification and discrimination tasks. 3 Conduction aphasia is known to result from two lesion patterns. The most well-known is that which involves the supramarginal gyrus and the underlying arcuate fasciculus. But very clear cases of conduction aphasia have also been identified with lesions involving auditory cortex in the left supratemporal plane with complete sparing of the supramarginal gyrus and the arcuate fasciculus. Unless otherwise indicated, the term “conduction aphasia” is used here to refer to this latter STP-lesioned form. It will be important in future research to tease apart the symptom complex of these two forms of conduction aphasia.
4.
Functional Neuroanatomy of Language
91
Figure 2 Lateral view of the left hemisphere with structures inside the sylvian fossa exposed. H, Transverse temporal gyrus of Heschl (which houses primary auditory cortex); PT, planum temporale; aSTP, anterior supratemporal plane; STG, superior temporal gyrus, note that the STG includes the STP; MTG, middle temporal gyrus; AG, angular gyrus; SMG, supramarginal gyrus; 44 and 45 refer to Brodmann’s designations and together comprise Broca’s area; PCG, precentral gyrus (primary motor cortex); FO, frontal operculum; PO, parietal operculum; I, insula. The term pSTP is used in this chapter to refer to the combination of Heschl’s gyrus and the PT.
right pSTP, then it should be the case that if the pSTP were damaged bilaterally, a profound auditory comprehension deficit should result. This is exactly the most common etiology of word deafness (WD) an acquired syndrome which is characterized by a profound speech comprehension deficit (Buchman, Garron, Trost-Cardamone, Wichter, & Schwartz, 1986). The classic functional–anatomic interpretation of WD is that of a disconnection syndrome in which Wernicke’s area is isolated from auditory input (Geschwind, 1965). An alternative interpretation suggested here is that WD results from damage to cortical speech perception systems (Hickok & Poeppel, to appear). On this latter view, WD should be considered a form of aphasia, rather than a form of auditory agnosia peripheral to the aphasias. It is worth pointing out in this context that WD is the only aphasic syndrome dominated by a profound deficit in speech perception. To put it
92
Gregory Hickok
another way, none of the left unilateral-lesion-based aphasic syndromes have a speech perception deficit as a major component of the syndrome, which in turn further supports the view that speech perception is mediated bilaterally. This is not to say that speech perception deficits (as measured by syllable identification and discrimination) never occur in the unilateral aphasias. On the contrary, abnormal performance on such tests have been found in association with a range of aphasic types, but such deficits are not predictive of auditory comprehension performance (Blumstein, 1995; Blumstein, Baker, & Goodglass, 1977), suggesting that speech perception processes are functioning sufficiently well to support some degree of access to phonological information. This is true even for Wernicke’s aphasics: Blumstein et al. (1977) showed that the comprehension deficit in Wernicke’s aphasia is not predominantly phonological in nature, as one would expect if a speech perception deficit was the source of their comprehension problems, but rather predominantly semantic in nature. For this reason it has been argued that the comprehension deficit in Wernicke’s aphasia represents a disorder somewhere in the mapping between phonological representations and the meanings with which they are associated (Geschwind, 1965). This in turn suggests that while speech perception processes are bilaterally organized, higher levels of language are more asymmetrically processed in the left hemisphere. The Isolated Right Hemisphere Has Speech Perception Capacities Evidence from split brain studies and intracarotid amobarbital studies suggests that the isolated right hemisphere has the capacity to both discriminate speech sounds (Boatman et al., 1998), and to understand syntactically simple speech (McGlone, 1984: Wada & Rasmussen, 1960; Zaidel, 1985). These findings are consistent with the view that speech perception systems are bilaterally organized. Bilateral Activation in the Physiological Response to Speech Intraoperative recordings from single units in awake patients undergoing surgical treatment for epilepsy have suggested the existence of cells responsive to different aspects of the speech signal such as a particular class of phonemic clusters, mono- versus multisyllabic words, task-relevant versus task-irrelevant speech, and natural versus distorted or backward speech (Creutzfeldt et al., 1989). The vast majority of speech responsive cells sere found in the middle portion of the superior temporal gyrus (STG) (recordings from more posterior sites were not made); very few sites in anterior STG or middle or inferior temporal gyrus yielded de-
4.
Functional Neuroanatomy of Language
93
tectable responses. Furthermore these units were found in equal distribution in the left and right hemispheres. Bilateral activation of pSTG is also a consistent finding of positron emission tomography (PET) (Mazoyer et al., 1993; Petersen, Fox, Posner, Mintun, & Raichle, 1988; Price et al., 1996; Zatorre, Evans, Meyer, & Gjedde, 1992) functional magnetic resonance imaging (fMRI)(Binder et al., 1994; Millen, Haughton, & Yetkin, 1995), and magnetoencephalography (MEG) (Gage, Poeppel, Roberts, & Hickok, 1998; Poeppel et al., 1996) studies of passively presented speech stimuli. Some studies have reported that the left hemisphere activation is slightly more extensive; this asymmetry may reflect higher levels of language processing which do seem to be more strongly lateralized. Thus, it seems that there is good evidence in favor of the view that speech perception systems are organized bilaterally in pSTP. There is also evidence suggesting that the left and right hemisphere make nonidentical contributions to the speech perception process. This evidence has been reviewed elsewhere (Hickok & Poeppel, to appear) and will not be detailed here.
EVIDENCE FOR LEFT POSTERIOR SUPRATEMPORAL PLANE PARTICIPATION IN SPEECH PRODUCTION The observation that conduction aphasia is associated with damage to the pSTP is one source of evidence that this region participates in speech production. There are several additional lines of evidence which also support this view. Cortical Stimulation of the Left Superior Temporal Gyrus Results in Speech Production Errors Intraoperative cortical stimulation of left pSTG has been shown to produce speech production errors characterized by “distortion and repetition of words and syllables” (Penfield & Roberts, 1959). Functional Imaging Studies of Speech Production Implicate Left Posterior Supratemporal Plane A magnetic source imaging (MSI) study of speech production during object naming has identified the left posterior superior temporal region as the source for a signal that is produced just prior to speech articulation (Levelt, Praamstra, Meyer, Helenius, & Salmelin, 1998); the authors write that the left pSTG is, “a favorite site for phonological encoding in word production” (p. 562). And a recent fMRI study has provided direct
94
Gregory Hickok
evidence that auditory cortex in the left supratemporal plane participates in speech production (Hickok et al., 1999). In this study subjects were asked to name (subvocally) visually presented objects. Despite the fact that there was no auditory stimulus, activation was noted in nonprimary unimodal auditory cortex in the pSTP. No activation was noted in right auditory cortex. These findings confirm and refine the MSI localization.
Production Deficits in Word Deafness If we are correct in hypothesizing that (i) conduction aphasia is the result of damage to speech perception systems in the left hemisphere, (ii) that these same systems in pSTP participate in speech production, and (iii) that word deafness is the result of damage to speech perception systems in pSTP bilaterally, then we are lead to the following prediction: the output of word deaf patients should resemble the output of conduction aphasics. A common textbook assumption regarding word deafness is that production is perfectly intact, but in fact this is not typically the case. In a review of 37 cases of word deafness, Buchman et al. (1986) report that more than 70% (22/31) of the cases for which there was sufficient information, presented with paraphasic errors consistent with the predicted effect.
Behavioral Evidence for a Relation between Speech Perception and Production The evidence above makes a reasonably strong case for the view that systems in the left pSTP participate not only in perception, but also in production. Are the very same pSTP cortical networks involved in both processes? Or are there separate systems within auditory cortex, one involved with speech perception, the other with speech production? There is some behavioral evidence which speaks to the question. Mackay, Wulf, Yin, and Abrams (1993) studied the effect of concurrent speech production on the verbal transformation effect (Warren, 1968), that is, the tendency to misperceive phonemes with repeated and prolonged presentation of a single word. In a series of experiments, they found that this tendency was reduced during concurrent speech production, even when production was subvocal. They argue that these findings support a processing model in which speech production and speech perception systems share processing units. Based on this evidence, we might then hypothesize that the same neural networks in left pSTP participate both in perception and production.
4.
Functional Neuroanatomy of Language
95
RELATION BETWEEN PHONOLOGICAL ENCODING, REPETITION, AND NAMING DEFICITS IN CONDUCTION APHASIA So far we have considered evidence which suggests (i) that the preserved comprehension ability in conduction aphasia is due to the speech perception capacity of the right hemisphere, and (ii) that the phonemic paraphasias often present in the speech output of conduction aphasics can be accounted for in terms of damage to phonological processing systems in the left pSTP which participate both in speech perception and speech production. What about other aspects of the symptom complex of conduction aphasia, specifically the repetition deficit and the naming difficulties? I will propose that both of these symptoms might be reducible to difficulties in phonological encoding, that stage in language production where the phonological segments associated with selected lexical items are assembled (e.g., Levelt, 1989). The idea that a deficit in phonological encoding is the source of the production deficit in conduction aphasia is not new (Buckingham, 1992; Kohn, 1984; Shallice & Warrington, 1977), and so these issues will be discussed only briefly. Repetition Deficit The difficulty with verbatim repetition of speech is the aspect of conduction aphasia which is commonly considered to be the defining feature of the disorder. In fact conduction aphasia was first identified by Wernicke on the basis of paraphasias is speech production, not repetition ability. Based on Wernicke’s assertion that (i) the conduction aphasics’ paraphasias resulted from an interruption of the direct link between sensory and motor language systems and (ii) this pathway was critical for repetition of speech, Lichtheim proposed repetition as a simple diagnostic test of the integrity of this pathway. Repetition, then, was originally viewed simply as a diagnostic test which specifically targeted the functional component thought to underlie the speech production deficit in conduction aphasia. It was not viewed as an independent symptom. This is roughly the view taken by modern researchers (Buckingham, 1992; Shallice & Warrington, 1977)—at least for one type of conduction aphasia, referred to as reproduction conduction aphasia—and the view that is adopted here. Naming Difficulties One common feature of conduction aphasia is naming difficulty (Benson et al., 1973; Goodglass, 1993). While virtually every aphasic has word
96
Gregory Hickok
finding problems, it is interesting that conduction aphasics tend to make naming errors that are phonologically related to the target (Kohn, 1984), and tend to respond to phonemic but not semantic cueing (Li & Williams, 1990). This has suggested to some that the naming deficit in conduction aphasia stems from the proposed phonological encoding disorder. An experiment by Vigliocco, Antonini, & Garrett (1998) is enlightening in this respect. These authors examined the tip-of-the-tongue (TOT) phenomenon in Italian speaking subjects. The TOT phenomenon is interesting in the present context because it represents a state in which the speaker has accessed conceptual/semantic information but cannot access the phonological shape of the associated word. The motivation for studying Italian speaking subjects was that the grammatical gendermarking system in Italian provided a means for determining at what level of representation (e.g., lexical/lemma level or phonological encoding level) the difficulty arose in the TOT state. Their study showed that Italian speakers have access to grammatical gender information in the TOT state. Since grammatical gender information is typically assumed to be represented at the lemma level (Garrett, 1992; Levelt, 1989), it is reasonable to conclude that the locus of processing difficulty is at the level of phonological encoding. The implications of this finding for conduction aphasia are as follows. The TOT state is a form of naming failure which arises from difficulties in phonological encoding. Conduction aphasia, which has been characterized as a deficit in phonological encoding , is associated with naming deficits. The left pSTP is implicated both in the etiology of conduction aphasia, and in the phonological encoding stage of speech production based on functional imaging studies of object naming (Hickok et al., 1999; Levelt et al., 1998). This constitutes rather strong circumstantial evidence suggesting that the naming deficit in conduction aphasia stems from difficulties in phonological encoding. The observation that naming difficulties in conduction aphasia often resemble a TOT state is further evidence supporting this view (Goodglass, Kaplan, Weintraub, & Ackerman, 1976).
OTHER AREAS PREVIOUSLY IMPLICATED IN SPEECH PERCEPTION MAY REFLECT A NETWORK SUPPORTING AUDITORY–MOTOR INTERACTION The claim put forth here is that the pSTP, bilaterally, is the primary substrate for speech perception. How does this hypothesis relate to previous claims that Broca’s area and/or the supramarginal gyrus is/are impor-
4.
Functional Neuroanatomy of Language
97
tant for speech perception? Hickok and Poeppel (to appear) suggested that Broca’s area and the SMG are involved in speech perception only indirectly through their role in phonological working memory which may be recruited during the performance of certain speech perception tasks. Evidence for this claim includes the observation that both of these regions are active during the performance of phonological working memory tasks as measured by hemodynamic-based functional imaging studies (Awh et al., 1996; Paulesu, Frith, & Frackowiak, 1993), and the observation that they are not reliably activated during passive listening to speech sounds—a task that certainly involves speech perception. Hickok and Poeppel (to appear) go on to suggest that the pSTP, SMG, and Broca’s area serve as a network for auditory–motor interaction. Specifically, they argue that the pSTP is important for generating soundbased representations of speech which interface with lexical–conceptual systems on the one hand, and with speech-related motor–articulatory systems, such as Broca’s area on the other. In this scheme, the SMG serves as an interface between speech-related auditory and motor systems, and together this circuit comprises an important substrate for phonological working memory (Aboitiz & García, 1997). This hypothesis was motivated by analogy to work in the visual domain which seems to indicate that the parietal lobe is important for visual–motor integration (Rizzolatti, Fogassi, & Gallese, 1997).
IMPLICATIONS FOR ANATOMICAL MODELS OF LANGUAGE The present view of conduction aphasia is rather different from the classic view, which as noted at the outset, explains the disorder as a disconnection syndrome. How does this proposal fit into a more general model of the neuroanatomy of the aphasias? One possible formulation of a more general model is summarized in diagram form in Figure 3. Toward a Model of the Functional Neuroanatomy of Language According to this model, the pSTP (bilaterally) is the primary location of a network supporting sound-based computation of speech. This system supports both speech perception and aspects of speech production. This sound-based system interfaces with widely distributed conceptual knowledge systems (Damasio, Grabowski, Tranel, Hichwa, & Damasio, 1996) via auditory–conceptual interface systems in cortex located primarily at the junction of the left temporal, occipital, and parietal lobes. Damage to this region is associated with impaired comprehension, but
Gregory Hickok
98
Figure 3
A rough model of the functional neuroanatomy of language.
spared repetition (Damasio, 1992). The idea that there exists networks responsible for binding together representations of different sorts (in this case phonological and semantic) is not new. Damasio (1989) and Mesulam (1990, 1998) have been arguing for such “interface zones” (or “convergence zones” in Damasio’s terminology), for more than a decade. And Geschwind (1965) suggested that Wernicke’s area “functions importantly as the ‘storehouse’ of auditory associations. . . . It is probably thus that Wernicke’s area attains its essential importance in ‘comprehension,’ i.e. the arousal of associations” (p. 286). This notion is also consistent with psycholinguistic models which postulate an intermediate representational level (the “lemma” level) between sound- and meaning-based representations (Dell, Schwartz, Martin, Saffran, & Gagnon, 1997; Garrett, 1992; Levelt, 1989; Vigliocco et al., 1998). The present model assumes that the right hemisphere systems supporting speech perception can provide input to this left hemisphere auditory– conceptual interface systems. This assumption is driven by the observation that damage to the left pSTP does not significantly impair language
4.
Functional Neuroanatomy of Language
99
comprehension. There may also be a projection between the left and right pSTP (Damasio & Damasio, 1979). The sound-based system interfaces not only with the conceptual knowledge system, but also with frontal motor systems via an auditory–motor interface system in the inferior parietal lobe. This circuit is the primary substrate for phonological working memory (Awh et al., 1996; Paulesu et al., 1993), but also probably plays a role in volitional speech production. In the present framework, this auditory cortex–SMG–Broca’s area circuit replaces the notion of a direct sensory–motor link via the arcuate fasciculus. Connectivity studies in nonhuman primates is consistent with the claim that there is not a direct (i.e., one synapse) connection between auditory cortex and the ventral posterior frontal lobe, and with the claim that the parietal lobe is an important interface between these regions (Aboitiz & García, 1997; Romanski, Bates, & Goldman-Rakic, 1999).4 Following the lead of several authors (Friedrich, Glenn, & Marin, 1984; McCarthy & Warrington, 1984; Wernicke, 1874/1977) the present model also hypothesizes a link between conceptual representations and frontal lobe systems. On the assumption that conceptual representations are highly distributed, this “link” likely involves the convergence of many inputs onto interface systems in frontal cortex (Damasio, 1989; Mesulam, 1998). An example of one such input comes from (i) the observation that regions in the left inferior temporal lobe are important for accessing object-based lexical/conceptual knowledge for production (Damasio et al., 1996), and (ii) connectivity data in nonhuman primates showing projections from inferior temporal lobe to frontal cortex (Bullier, Schall, & Morel, 1996). Together, these findings suggest the possibility that a network involving projections from the inferior temporal lobe (and likely other structures) to frontal systems play a role in speech production. The present hypothesis is that this is (one part of) the previously proposed concept-to-motor route for speech production. 4
There is an alternative formulation of the respective roles of the auditory cortex and SMG in speech production which is suggested by the monkey connectivity data. As mentioned, auditory cortex does not project to the homologue of Broca’s area in the macaque, but it does project to the parietal lobe which in turn projects to “Broca’s area.” Auditory cortex has additional projections to the frontal lobe just anterior and superior to the Broca’s area homologue (Romanski et al., 1999). Perhaps the auditory–parietal–Broca’s area circuit is used for conscious access to phonological representations as when one is actively rehearsing speech, while an auditory-anterior/superior frontal–Broca’s area circuit is used for natural speech production where access to sound based information proceeds in a relatively automatic and unconscious fashion. This hypothesis predicts that conduction aphasia which results from auditory cortex lesions versus SMG lesions should yield different patterns of production deficits that cleave along an automatic versus controlled process line.
100
Gregory Hickok
Relation to Aphasia Syndromes This rough sketch of a model can be used to interpret the clinical features of the fluent aphasias. Conduction aphasia One form of conduction aphasia results from damage to sound-based representations of speech in the left pSTG, which are used both for comprehension and production. Comprehension is relatively spared because the right pSTP can support speech perception operations sufficient for comprehension tasks. Production contains phonemic paraphasias because the left (but not right) pSTP is involved in “