The Cognitive Neuroscience of Human Communication
The Cognitive Neuroscience of Human Communication
Vesna Mildner
L...
62 downloads
1847 Views
4MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
The Cognitive Neuroscience of Human Communication
The Cognitive Neuroscience of Human Communication
Vesna Mildner
Lawrence Erlbaum Associates New York London
Lawrence Erlbaum Associates Taylor & Francis Group 270 Madison Avenue New York, NY 10016
Lawrence Erlbaum Associates Taylor & Francis Group 2 Park Square Milton Park, Abingdon Oxon OX14 4RN
© 2008 by Taylor & Francis Group, LLC Lawrence Erlbaum Associates is an imprint of Taylor & Francis Group, an Informa business Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-13: 978-0-8058-5436-7 (Softcover) 978-0-8058-5435-0 (Hardcover) No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Mildner, V (Vesna) The cognitive neuroscience of human communication / Vesna Mildner. p. cm. Includes bibliographical references and index. ISBN 0-8058-5435-5 (alk. paper) -- ISBN 0-8058-5436-3 (pbk.) 1. Cognitive neuroscience. 2. Communication--Psychological aspects. 3. Communication--Physiological aspects. I. Title. QP360.5.M53 2006 612.8’233--dc22 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com
2005049529
For Boris, without whom none of this would be possible or even matter.
Contents Foreword .............................................................................................................. xi Raymond D. Kent Preface ...............................................................................................................xiii Chapter 1 Central Nervous System .................................................................. 1 The Development of the Central Nervous System ................................................ 1 Structure and Organization of the Central Nervous System ................................ 5 Sensation and Perception .................................................................................... 24 Neural Bases of Speech Perception and Production ........................................... 26 Hearing, Listening and the Auditory Cortex ................................................. 26 Movement and Speech Production................................................................. 30 Relationship Between Speech Production and Perception ............................ 34 Neighboring Location of Motor and Sensory Neurons ............................. 35 Multimodal Neurons ................................................................................. 36 Parallel and Recurrent Pathways ............................................................... 37 Chapter 2 Sex Differences ............................................................................. 39 Structural Differences......................................................................................... 39 Differences in Functional Organization of the Brain ......................................... 40 Behavioral and Cognitive Differences ................................................................ 40 Chapter 3
Brief History of Neurolinguistics from the Beginnings to the 20th Century ....................................................................... 45
Chapter 4 Research Methods ......................................................................... 51 Clinical Studies ................................................................................................... 51 Studies of Split-Brain Patients ....................................................................... 53 Cortical Stimulation ............................................................................................ 54 Transcranial Magnetic Stimulation (TMS).................................................... 55 Wada Test ............................................................................................................ 55 Neuroradiological Methods ................................................................................ 56 Computerized (Axial) Tomography—C(A)T ................................................. 56 Magnetic Resonance Imaging (MRI) ............................................................ 56 Functional Magnetic Resonance Imaging (fMRI)......................................... 57 Recording of Activity ......................................................................................... 57 Electrophysiological Methods ........................................................................ 58 Single-Unit or Single-Cell Recording ....................................................... 58 Electroencephalography (EEG) ................................................................ 59 vii
viii
The Cognitive Neuroscience of Human Communication
Event-Related Potentials (ERP) ................................................................ 59 Cortical Cartography................................................................................. 60 Magnetoencephalography (MEG) ............................................................. 60 Radioisotopic Methods .................................................................................. 61 Positron Emission Tomography (PET)...................................................... 61 Single-Photon Emission Computed Tomography (SPECT) ...................... 62 Ultrasound Methods ....................................................................................... 62 Functional Transcranial Doppler Ultrasonography (fTCD)...................... 62 Summary ........................................................................................................ 62 Behavioral Methods ....................................................................................... 63 Paper-and-Pencil Tests .............................................................................. 64 Word Association Tests ............................................................................. 64 Stroop Test................................................................................................. 64 The Wisconsin Card Sorting Test (WCST)............................................... 64 Priming and Interference .......................................................................... 64 Shadowing ................................................................................................. 65 Gating ........................................................................................................ 65 Dichotic Listening ..................................................................................... 66 Divided Visual Field.................................................................................. 67 Dual Tasks ................................................................................................. 67 Summary ........................................................................................................ 68 Aphasia Test Batteries......................................................................................... 68 Chapter 5
The Central Nervous System: Principles, Theories and Models of Structure, Development and Functioning .................... 71 Principles ............................................................................................................ 71 Hierarchical Organization.............................................................................. 71 Parallel Processing ......................................................................................... 72 Plasticity ......................................................................................................... 72 Lateralization of Functions ............................................................................ 73 Theories and Models........................................................................................... 73 Parallel or Serial Processing? ........................................................................ 74 Localistic Models ........................................................................................... 75 Wernicke–Geschwind Model .................................................................... 76 Hierarchical Models ....................................................................................... 79 The Triune Brain ....................................................................................... 79 Luria’s Model of Functional Systems ........................................................ 80 Jurgens’ Model of Neural Vocalization Control ....................................... 82 Modular Models ............................................................................................. 82 Cascade Models ............................................................................................. 84 Interactive Models .......................................................................................... 85 Connectionist Models .................................................................................... 86 Neural Networks ............................................................................................ 86 Other Theories and Models............................................................................ 92
Contents
ix
Motor Theory of Speech Perception ......................................................... 93 Analysis by Synthesis ................................................................................ 94 Auditory Theory ........................................................................................ 95 Neural (Phonetic, Linguistic) Feature Detectors....................................... 95 Theory of Acoustic Invariance.................................................................. 95 The Cohort Theory.................................................................................... 96 Trace Model............................................................................................... 96 The Neighborhood Activation Model (NAM) .......................................... 96 PARSYN ................................................................................................... 97 The Mirror–Neuron System ...................................................................... 97 Chapter 6 Lateralization and Localization of Functions ............................... 99 Lateralization of Functions ................................................................................. 99 Verbal Versus Nonverbal and Language Versus Spatial Information ......... 103 Analytic Versus Holistic Approach to Processing ....................................... 107 Serial or Sequential Versus Parallel Processing .......................................... 108 Local Versus Global Data Representation ................................................... 109 High Frequencies Versus Low Frequencies ..................................................110 Categorical Versus Coordinate .....................................................................113 Developmental Aspects of Lateralization..........................................................113 Neuroanatomic Asymmetries .......................................................................119 Sensory Asymmetries .................................................................................. 120 Motor Asymmetries ..................................................................................... 121 Asymmetries in Other Species .................................................................... 122 Factors Influencing Functional Cerebral Asymmetry ...................................... 123 Localization of Functions ................................................................................. 127 Lateralization and Localization of Emotions .............................................. 134 Summary........................................................................................................... 137 Chapter 7 Learning and Memory ................................................................ 139 Plasticity............................................................................................................ 139 Critical Periods ................................................................................................. 144 Types of Memory .............................................................................................. 149 Sensory Memory .......................................................................................... 149 Short-Term/Working Memory ..................................................................... 150 Long-Term Memory ..................................................................................... 154 Neural Substrates of Memory ........................................................................... 155 Chapter 8 Speech and Language...................................................................161 Speech and Language Functions and Their Location in the Brain .................. 163 Anatomic Asymmetries and Lateralization of Speech and Language ..............167 Split-Brain Patients ........................................................................................... 168 Healthy Subjects ............................................................................................... 170
x
The Cognitive Neuroscience of Human Communication
Speech Production and Perception ................................................................... 172 Speech Production ....................................................................................... 172 Speech Perception .........................................................................................176 Phonetics and Phonology .................................................................................. 179 Tone and Prosody.............................................................................................. 185 Lexical Level and Mental Lexicon ................................................................... 190 Word Recognition ........................................................................................ 195 Perceptual Analysis of Linguistic Input....................................................... 197 Word Categories ........................................................................................... 199 Sentence Level: Semantics and Syntax............................................................. 205 Discourse and Pragmatics..................................................................................210 Reading ............................................................................................................. 212 Writing .............................................................................................................. 215 Calculation .........................................................................................................216 Is Speech Special?..............................................................................................217 Language Specificities ...................................................................................... 220 Bilingualism...................................................................................................... 222 Speech and Language Disorders....................................................................... 229 Aphasia ......................................................................................................... 231 Recovery of Language Functions: Functional Cerebral Reorganization ..... 234 Agraphia and Alexia .................................................................................... 237 Motor Speech Disorders ................................................................................... 241 Dysarthria .................................................................................................... 241 Apraxia of Speech ........................................................................................ 242 Stuttering ...................................................................................................... 243 Other Causes of Speech and Language Disorders ............................................ 244 Schizophrenia............................................................................................... 244 Epilepsy and Tumors .................................................................................... 245 Right-Hemisphere Damage .......................................................................... 246 Epilogue ........................................................................................................... 249 Glossary ........................................................................................................... 251 Appendix .......................................................................................................... 295 References ........................................................................................................ 299 Author Index ....................................................................................................331 Subject Index ................................................................................................... 343
Foreword Raymond D. Kent As humans try to understand themselves, one of the greatest fascinations—and most challenging problems—is to know how our brains create and use language. After decades of earnest study in a variety of disciplines (e.g., neurology, psychology, psycholinguistics, neurolinguistics, to name a few), the problem of the brain and language is now addressed especially by the vigorous interdisciplinary specialty of cognitive neuroscience. This specialty seeks to understand the neural systems that underlie cognitive processes, thereby taking into its intellectual grasp the dual complexities of neuroscience and cognition. In her extraordinary book, Vesna Mildner gives the reader a panoramic view of the progress that cognitive neuroscience has made in solving the brain–language problem. Mildner covers her topic in eight chapters that can be read in any order. Each chapter is a tightly organized universe of knowledge; taken together, the chapters are complementary in their contribution to the overall goal of the book. The first chapter addresses basic aspects of the development, structure, and functioning of the human central nervous system (CNS), arguably the most complexly organized system humans have ever tried to fathom. The author systematically identifies and describes the tissues and connections of the CNS, thereby laying the foundation for the succeeding chapters that consider the topics of sex differences, the history of neurolinguistics, research methods, models and theories of the central nervous system, lateralization and localization of functions, learning and memory, and—the culminating chapter—speech and language. The sweep of information is vast, but Mildner succeeds in locking the pieces together to give a unified view of the brain mechanisms of language. Science is a procession of technology, experiment, and theory. Mildner’s comprehensive review shows how these three facets of scientific progress have shaped the way we comprehend the neurological and cognitive bases of language. From early work that relied on “accidents of nature” (brain damage resulting in language disorders) to modern investigations using sophisticated imaging methods, the path to knowledge has been diligently pursued. The unveiling of the brain through methods such as functional magnetic resonance imaging and positron emission tomography has satisfied a scientific quest to depict the neural activity associated with specific types of language processing. Today we stand at a remarkable confluence of information, including behavioral experiments on normal language functioning, clinical descriptions of neurogenic speech and language disorders, and neuroimaging of language processes in the intact living brain. But the profound potential of this synthesis is difficult to realize because the knowledge is spread across a huge number of journals and books. Vesna Mildner offers us a precious gift of scholarship, as she distills the information from more than 600 references to capture the science of brain and language. xi
Preface This book is intended for those interested in speech and its neurophysiological basis: phoneticians, linguists, educators, speech therapists, psychologists, and any combination of cognitive and/or neuro- descriptions added. In order to get a comprehensive picture of speech production and perception, or representation of speech and language functions in the brain, it is usually necessary to go through page after page, actual and virtual, of texts on linguistics, psychology, anatomy, physiology, neuroscience, information theory, and other related areas. In most of them language is covered in one or at best a few very general chapters, with speech as a specific, but the most uniquely human means of communication, receiving even less attention and space. On the other hand, the books that focus on language do not have enough information on the neurophysiological bases of speech and language either with respect to production or perception. My intention was to make speech the central topic, and yet provide sufficient up-to-date information about the cortical representation of speech and language, and related topics (e.g., research methods, theories and models of speech production and perception, learning and memory). Data on clinical populations are given in parallel to studies of healthy subjects, because such comparisons can give a better understanding of intact and disordered speech and language functions. The book is organized into eight chapters. They do not have to be read in the order they are written. Each of them is independent and may be read at any time or skipped entirely if the reader feels that he or she is not interested in the particular topic or knows enough about it. However, to those who are just getting acquainted with the topic of the neurophysiological bases of speech and language I recommend starting with chapter 1 and reading on through to the last chapter. The first chapter is an overview of the development, structure, and functioning of the human central nervous system, particularly the brain. It is perhaps the most complex chapter with respect to terminology and the wealth of facts, but the information contained therein is necessary for a better understanding of the neurophysiological bases of speech and language. When introduced for the first time, each technical term (anatomical, physiological, evolutional, etc.) is given in English and Greek/Latin. Besides the sections on the development, structure, and organization of the central nervous system, the chapter includes sections on sensation and perception and on the neural bases of perception and production of speech. The latter section deals with hearing and the auditory cortex, with movement and speech production, and addresses the various ways in which speech perception and production are related. Chapter 2 is a brief account of differences between the sexes in neuroanatomy, development, and behavior. Awareness of these differences is important for a better understanding of the linguistic development and functioning of males and
xiii
xiv
The Cognitive Neuroscience of Human Communication
females, since these differences frequently become apparent in various aspects of speech/language disorders (e.g., aphasias and developmental dyslexia). In chapter 3, I present chronologically the major ideas, theories, and historical milestones in research on the mind–brain relationship (particularly with respect to speech and language). In addition to the well-known names (e.g., Broca and Wernicke), the chapter includes persons who have been frequently unjustly neglected in neurolinguistic literature in spite of their important contributions. The chapter sets the stage for the results of research that are discussed throughout the rest of the book, and that span the second half of the 20th century to the present. Chapter 4 is a review of research methods. It includes the descriptions, with the advantages and the drawbacks, of the techniques that are at present the methods of choice in clinical and behavioral studies (e.g., fMRI), as well as those that are for various reasons used less frequently but their results are available in the literature (e.g., cortical stimulation). The chapter includes a review of the studies of split-brain patients, cortical stimulation studies, radiological methods, electrophysiological methods, ultrasound and radioisotopic techniques, and the most frequent behavioral methods (e.g., dichotic listening, divided visual field, gating, priming, and Stroop). In chapter 5, I examine different models and theories—from the older, but still influential ones (e.g., Wernicke–Geschwind model) to the most recent that are based on modern technologies (e.g., neural networks). The chapter starts with the short description of the most important principles of the central nervous system functioning (e.g., hierarchical organization, parallel processing, plasticity, and localization of functions), which theories and models explain. Chapter 6 explains the key terms and dichotomies related to functional cerebral asymmetry (e.g., verbal–spatial, local–global, analytic–holistic), and also some less frequently mentioned ones (e.g., high vs. low frequencies, categorical vs. coordinate). It includes a section on developmental aspects of lateralization, within which the various aspects of asymmetry are considered: neuroanatomical asymmetry, motor asymmetry, asymmetry of the senses, and asymmetry in other species. There is also a section on the factors that affect functional asymmetry of the two hemispheres, and a section on the lateralization of functions, including cerebral representation of various functions. Chapter 7 deals with the different types of learning and memory, with particular emphasis on speech and language. The existing classifications of learning and memory types are discussed and are related to their neural substrates. There are sections on nervous system plasticity and critical periods, as important factors underlying the acquisition and learning of the first and all subsequent languages. Finally, chapter 8, albeit the last, is the main chapter of the book, and is as long as the rest of the book. It is subdivided into sections corresponding to different levels of speech and language functions, and includes sections on bilingualism and speech and language disorders. Here are some of the section titles: • Speech and Language Functions and Their Locations in the Brain • Anatomic Asymmetries and Lateralization of Speech and Language • Speech Production and Perception
Preface
• • • • • • • • • • • •
xv
Phonetics and Phonology Tone and Prosody Lexical Level and the Mental Lexicon Sentence Level—Semantics and Syntax Discourse and Pragmatics Reading, Writing, Calculating Is Speech Special? Language Specificities Bilingualism Speech and Language Disorders (e.g., Aphasia, Dyslexia) Motor Speech Disorders (e.g., Apraxia of Speech, Stuttering) Other Causes of Speech and Language Disorders (e.g., Epilepsy, RightHemisphere Damage).
The reference list contains more than 600 items and includes the most recent research as well as seminal titles. The glossary has almost 600 terms, which will be particularly helpful to the readers who wish to find more information on topics that are covered in the test. I felt that the book would read more easily if extensive definitions and additional explanations were included in the glossary rather than making frequent digressions in the text. Also, some terms are defined differently in different fields, and in those cases the discrepancies are pointed out. A comprehensive subject index and author index are included at the end. Relevant figures can be found throughout the text, but there is an added feature that makes the book more reader-friendly. In the appendix there are figures depicting the brain “geography” for easier navigation along the medial–lateral, dorsal–ventral, and other axes (Figure A.1). Brodmann’s areas with the cerebral lobes (Figure A.2), the lateral view of the brain with the most important gyri, sulci, and fissures (Figure A.3), the midsagittal view, including the most important brainstem and subcortical structures (Figure A.4), the limbic system (Figure A.5), and the coronal view with the basal ganglia (Figure A.6). Since many brain areas are mentioned in several places and contexts throughout the book, rather than leafing back and forth looking for the fixed page where the area was mentioned for the first time, or repeating the illustrations, the figures may be referred to at any point by turning to the appendix. Many friends and colleagues have contributed to the making of this book. First of all I’d like to thank Bill Hardcastle for getting me started and Ray Kent for thought-provoking questions. Special thanks go to Damir Horga, Nadja Runji´c;, and Meri Tadinac for carefully reading individual chapters and providing helpful suggestions and comments. I am immeasurably grateful to Dana Boatman for being with me every step of the way and paying attention to every little detail—from chapter organization to relevant references and choice of terms—as well as to the substance. She helped solve many dilemmas and suggested numerous improvements. Her words of encouragement have meant a lot. Jordan Bi´cani´c was in charge of all the figures. He even put his vacation on hold until they were all completed, and I am thankful that he could include work on this book in
xvi
The Cognitive Neuroscience of Human Communication
his busy schedule. Many thanks to Ivana Bedekovi´c, Irena Martinovi´c, Tamara ˘ Šveljo, and Marica Zivko for technical and moral support. I am grateful to Lawrence Erlbaum Associates and Taylor & Francis Group for giving me the opportunity to write about the topic that has intrigued me for more than a decade. Emily Wilkinson provided guidance and encouragement, and promptly responded to all my queries. Her help is greatly appreciated. I also with to thank Joy Simpson and Nadine Simms for their assistance and patience. Michele Dimont helped bring the manuscript to the final stage with much enthusiasm. Finally, I wish to thank my husband, Boris, for all his help, patience, support, and love. Naturally, it would be too pretentious to believe that this book has answers to all questions regarding speech and language. I hope that it will provide the curious with enough information to want to go on searching. Those who stumble upon this text by accident I hope will become interested. Most of all, I encourage readers to share my fascination with the brain, as well as with speech and language, as unique forms of human communication. —Vesna Mildner
1 Central Nervous System This chapter is an overview of the development, structure, and functioning of the central nervous system, with special emphasis on the brain. All areas that are discussed later, in the chapter on speech and language, are described and explained here, in addition to the structures that are essential for the understanding of the neurobiological basis of speech and language. More information and details, accompanied by excellent illustrations, may be found in a number of other sources (Drubach, 2000; Gazzaniga, Ivry, & Mangun, 2002; Kalat, 1995; Kolb & Whishaw, 1996; Pinel, 2005; Purves et al., 2001; Thompson, 1993; Webster, 1995). For easier reference and navigation through these descriptions, several figures are provided in the appendix. In Figure A.1 there are the major directions (axes): lateral—medial, dorsal—ventral, caudal—rostral, superior—inferior, and anterior—posterior. Brodmann’s areas and cortical lobes are shown in Figure A.2. The most frequently mentioned cortical structures are shown in Figures A.3 through A.6. These and other relevant figures are included in the text itself. At the end of this chapter there is a section on the neural bases of speech production and perception and their interrelatedness.
THE DEVELOPMENT OF THE CENTRAL NERVOUS SYSTEM Immediately after conception a multicellular blastula is formed, with three cell types: ectoderm, mesoderm, and endoderm. Bones and voluntary muscles will subsequently develop from mesodermal cells, and intestinal organs will develop from endodermal cells. The ectoderm will develop into the nervous system, skin, hair, eye lenses, and the inner ears. Two to 3 weeks after conception the neural plate develops on the dorsal side of the embryo, starting as an oval thickening within the ectoderm. The neural plate gradually elongates, with its sides rising and folding inward. Thus the neural groove is formed, developing eventually, when the folds merge, into the neural tube. By the end of the 4th week, three bubbles may be seen at the anterior end of the tube: the forebrain (prosencephalon), the midbrain (mesencephalon), and the hindbrain (rhombencephalon). The rest of the tube is elongated further and, keeping the same diameter, becomes the spinal cord (medulla spinalis). The forebrain will eventually become the cerebral cortex (cortex cerebri). During the 5th week the forebrain is divided into the diencephalon and the telencephalon. At the same time the hindbrain is divided into the metencephalon and myelencephalon. In approximately the 7th gestation week the telencephalon is transformed into cerebral hemispheres, the diencephalon into the thalamus and related structures, while the metencephalon develops 1
2
The Cognitive Neuroscience of Human Communication
into the cerebellum and the pons, and the myelencephalon becomes the medulla (medulla oblongata). During the transformation of the neural plate into the neural tube, the number of cells that will eventually develop into the nervous system is relatively constant—approximately 125,000. However, as soon as the neural tube is formed, their number rises quickly (proliferation). In humans that rate is about 250,000 neurons per minute. Proliferation varies in different parts of the neural tube with respect to timing and rate. In each species the cells in different parts of the tube proliferate in unique ways that are responsible for the species-specific folding patterns. The immature neurons that are formed during this process move to other areas (migration) in which they will undergo further differentiation. The process of migration determines the final destination of each neuron. The axons start to grow during migration and their growth progresses at the rate of 7 to 170 μm per hour (Kolb & Whishaw, 1996). Between the eighth and tenth week after conception the cortical plate is formed; it will eventually develop into the cortex. Major cortical areas can be distinguished as early as the end of the first trimester. At the beginning of the third month, the first primary fissures are distinguishable, for example, the one separating the cerebellum from the cerebrum. Between the 12th and the 15th week the so-called subplate zone is developed, which is important for the development of the cortex. At the peak of its development (between the 22nd and the 34th week) the subplate zone is responsible for the temporary organization and functioning. During that time the first regional distinctions appear in the cortex: around the 24th week the lateral (Sylvian) fissure and the central sulcus can be identified; secondary fissures appear around the 28th week; tertiary fissures start to form in the third trimester and their development extends into the postnatal period (Judaš & Kostovi´c, 1997; Kostovi´c, 1979; Pinel, 2005; Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Further migration is done in the insideout manner: the first cortical layer to be completed is the deepest one (sixth), followed by the fifth, and so on, to the first layer, or the one nearest to the surface. This means that the neurons that start migrating later have to pass through all the existing layers. During migration the neurons are grouped selectively (aggregation) and form principal cell masses, or layers, in the nervous system. In other words, aggregation is the phase in which the neurons, having completed the migration phase and reached the general area in which they will eventually function in the adult neural system, take their final positions with respect to other neurons, thus forming larger structures of the nervous system. The subsequent phase (differentiation) includes the development of the cell body, its axon and dendrites. In this phase, neurotransmitter specificity is established and synapses are formed (synaptogenesis). Although the first synapses occur as early as the end of the 8th week of pregnancy, the periods of intensive synaptogenesis fall between the 13th and the 16th week and between the 22nd and the 26th week (Judaš & Kostovi´c, 1997). The greatest synaptic density is reached in the first 15 months of life (Gazzaniga et al., 2002). In the normal nervous system development these processes are interconnected and are affected by intrinsic and environmental factors (Kostovi´c, 1979; Pinel, 2005; Spreen et al., 1984). In most cases the axons
Central Nervous System
3
immediately recognize the path they are supposed to take and select their targets precisely. It is believed that some kind of a molecular sense guides the axons. It is possible that the target releases the necessary molecular signals (Shatz, 1992). Some neurons emit chemical substances that attract particular axons, whereas others emit substances that reject them. Some neurons extend one fiber toward the surface and when the fiber ceases to grow, having reached the existing outer layer, the cell body travels along the fiber to the surface, thus participating in the formation of the cortex. The fiber then becomes the axon, projecting from the cell body (now in the cortex) back to the original place from which the neuron started. This results in the neuron eventually transmitting the information in the direction opposite to that of its growth (Thompson, 1993). The neurons whose axons do not establish synapses degenerate and die. The period of mass cell death (apoptosis) and the elimination of unnecessary neurons is a natural developmental process (Kalat, 1995). Owing to great redundancy, pathology may ensue only if the cell death exceeds the normal rate (Strange, 1995). The number of synapses that occur in the early postnatal period (up to the second year of life) gradually decreases (pruning) and the adult values are reached after puberty. Since these processes are the most pronounced in the association areas of the cortex, they are attributed to fine-tuning of associative and commissural connections in the subsequent period of intensive cognitive functions development (Judaš & Kostovi´c, 1997). Postmortem histological analyses of the human brain, as well as glucose metabolism measurements in vivo, have shown that in humans, the development and elimination of synapses peak earlier in the sensory and motor areas of the cortex than in the association cortex (Gazzaniga, Ivry, & Mangun, 2002). For example, the greatest synaptic density in the auditory cortex (in the temporal lobe) is reached around the third month of life as opposed to the frontal lobe association cortex, where it is reached about the 15th month (Huttenlocher & Dabholkar, 1997; after Gazzaniga et al., 2002). In newborns, glucose metabolism is highest in the sensory and motor cortical areas, in the hippocampus and in subcortical areas (thalamus, brainstem, and vermis of the cerebellum). Between the second and third month of life it is higher in the occipital and temporal lobes, in the primary visual cortex, and in basal ganglia and the cerebellum. Between the 6th and the 12th month it increases in the frontal lobes. Total glucose level rises continuously until the fourth year, when it evens out and remains practically unchanged until age 10. From then until approximately age 18 it gradually reaches the adult levels (Chugani, Phelps, & Mazziotta, 1987). Myelination starts in the fetal period and in most species goes on until well after birth. From the eighth to the ninth month of pregnancy brain mass increases rapidly from approximately 1.5 g to about 350 g, which is the average mass at birth (about 10% of total newborn’s weight). At the end of the first year, the brain mass is about 1,000 g. During the first 4 years of life it reaches about 80% of the adult brain mass—between 1,250 and 1,500 g. This increase is a result of the increase in size, complexity, and myelination—and not of a greater number of neurons (Kalat, 1995; Kostovi´c, 1979; Spreen, Tupper, Risser, Tuokko, & Edgell, 1984; Strange, 1995). Due to myelination and proliferation of glial cells, the brain
4
The Cognitive Neuroscience of Human Communication
volume increases considerably during the first 6 years of life. Although the white matter volume increases linearly with age and evenly in all areas, the gray matter volume increases nonlinearly and its rate varies from area to area (Gazzaniga et al., 2002). Brain growth is accompanied by the functional organization of the nervous system, which reflects its greater sensitivity and ability to react to environmental stimuli. One of the principal indicators of this greater sensitivity is the development of associative fibers and tracts; for example, increasing and more complex interconnectedness is considered a manifestation of information storage and processing. Neurophysiological changes occurring during the 1st year of life are manifested as greater electrical activity of the brain that can be detected by EEG and by measuring event-related potentials (ERPs; Kalat, 1995). Positron emission tomography (PET) has revealed that the thalamus and the brainstem are quite active by the fifth week postnatally, and that most of the cerebral cortex and the lateral part of the cerebellum are much more mature at 3 months than at 5 weeks. Very little activity has been recorded in the frontal lobes until the age of about 7.5 months (Kalat, 1995). Concurrent with many morphological and neurophysiological changes is the development of a number of abilities, such as language (Aitkin, 1990). In most general terms, all people have identical brain structure, but detailed organization is very different from one individual to the next due to genetic factors, developmental factors, and experience. Genetic material in the form of the DNA in the cell nucleus establishes the basis for the structural organization of the brain and the rules of cell functioning, but development and experience will give each individual brain its final form. Even the earliest experiences that we may not consciously remember leave a trace in our brain (Kolb & Whishaw, 1996). Changes in cortical layers are closely related to changes in connections, especially between the hemispheres. Their growth is slow and dependent on the maturation of the association cortex. Interhemispheric or neocortical connections (commissures) are large bundles of fibers that connect the major cortical parts of the two hemispheres. The largest commissure is the corpus callosum, which connects most cortical (homologous) areas of the two hemispheres. It is made up of about 200 million neurons. Its four major parts are the trunk, splenium (posterior part), genu (anterior part), and the rostrum (extending from the genu to the anterior commissure). The smaller anterior commissure connects the anterior parts of temporal lobes, and the hippocampal commissure connects the left and the right hippocampus. The hemispheres are also connected via massa intermedia, posterior commissure and the optic chiasm (Pinel, 2005). Most interhemispheric connections link the homotopic areas (the corresponding points in the two hemispheres; Spreen et al., 1984), but there are some heterotopic connections as well (Gazzaniga et al., 2002). Cortical areas where the medial part of the body is represented are the most densely connected (Kolb & Whishaw, 1996). It is believed that neocortical commissures transfer very subtle information from one hemisphere to the other and have an integrative function for the two halves of the body and the perceptual space. According to Kalat (1995), information reaching one hemisphere takes about 7 to 13 ms to cross over to the opposite one. Ringo, Doty, Demeter, &
Central Nervous System
5
Simard (1994), on the other hand, estimate the time of the transcortical transfer to be about 30 ms. Ivry and Robertson (1999) talk about several milliseconds. In their experiments on cats, Myers and Sperry (as cited in Pinel, 2005) have shown that the task of the corpus callosum is to transfer the learned information from one hemisphere to the other. The first commissures are established around the 50th day of gestation (anterior commissure). Callosal fibers establish the interhemispheric connections later and the process continues after birth until as late as age 10 (Kalat, 1995; Lassonde, Sauerwein, Chicoine, & Geoffroy, 1991). Corpus callosum of left-handers was found to be about 11% thicker than that of the right-handers, which was attributed to greater bilateral representation of functions (Kalat, 1995; Kolb & Whishaw, 1996). There is disagreement among authors considering the sex differences in callosal size (for more information, see chap. 2, this volume). Myelination of corpus callosum proceeds during postnatal development and it is one of the parts of the nervous system whose myelination begins and ends last. It is thought that the callosal evolution has an impact on hemispheric specialization (Gazzaniga et al., 2002). In Alzheimer’s patients, the area of corpus callosum, especially of its medial part (splenium), is significantly smaller than in healthy individuals (Lobaugh, McIntosh, Roy, Caldwell, & Black, 2000). After the age of 30 the brain mass gradually decreases and by the age of 75 it is approximately 100 g smaller (Kolb & Whishaw, 1996). Although the brains of people in their seventies have fewer neurons than the brains of younger people, in healthy elderly individuals the decrease is compensated for by the dendrites of the remaining neurons becoming longer and branching more (Kalat, 1995). Some recent studies have revealed that in rare cases and in a very limited way in some parts of the brain, particularly in the hippocampus and the olfactory bulb (bulbus olfactorius), a small number of neurons may develop after birth and during lifetime (Purves et al., 2001). However, whether these newly formed neurons have any function in the adult nervous system remains to be determined (Drubach, 2000; Gage, 2002; Gazzaniga et al., 2002; Gould, 2002).
STRUCTURE AND ORGANIZATION OF THE CENTRAL NERVOUS SYSTEM The nervous system consists of nerve cells—neurons—and glial cells (that will be discussed later). The neuron is a functional and structural unit of the brain. It consists of the cell body (soma) with the nucleus built from DNA, and other structures characteristic of cells in general, one or more dendrites, and one axon that ends with the presynaptic axon terminals (Figure 1.1). The neuron transmits information to other cells and receives information from them. The dendrites and the body receive information while the axon transmits information to other neurons. The space between neurons is filled with extracellular fluid so that in general they are not in direct contact. The size of the smallest neurons is approximately 7 to 8 μm, whereas the largest ones range in size between 120 and 150 μm (Judaš & Kostovi´c, 1997). Axons of some human neurons may be one meter or longer, whereas others do not
6
The Cognitive Neuroscience of Human Communication Dendrite
Cell membrane Soma (cell body) Nucleus
Myelin sheath Node of Ranvier
Dendritic spines
Axon hillock
Axon
Axon terminals
FIGURE 1.1. A stylized neuron.
exceed several tens or hundreds of micrometers. Most axons are between several millimeters and several centimeters long. The diameter of the thinnest axons is about 0.1 μm. At the end of each axon there are usually several smaller fibers that end with the terminal node. Each node synapses with another cell. A neuron may have a few short fibers or a huge number. The greater the number of dendrites, the greater the receptive ability of the neuron. In the cerebral cortex, many neurons’ dendrites are covered by literally thousands of little processes—dendritic spines. Since each one of them is a postsynaptic part of the synapse, the number of connections is greatly increased. These synapses are most probably excitatory. All neurons, from the simplest to the most complex organisms, rely on identical electrochemical mechanisms for information transmission. The considerable differences in neuronal organization, for example, in the patterns of their interconnections, are responsible for the functional differences that distinguish, for instance, the humans from other species. Neurons may be grouped into pathways or tracts (a simple series of neurons)—for example, the auditory pathway; into neuronal circles or networks; and into neuronal systems—for example, the auditory system. Each neuron may have connections to thousands of other neurons, which means that it may affect their activity. It can in turn be influenced by thousands of other neurons with excitatory or inhibitory results. There are no unnecessary or reserve neurons—each one has a function. Neuronal populations differ in size, shape, manner of information processing, and transmitters that they use to communicate with other neurons. Those that occupy neighboring positions in the brain and share common functions usually belong to the same population and have identical physical and functional properties. This principle is metaphorically referred to as “Neurons that fire together wire together.” This means that some
Central Nervous System
7
are specialized for visual information, others for auditory stimuli, and still others for emotional expressions. These functions are not interchangeable. However, in spite of such highly specialized properties, there are limited possibilities for the neurons neighboring those that have been injured to take over and assume the new function, different from their original one, which results in the neurofunctional reorganization of the entire affected area. This issue will be addressed in more detail in the context of plasticity in chapter 7, this volume. Motor (efferent) neurons have richly arborized dendrites, a large body, and a long myelinated axon. They send out their fibers from the nervous system toward the body (parts) and at their ends synapse with muscle fibers and gland cells. They control the activity of skeletal muscles, smooth muscles, and glands. They are controlled by several systems in the brain that are called motor systems. Sensory (afferent) neurons extend from the body to the brain. Their cell bodies are located along the spinal cord in groups. They are the so-called ganglia. One of their most important properties is selective detection and enhancement of particular stimulus features. The cerebral cortex is made up of several hundred different types of neurons. They are either pyramidal neurons (principal neurons) or extrapyramidal (interneurons). The principal neurons of a particular area are responsible for the transmission of the final information into other cerebral areas after the processing of incoming information. They are excitatory neurons and make up about 70% of all cortical neurons. They are rich in dendritic spines that contribute to the richness of connections; their axons are long and make projection, association, or commissural fibers. Numerous collateral branches (collaterals) of these axons are the greatest sources of excitatory postsynaptic potentials in the cerebral cortex (Judaš & Kostovi´c, 1997). Interneurons are mainly inhibitory and make up to 30% of all cortical neurons. Their dendrites have no dendritic spines; their axons are short and establish local connections. Neuronal feedback is essential for optimal brain functioning; the brain adjusts its activity on the basis of it. The number of neurons is greatly (perhaps tenfold) exceeded by the number of glial cells (glia). They make up about 50% of the total brain tissue volume (Judaš & Kostovi´c, 1997). They are not neural cells because they do not transmit information. Their function is not entirely clear, but their roles include absorbing substances in the brain that are not necessary or are excessive (e.g., at the synapses they often absorb excess neurotransmitters); after a brain injury proliferating in the location of neuron damage and removing cell debris, making the so-called glial scars; forming myelin sheaths; establishing the blood–brain barrier; guiding migrating neurons on the path to their final destinations, and so forth (Thompson, 1993; Judaš & Kostovi´c, 1997). Many types of glial cells communicate among themselves and with neurons as well. The neuron–glia–neuron loop is therefore considered to be a more precise description of a communication unit, rather than the simple neuron–neuron connection (Gazzaniga et al., 2002). The myelin sheath is basically fat. It enables faster propagation of the action potential along axons. There are interruptions in the sheath where the axon is in direct contact with the extracellular fluid, enabling the occurrence of action
8
The Cognitive Neuroscience of Human Communication
potentials. These interruptions are the so-called nodes of Ranvier (nodi Ranvieri). Myelinated axon segments between the two unmyelinated points are between 200 μm and 1 mm long. Their length depends on the axon type: the greater the axon diameter, the longer the myelinated segments. Consequently, longer myelinated segments, in other words, longer distances between the two nodes, will result in faster impulse conduction. The width of each node is about one μm. A synapse is a functional connection between two neurons or between a neuron and another target cell (e.g., muscle). It is the point at which the information in the form of a nerve impulse (signal) is transmitted. A tiny space, between 10 to 100 nanometers wide, separates the axon terminal of one cell from the body or a dendrite of another cell with which it communicates. That space is called the synaptic cleft. Synapses can be found only in nerve tissue, because they are formed only between neurons and their target cells. Synapses are functionally asymmetrical (polarized), which means that signal transmission is one-way only. Having said that, it is important to bear in mind the existence of the neuronal feedback—a process that enables communication among nerve cells. There are two kinds of synapses—chemical and electrical. Most synapses in the brain of mammals are chemical (Figure 1.2; the illustration shows synaptic transmission at a chemical synapse; adapted from Purves et al., 2001). They may be excitatory or inhibitory. Excitatory synapses increase the activity of the target cell, in other words, the probability of occurrence of action potential. Inhibitory synapses decrease target cell activity. The signals are transmitted by means of neurotransmitters that are released (provided that the threshold of activation has been reached) from the presynaptic neuron into the synaptic cleft, from which they are taken up by the corresponding receptors in the postsynaptic membrane. This transfer is very precise and takes less than 1 millisecond. Neurotransmitters are chemical substances that are produced in the presynaptic neuron and stored in the vesicles in the presynaptic axon terminal. About a hundred different kinds of neurotransmitters are known at present. Different neuronal populations produce and react to only one (or a very limited number) type of neurotransmitter. Apart from the excitatory and inhibitory neurotransmitters there are the so-called conditional neurotransmitters, whose activity is affected by the existence of another neurotransmitter or by the neuronal circuit activity (Gazzaniga et al., 2002). Activation of a single excitatory synapse in the neuron is not sufficient for it to fire: several excitatory synapses have to be activated simultaneously (spatial summation) in order to reach the threshold of action potential. Even simultaneous activation of several synapses may not always result in firing. In such cases these groups of synapses must be activated several times in a row, in short intervals (temporal summation). The same principles of spatial and temporal summation apply to inhibitory synapses. A normal neuron constantly integrates temporal and spatial pieces of information and “makes decisions” on whether to fire or not (neuronal integration). The moment of making a positive decision is the point of reaching the action potential threshold at the axon hillock, which is the result of domination of excitatory over inhibitory effects. It is also possible that a subliminal stimulus (i.e., the one that is not sufficient in itself to reach the action potential thresholds)
Central Nervous System
9
Presynaptic neuron Synaptic vesicle
Synaptic cleft
Postsynaptic neuron
Synaptic vesicle fusing Presynaptic membrane Synaptic cleft
Neurotransmitter
Postsynaptic membrane Postsynaptic neurotransmitter receptor
FIGURE 1.2. Chemical synapse; synaptic transmission. (From Purves et al., 2001. With permission.)
prepares the postsynaptic neuron for the arrival of another subliminal stimulus, which will (owing to that earlier stimulus) reach the action potential threshold. This phenomenon is called facilitation. It may be achieved by direct activation of excitatory synapses or by inactivation of inhibitory synapses (Judaš & Kostovi´c, 1997). Experience has a huge effect on the strengthening or weakening of synapses. As opposed to chemical synapses that exhibit a great deal of plasticity (i.e., may be changed in various ways with increased or decreased activity), electrical synapses are rigid and unchangeable. In electrical synapses there are no synaptic clefts between neurons: their membranes are in direct contact and their cytoplasms are connected through transmembranous channels (gap junction). As a consequence, such neurons have identical potentials, and electrical changes in one of them are immediately reflected on the other. At electrical synapses the transmission is very fast and it may be bidirectional (Purves et al., 2001). Learning and memory would be impossible in a nervous system that had only electrical synapses. However, some electrical synapses have been found in mammalian brains—mainly between cell bodies of neighboring neurons that share the same function.
10
The Cognitive Neuroscience of Human Communication
Nerve impulses are transmitted at the rate of about 1 to 100 meters per second. As it has been mentioned earlier, the rate at which the impulse travels through the axon depends on its diameter (the greater the diameter, the faster the transmission) and on whether it is myelinated or not (the impulse travels faster through myelinated fibers; Sternberg, 2003). The firing rate generally does not exceed 100 times in a second, although some neurons may fire at the very high rate of 1,000 times in a second. A common feature of all neurons is that they function on the all-or-none principle. This means that the action potential will either occur or not. The messages differ in the action potential frequency and the timing of impulses (Ferster & Spruston, 1995; Kalat, 1995). However, neurons may also determine the content of the message that is being transmitted by varying, among other things, the type, quantity, and rate of neurotransmitter release (Drubach, 2000). Nerves are bundles of nerve fibers. The nerves transmitting the information toward the central nervous system and from it are peripheral nerves. They transmit impulses from the periphery toward the center (sensory fibers, afferent pathways), from the center toward the periphery (motor fibers, efferent pathways), or are positioned between sensory and motor fibers (interneurons). Somatic nerves establish connections with the voluntary skeletal muscles and sense organs. Autonomic nerves are connected with internal organs and glands involved in autonomic aspects of reactions, usually related to emotional behavior (crying, perspiration, some activities of the heart and stomach). All organs are controlled by the sympathetic part of the autonomic nervous system, and some are controlled both by the sympathetic and parasympathetic segment. Peripheral nerves are cranial or spinal. Cranial nerves (nervi craniales) transmit sensory information from the face and head, and commands for motor control over face and head movements. This means that their functions are sensory, motor, or combined. There are 12 pairs of cranial nerves, seven of which are essential for speech production (V through XII). They are numerated from the anterior to the posterior part of the brain: I II
The olfactory nerve (fila olfactoria) transmits olfactory information from the nose to the brain (into the telencephalon). The optic nerve (nervus opticus), actually a part of the visual pathway, transmits visual information from the eyes (it is directly connected to the diencephalon). Nerves III through XII are directly connected to the brainstem:
III IV V
The occulomotor nerve (nervus occulomotorius). The trochlear nerve (nervus trochlearis) and VI—the abducens nerve (nervus abducens) are responsible for eye muscles. The trigeminal nerve (nervus trigeminus) transmits sensory information from the skin of the face and head and innervates the jaw muscles and tensor tympani muscle in the middle ear.
Central Nervous System
11
VII The facial nerve (nervus facialis) has afferent and efferent connections with parts of the face, ear (outer ear, stapedial muscle), tongue, and larynx. VIII The vestibulocochlear nerve (nervus statoacusticus or nervus vestibulocochleris) transmits information about sounds from the ears and about vestibular sense from vestibular part of the inner ear. Some efferent pathways related to that nerve have been found as well (Kent & Tjaden, 1997). IX The glossopharyngeal nerve (nervus glossopharyngeus) innervates laryngeal muscles that play an important role in the process of swallowing; it also transmits the sense of taste and sensory information from the outer ear. X The vagus nerve (nervus vagus) is important for the autonomic (parasympathetic) control of the heart and other internal organs; it innervates laryngeal and pharyngeal muscles important for phonation; and it transmits the sensations from the outer ear and the sense of taste from the area around epiglottis. XI The accessory nerve (nervus accessorius) innervates neck and shoulder muscles as well as the muscles controlling the soft palate. XII The hypoglossal nerve (nervus hypoglossus) innervates tongue muscles. There are 31 pairs of spinal nerves (nervi spinales). They protrude from the spinal column and innervate the muscles with their motor parts in the anterior portions (they come out from ventral roots). Or they receive afferent information with their sensory parts in the posterior portions (these enter the dorsal roots), to forward it through the spinal cord to the thalamus and the cerebral cortex. The spinal cord (medulla spinalis) is a tube-like structure that extends downward from the brainstem and connects the brain with the parts of the body below the neckline. It is a major thruway for (a) information from the cerebral cortex and other brain structures that control body movements going toward motor neurons (and indirectly through them to the muscles) and toward all the organs in the body (autonomic nervous system), and for (b) information from all receptors, including proprioceptive, and information from some peripheral organs to the brain. The spinal cord is also responsible for reflex muscle and autonomic responses to stimuli. On their way from the brain to the periphery the axons cross over from one side of the tract to the other. Consequently, the motor control of the right side of the body is situated in the left cerebral hemisphere, whereas the right hemisphere controls the left side of the body. Information from the periphery also projects to the hemisphere contralateral to the stimulated side (Drubach, 2000). The brain (encephalon) is a part of the central nervous system (together with the spinal cord). The principal parts of the brain are the following: brainstem, cerebellum, and cerebrum (Figure 1.3). The brainstem (truncus cerebri or truncus encephalicus) consists of the medulla (medulla oblongata; myelencephalon), pons (pons; metencephalon) and the midbrain (mesencephalon). It connects the cerebrum and the cerebellum with the spinal cord, but it also has functions of its own (Drubach, 2000; Judaš & Kostovi´c, 1997; Purves et al., 2001; Webster, 1995). In it there are centers, the
12
The Cognitive Neuroscience of Human Communication Cingulate sulcus Central sulcus Cingulate gyrus
Precuneus
Fornix
Parieto-occipital sulcus Corpus callosum
Thalamus
Superior colliculus Midbrain
Hypothalamus Optic chiasm
Pons
Pituitary Cerebellum
Medulla
Brainstem
Inferior colliculus
Anterior commissure
Spinal cord
FIGURE 1.3. Midsagittal view of the brain.
so-called nuclei, made up of groups of neurons, which regulate body temperature; cardiac, respiratory, and gastrointestinal systems; blood vessels; and consciousness. Among other things, they also control the responses to visual and auditory stimuli, movement and, to some extent, wakefulness and sleep. It is believed that motor control of speech stems directly from the basic centers for swallowing and breathing control. Afferent (ascending, sensory) and efferent (descending, motor) pathways go through the brainstem. Reticular formation (formatio reticularis) is an important part of the brainstem. It is a heterogeneous set of functionally very different structures (Judaš & Kostovi´c, 1997) through which pass almost all sensory and motor pathways. It is connected with other nuclei in the brainstem, cerebellum, diencephalon and the cerebrum. Injuries to the brainstem may cause paralysis, loss of sensation and/or control of corresponding functions and consciousness, coma, and even death. The cerebellum is located in the posterior part of the cranial cavity. Its main parts are the cortex, subcortical white matter, and subcortical nuclei. It is one of the philogenetically oldest brain structures. Its surface is convoluted, with fissures, sulci, and gyri much more densely folded than in the cerebrum. It has two hemispheres and a medial part (vermis). The cerebellar cortex is functionally organized in three parts. Various functions are differentiated by specific input– output neuronal connections. On average, its mass is about 145 g. Its cortex is about 1 to 1.5 mm thick and consists of three cellular layers. Despite its relatively small size in proportion to the total brain volume (10%), it contains more than 50% of all neurons of the brain (Judaš & Kostovi´c, 1997). There is a threepronged connection (cerebellar penduncles) with the rest of the nervous system. Initiation of a voluntary movement (e.g., reaching for an object) will at first reveal a neural activity in the cerebellum (and in the basal ganglia), followed by the activity in the motor cortex of the cerebrum, that has turned out to be responsible for precise performance of fine voluntary movements but not for their initiation
Central Nervous System
13
(Thompson, 1993). The cerebellum receives a complex set of sensory information from most modalities: vestibular and auditory, muscles, joints, skin, and the eyes. It is the central place for proprioceptive information. Information related to proprioception, motor planning, and vestibular sense is integrated in the cerebellar cortex. The number of afferent fibers by far exceeds the number of efferent ones; in humans the ratio is about 40 to 1 (Judaš & Kostovi´c, 1997). This means that the cerebellum is an important place for integration and processing of input information before transmitting the output signal to other parts of the brain. On the basis of continuous inflow of sensory information from the periphery, the cerebellum coordinates and smooths out activity of the muscles, and in cooperation with the vestibular system coordinates head movements and body position with all other activities of the body. In other words, it regulates the speed, range, force, and the orientation of movements (Webster, 1995). It seems that the cerebellum plays an important role in learning, particularly in learning and remembering skilled movements, but also in nonmotor learning and numerous cognitive processes (Drubach, 2000). Injuries to the cerebellum may cause clumsiness (particularly purposeful hand movements), balance difficulties, decreased muscle tone, incomprehensible speech, imprecise eye movements, and impaired planning and timing of activities (Drubach, 2000; Kalat, 1995). It might be said that the main role of the cerebellum is to perform temporal calculations that may be used in various perceptive and motor functions, including speech and language. For example, estimating the duration of a sound is impaired in individuals with cerebellar injuries (Gazzaniga et al., 2002; Kent & Tjaden, 1997). As opposed to cerebral injuries, cerebellar injuries cause impairments on the ipsilateral side of the body (due to double crossing of pathways). Body parts are mapped onto the surface of the cerebellar cortex, similarly to the representations found in the cerebral cortex, but due to the lack of association or commissural fibers, each area receives a separate afferent projection and acts as a separate functional unit (Judaš & Kostovi´c, 1997). It is interesting to note that the cytoarchitectonic organization of the cerebellum is identical in all mammals—from mouse to human. The cerebellum and the motor cortex of the cerebrum constitute a constantly active movement control system (Thompson, 1993). The diencephalon is located between the brainstem and the cerebral hemispheres. Its principal parts are the thalamus, hypothalamus, epithalamus, and the subthalamus. Thalamus (originating from the Greek word thalamos, meaning bed or bedroom) is a structure almost in the very center of the brain, where numerous fibers synapse and cross. The visual and the auditory signals pass through it and it is also involved in movement control. It has connections with the cerebellum and the basal ganglia. Messages from the limbic system (of which the hypothalamus and epithalamus are important parts) are relayed to the thalamus. The thalamus integrates and interprets signals prior to forwarding them to other parts of the brain. It also plays an important role in memory. With the exception of olfactory information, all stimuli are processed in thalamic nuclei (seven groups in all) before reaching the cerebral cortex. Primary sensory and motor cortical areas receive direct projections from thalamic nuclei. Below the
14
The Cognitive Neuroscience of Human Communication
thalamus are the subthalamus and hypothalamus, responsible for emotions, basic body functions, body temperature, pituitary gland activity, hormone release, food and liquid intake, sexual behavior, and circadian rhythms. Below the posterior part of the thalamus (pulvinar) there is the lateral geniculate complex (corpus geniculatum laterale) that is a part of the visual system, and the medial geniculate complex (corpus geniculatum mediale) that is a part of the auditory system. Recent studies stress the importance of subcortical structures, particularly thalamus and hypothalamus for higher cognitive functions, including language and speech. The hypothalamus is believed to have a role in memory formation (Kent & Tjaden, 1997). Thalamus attracts attention and directs it to verbal information, recall, and so forth. Its role is to enhance and emphasize the information to which the attention is directed at the moment. However, the consequence of thalamic injuries on naming, word finding, arithmetic, verbal short-term memory, and fluency are most frequently transient and short-lived, as opposed to cortical injuries (Bradshaw & Nettleton, 1983). Thalamic injuries typically cause loss of sensation, motor disorders, and consciousness problems. The cerebrum consists of white (about 39% of volume) and gray (about 61% of volume) matter. There are about 1011 to 1012 neurons in the cerebrum, of which about 50 billion are directly involved in information processing (Kolb & Whishaw, 1996; Strange, 1995). Each neuron makes about 1,000 to 3,000 connections with other neurons, which means that the number of connections is huge—about 1014 (Churchland, 1988). In the cerebral cortex there are about 10 billion neurons. The mass of the adult brain ranges from 1,100 to 1,700 g. On average, adult male brains weigh approximately 1,450 g, and adult female brains about 1,300 g, which makes about 2.5% of total body mass. The smaller mass of the female brain does not imply poorer abilities. First of all, smaller mass is compensated for by richer connections, and second, it seems that the ratio of brain mass to total body mass is a better indicator of brain development than its absolute mass (Drubach, 2000). For example, whales and elephants have larger brains than humans, but the density of their cortical cells is most likely smaller. The differences in cortical size are also associated with the differences in the cerebral cortex circuits (Hill & Walsh, 2005). In any case, of all the species, humans have the largest brain with respect to their body size. Note also that the newborn’s ratio of brain mass to body mass is greater than in the adult (Sternberg, 2003). For as yet unknown reasons, in the past 3 million years there has been an explosion in the size of the human brain, unrecorded in any other species. This sudden growth is mostly attributable to the considerable enlargement of the cortex (Thompson, 1993). The brain is immersed in the cerebrospinal fluid and protected by the firm bony shell of the skull and three tissue layers (meninges). It is connected to the rest of the body (i.e., to the peripheral nervous system) by means of 12 pairs of cranial nerves. The connection with the spinal cord is realized through rich nerve connections (descending efferent and ascending afferent pathways; Judaš & Kostovi´c, 1997). It is generally believed that brains of exceptional people do not differ from those of average individuals with respect to mass, structure, and functioning. However, there is some evidence obtained by magnetic resonance imaging that the
Central Nervous System
15
correlation between intelligence and brain size is about 0.35 (Kalat, 1995). Moreover, by comparing Einstein’s brain with several dozen brains of average people Wittleson found considerable differences in the position of Sylvian fissure as well as in the area and thickness of the inferior temporal lobe (Gazzaniga et al., 2002). Human brains differ from the brains of other primates mainly in the richness of their associative (corticocortical) and projection (subcortical) connections and in quantitatively different organization. For example, the prefrontal area is twice as large as that in other primates, whereas the motor, olfactory, and visual areas are smaller (Kalat, 1995; Kolb & Whishaw, 1996). Philogenetic studies have shown that the proportions of various functional areas change during the development of the species. White matter consists of bundles of myelinated neuron axons (myelin is responsible for its white color) that connect the two hemispheres (commissural fibers), the cerebral cortex with lower parts of the nervous system (afferent and efferent projection fibers), or that connect various parts of the same hemisphere (short and long association fibers). Deep within the white matter are the ventricles (ventriculi; Judaš & Kostovi´c, 1997). Gray matter is made of the cell bodies and their dendrites. It is found on the surface (cortex cerebri, cortex) and deep within the brain separated from the cortex by white matter (basal ganglia; Webster, 1995). The area of the convoluted surface layer is about 2,200 square cm and its thickness varies between 1.5 and 4.5 mm. The fissures (fissurae) and sulci divide the surface into lobes (lobi), lobules (lobuli), and gyri (Judaš & Kostovi´c, 1997). Most of the cortex is organized into six layers of cells that have ontogenetically developed from the inside out and make up the neocortex. This term is synonymous with the term isocortex, reflecting the fact that each part of the adult cerebral cortex developed from the same developmental base, and makes up almost 90% of the total cerebral cortex (Judaš & Kostovi´c, 1997). During development, in a smaller part of the isocortex the number of layers either decreases or increases. In evolutionarily older parts of the cortex (allocortex) there are commonly fewer than six layers: two in the paleocortex, three in the archicortex, and five in the mesocortex. These parts make up the limbic system. The five basic functional groups of cortical areas to the largest extent correspond to the basic types of cerebral cortex. They are: (a) primary sensory and motor areas; (b) unimodal association areas; (c) heteromodal association areas; (d) limbic areas; and (e) paralimbic areas. The term cortex is commonly used to refer to neocortex. The cells vary in form, size, and distribution across areas, so cytoarchitectonically we talk about nuclei, layers (laminae), areas, and regions that have different cell structure (Judaš & Kostovi´c, 1997). This diversity is partly responsible for the complexity of brain structure. Kolb and Whishaw (1996) have summarized the principles of cortical organization: 1. The cortex is made of many different types of neurons that are organized into six layers (laminae). 2. The cortex is organized into functional columns (columnae), which means that the neurons that share similar functions are grouped into
16
The Cognitive Neuroscience of Human Communication
3. 4. 5. 6.
columns that stretch throughout the cortex, making the cortical column the principal organizational and functional unit. There are multiple representations of sensory and motor functions in the cortex. These functions are plastic. Cortical activity is influenced by feedback from several regions of the forebrain (e.g., from the limbic system or the basal ganglia). The cortex operates on the principles of hierarchical and parallel information processing.
The cerebrum is divided by the longitudinal fissure (fissura longitudinalis cerebri) into two hemispheres that are connected by three large systems of commissural fibers: corpus callosum, anterior commissure (commissura anterior), and the hippocampal commissure (commissura hippocampi). Corpus callosum is the largest and the most important interhemispheric connection. Each hemisphere is divided morphologically into four lobes clearly delimited by anatomical landmarks (fissures and sulci): (a) central sulcus, which is sometimes referred to as Rolandic fissure (fissura centralis Rolandi); (b) lateral fissure, also called the Sylvian fissure (fissura lateralis cerebri Sylvii); and (c) the parieto–occipital fissure (sulcus) or incision (fissura s. incisura parietooccipitalis). The sulci and fissures alternate with gyri. At the bottom of the Sylvian fissure there is the insula that it sometimes referred to as the fifth lobe (Drubach, 2000; Judaš & Kostovi´c, 1997; Figure 1.4). At the beginning of the 20th century Corbinian Brodmann used different staining techniques on samples of brain tissue to establish a system of cell types with respect to their structure (cytoarchitecture). He drew brain sketches using different symbols to represent cell groups differing in shape, density, and laminar organization, assuming that cells that have the same or similar structure perform Central sulcus Precentral gyrus Postcentral gyrus Superior frontal gyrus Superior parietal lobule Middle frontal Supramarginal gyrus gyrus Occipital gyri
Pars opercularis + Pars triangularis
Parieto-occipital sulcus Angular gyrus
Inferior frontal gyrus Orbital gyri Lateral fissure Temporal pole Superior temporal gyrus
Pre-occipital notch
Middle temporal gyrus Inferior temporal gyrus
FIGURE 1.4. Lateral view of the brain.
Central Nervous System
17
the same or similar functions. This yielded about 50 cytoarchitectonically relatively homogenous areas that are called Brodmann’s areas and are marked by Arabic numerals (Figure 1.5). The frontal lobe (lobus frontalis) is anterior to the central sulcus and above the lateral fissure. It is the seat of primary and secondary motor areas. The primary motor area corresponds to Brodmann’s area 4. The secondary motor area includes premotor cortex and the supplementary motor area (dorsolateral and medial parts of Brodmann’s area 6, respectively), Brodmann’s areas 8, 44, and 45, and the posterior cingulate area. Each location in the primary motor areas controls a particular group of muscles on the opposite side of the body. Secondary motor areas are anterior to the primary ones. Each of them controls several primary centers and they are responsible for complex movements and voluntary muscle control. In the language-dominant hemisphere (usually left, but see chap. 8, this volume, for discussion), in Brodmann’s area 44 and probably 45, there are centers that control speech (Broca’s area) and writing. The prefrontal region is responsible for memory storage and retrieval, ethical attitudes, decision making, and psychological makeup of a person (Judaš & Kostovi´c, 1997). In humans, this region constitutes about half of the entire frontal lobe. It is connected with almost all other parts of Parietal lobe
Frontal lobe 4 3
6 8 46
10
7
2 19
40
9 45 44
43
47 11
5
1
52 38
22
39
41
18 17
42
Occipital lobe
37
21
Insula Temporal lobe
20
Frontal lobe
4
6
31 2
8 9
7 31
24
23
32 33
10
Parietal lobe
5
30
26 27 29
11
18
25 38
19
17 34 28
35 36
37
19
Occipital lobe
18
20
Temporal lobe
FIGURE 1.5. Brodmann’s areas in lateral and midsagittal view, including the four lobes.
18
The Cognitive Neuroscience of Human Communication
the brain and receives information from all sensory areas, memory and emotional stores (Kalat, 1995; Webster, 1995). The prefrontal cortex sends projections into all areas from which it receives them, into premotor and motor regions in both hemispheres. The motor cortex is considered to be a place where a multitude of signals involved in initiating and shaping motor control from other parts of the cortex and deeper levels, such as basal ganglia or the cerebellum, are integrated (McKhann, 1994). It seems that, in addition to the neurons in the primary motor cortex, there are a number of circuits, including the cerebellum, basal ganglia, and the thalamus, that generate motor activity. Consequently, it is logical that the motor aspect of language relies on similar circuits. Almost all types of behavior involve both frontal lobes, so in most tasks, with the exception of higher cognitive functions, unilateral brain injury will have negligible consequences. Frontal lobes are crucial in learning a new task that requires active control. Once the activity has become routine other parts of the brain may assume control (Lieberman, 1991). Raichle (1994) proposed that the parts of the nervous system that are involved in learning certain motor patterns are not the same ones that are used for performing the once learned patterns. Functional magnetic resonance imaging (fMRI) data revealed increased activity in the supplementary motor area and lateral premotor cortex after initial task presentation—when the planning of movement starts. As planning turns into execution the activity shifts toward the more posterior regions, and as the movement becomes more complex, the structures outside the primary motor cortex are activated (Gazzaniga et al., 2002). Some of the manifestations of frontal lobe injuries are impaired motor functions, ignoring of social conventions, inflexible and unorganized behavior, inability to correct errors, perseveration, poor temporal memory, poor egocentric orientation, changed social and sexual behavior, and disorders related to damage of face representations, including language and speech or some of their segments (Kolb & Whishaw, 1996). Positron emission tomography (PET) data have revealed that frontal lobes are activated during internal generation of stimuli (by the subject), as opposed to external stimuli, which has led to the conclusion that they probably play a part in conscious distinction between actual and imagined stimuli (Drubach, 2000). During evolution, frontal lobes of humans have undergone enormous enlargement, particularly in the anterior portions. This enlargement is related to the higher cognitive abilities characteristic of humans (Gazzaniga et al., 2002). In fact, there seems to be a correlation between evolutionary patterns and gene function in humans. Characterization of genes for neurological disorders (such as mental retardation, autism, and dyslexia) that affect intelligence, social organization, and higher order language will hopefully shed more light on human evolutionary history (Hill & Walsh, 2005). The parietal lobe (lobus parietalis) is bordered anteriorly by the central sulcus, posteriorly by the parieto–occipital fissure (sulcus), and laterally by the Sylvian (lateral) fissure. Immediately posterior to the central sulcus there are primary somesthetic centers that receive information from sensory organs (proprioception, touch, pressure, temperature, pain; Brodmann’s areas 1, 2, and 3). In the postcentral gyrus there are four parallel representations of the body (Kalat,
Central Nervous System
19
1995). A part of that lobe is the angular gyrus (gyrus angularis), which plays a very important role in word reading and arithmetic (in the left hemisphere). The inferior parietal lobe is involved in writing, which makes that lobe (together with the temporal lobe) essential for language processing and comprehension. That lobe is also important for orientation on one’s own body, particularly with respect to the left–right orientation. The disorders that occur as a consequence of injury to this lobe (apraxia, tactile agnosia, alexia, agraphia, acalculia, autotopagnosia) suggest that it is important for secondary processing of input information (i.e., for coordination of input information from sense organs and output commands to the muscles). It is particularly important for the association and coordination of visual and spatial information (Kalat, 1995). This lobe also seems to be the seat of the short-term/working memory (Kolb & Whishaw, 1996; for more discussion on types of memory see chapter 7, this volume). The temporal lobe (lobus temporalis) is inferior to the lateral (Sylvian) fissure, and extends posteriorly to the parieto–occipital fissure (sulcus). When the lateral fissure is pulled open one to three Heschl’s gyri (gyri temporales transversi Heschl) are revealed. This area is the seat of cortical representation of the sense of hearing—the auditory cortex (Brodmann’s areas 41 and 42). Immediately posterior to Heschl’s gyri is a relatively flat area, the so-called planum temporale. The posterior part of the superior temporal gyrus (Brodmann’s area 22), commonly in the left hemisphere, is the seat of secondary processing of auditory speech stimuli—the center for speech perception (Wernicke’s area). Although processing of music stimuli has been associated with right-hemisphere function, there is evidence that hemispheric differences are dependent on proficiency in music (Ivry & Robertson, 1999; Pinel, 2005). Amusia patients reveal that the anterior parts of the superior temporal gyri are responsible for music production and processing (Gazzaniga et al., 2002; Grbavac, 1992). The cortical representation of vestibular function is also in the superior temporal gyrus. The temporal lobe is involved in complex aspects of visual information processing, for example, in face recognition (Kalat, 1995). The temporal lobe in the left hemisphere is the seat of verbal long-term memory (recollection of stories, word lists, etc., regardless of modality of their presentation). The temporal lobe in the right hemisphere is the seat of nonverbal long-term memory (geometrical drawings, faces, tunes etc.; Kolb & Whishaw, 1996). The temporal lobe is also the seat of learning and memory functions that require conscious effort—declarative or explicit learning (Kandel & Hawkins, 1992). Injuries to the temporal lobe cause disorders in stimulus categorization, and Kolb and Whishaw (1996) have grouped the consequences of such disorders into eight groups: 1. 2. 3. 4. 5.
Disorders of auditory sense and perception. Disorders of selective attention in the auditory and visual modality. Disorders of visual perception. Disorders in the organization and categorization of verbal material. Disorders in language comprehension, including the inability to use context.
20
The Cognitive Neuroscience of Human Communication
6. Disorders in visual and auditory long-term memory. 7. Personality and affective behavior changes (motivation, fear). 8. Changes in sexual behavior. Obviously, contribution of the frontal, parietal, and temporal lobes to all of the above processes is additive and probably hierarchically organized. The fact that similar types of behavior may occur after injuries to different areas supports the claims that the same cognitive processes may be disordered in different ways. These cognitive processes are based on joint activities of large areas of neocortical and subcortical tissues, and may therefore be impaired as a consequence of functional disorders or injuries in any of the involved regions (Kolb & Whishaw, 1996). The occipital lobe (lobus occipitalis) is the area posterior to the parieto– occipital fissure (sulcus). It is the seat of the primary visual area (Brodmann’s area 17). Next to that area are Brodmann’s areas 18 and 19, which are responsible for secondary visual processing. Due to its striped appearance the primary visual area is also known as the striate cortex or the striate area (area striata). The neighboring areas are called the extrastriate visual area (area extrastriata) (Judaš & Kostovi´c, 1997). The occipital lobe is exclusively responsible for visual information processing (Webster, 1995). Information from the left visual field reaches the right hemisphere, and information from the right visual field travels to the left hemisphere, but almost all regions of the visual cortex are reached by the information from the thalamus and from the opposite hemisphere (Lomber & Payne, 2002). The location of an injury will determine which visual field will be blind. Information about different visual attributes is not stored in one place. The cells responsible for visual perception are highly specialized: some are activated only by vertical lines, others by horizontal lines, whereas others are sensitive to color or specific forms, movements of particular speed, and so forth. Each of the cells contributes to the overall mental image. Apart from being highly specialized, the cells are hierarchically determined, so that the first line processes only the simplest visual data, such as contrast, the second line processes shapes, and the following layer interprets them. In this way, each subsequent layer provides additional data to the mental image until the object is recognized (Drubach, 2000). The processing is not exclusively one way, from periphery to the visual cortex (bottom up); there are also feedback projections from the higher order visual areas that contribute to the analysis of the basic properties of the response and structural properties in the primary visual cortex (top down). In other words, processing of any visual stimulus is actually based on interaction among several cortical areas (networks) at different hierarchical levels (Galuske, Schmidt, Goebel, Lomber, & Payne, 2002). One path goes from the striate area through the extrastriate area into the temporal lobe (ventral pathway) and carries information about the properties and the appearance of the stimulus. This is the so-called what pathway, responsible for identification and discrimination of the stimulus. The other path goes into the parietal lobe (dorsal pathway) and carries information about the movement and spatial position of the object. This is the so-called where pathway. Knowledge about the object is located in a distributed cortical
Central Nervous System
21
system, so that the information about particular features is stored in the vicinity of the cortical areas that participate in the perception of these features (Ungerleider, 1995). A person suffering from cortical blindness has normal peripheral vision, but cannot perceive patterns and has no awareness of visual information due to an organic lesion in the visual cortex. Other disorders of visual perception (agnosia) may take the form of impaired ability to recognize colors, faces, objects, depth, movements, and so forth (Drubach, 2000; Kalat, 1995). The insula got its name from the fact that it is separated from the surrounding areas by a circular sulcus. It is located at the bottom of the Sylvian fissure and covered by parts of the parietal, frontal, and temporal lobes. These parts that hide the insula are called opercula (plural of operculum—Latin for lid), and depending on the lobe they belong to, are called the frontal operculum (operculum frontale), the fronto–parietal operculum (operculum frontoparietale), and the temporal operculum (operculum temporale; Judaš & Kostovi´c, 1997). The insula is considered to be an important structure for speech, and there has been some evidence that the insula in the dominant hemisphere, rather than Broca’s area, might be the seat of speech motor planning (Dronkers, 1996, 2000; Duffau, Capelle, Lopes, Faillot, & Bitar, 2000). The anterior portion of the insula is active during processing and integration of autonomic and body information. Its posterior part is connected to other neocortical areas; the connections with the cortical and subcortical structures, particularly with the thalamus and basal ganglia, reveal its importance for somatosensory, vestibular, and motor integration. It is an integrative multimodal association area for information arriving from different senses. The insula plays an important part in the cardiovascular, gastrointestinal, vestibular, olfactory, gustatory, visual, auditory, somatosensory, and motor processes. It seems that it plays a part in conditioned learning, affective and emotional components of nociception (perception of pain), stress-invoked immunosuppression, mood stability, sleep, and language (Flynn, Benson, & Ardila, 1999). The association cortex comprises parts of the cerebral cortex that receive data from several modalities, and thus its role is integrative rather than exclusively motor or sensory, which is crucial for higher mental processes. For example, the association cortex at the border between the parietal, temporal, and occipital lobes of the left hemisphere is essential for successful processing of language data. Although for a long time the prevalent opinion had been that most parts of the neocortex are associative, in the past 15 years it has become increasingly clear that the cortex is mainly sensory and motor, and that complex brains do not develop by expansion of the association cortex but rather through the increase of sensory and motor areas and connections among them (Kaas, as cited in Gazzaniga et al., 2002). The limbic system lies along the corpus callosum (Latin word limbus means borderline). It is related to biological rhythm, sexual behavior, feelings of fear, anger, and motivation (Figure 1.6). In other words, the limbic system controls and processes emotions, and manages endocrine and autonomic systems. Amygdala are particularly important for the regulation of drives, affective and motivational states, and autonomic and endocrine functions (Judaš & Kostovi´c, 1997; Sternberg,
22
The Cognitive Neuroscience of Human Communication
Corpus callosum Hypothalamic nuclei
Cingulate gyrus
Anterior thalamic nucleus
Fornix Olfactory bulb
Mammilary body
Olfactory tract Amygdaloid body
Parahippocampal gyrus Hippocampus
FIGURE 1.6.
The limbic system.
2003). During evolution some parts of the limbic system (e.g., the hippocampus) have assumed other functions as well. It is believed that, in higher animals, the hippocampus is one of the key structures in learning and memory. The hippocampus may host different types of information at the same time and one of its major roles is integrating different details or elements of episodic memory traces and coding current experience, to be subsequently stored in memory (Hampson & Deadwyler, 2002; Payne, Jacobs, Hardt, Lopez, & Nadel, 2002). More on the hippocampus may be found in chapter 7. Within the limbic system the hypothalamus is the key passage for different neuronal circuits (Judaš & Kostovi´c, 1997). The cingulate gyrus (gyrus cinguli) is located immediately above the corpus callosum. Physical and mental emotional expressions are integrated in the hypothalamus and in the insula, which explains the fact that each of these two types of emotions may have manifestations characteristic of the other (Drubach, 2000). Basal ganglia are located deep within the cerebrum (Figure 1.7). This structure is actually a group of nuclei (putamen, globus pallidus, nucleus caudatus, substantia nigra, subthalamus) made of gray matter. They interact with the cerebral cortex, thalamus, reticular formation and parts of the midbrain and spinal cord, and are important for motor functions (primarily voluntary and many involuntary ones), including speech (Webster, 1995). Due to their connections with the association areas of the cerebrum they have a direct influence on the affective, language, and other cognitive processes (Judaš & Kostovi´c, 1997; Lieberman, 1991). Their injury will result in weak and uncoordinated movements (Kalat, 1995), in several types of involuntary movements, such as jerks, tremor, and so forth (Judaš & Kostovi´c, 1997), and in cognitive disorders as well. Parkinson’s disease is the most frequent and the most extensively studied neurological disorder related to the basal ganglia.
Central Nervous System
23
Longitudinal fissure Ventral lateral thalamic nucleus
White matter
Cortex (gray matter)
Corpus callosum
Internal capsule Putamen Lateral fissure
Basal ganglia
Caudate nucleus
Globus pallidus
Fornix
Amygdala
Substantia nigra
Subthalamic nucleus
Mammillary body
FIGURE 1.7. Coronal (frontal) view of the brain with the basal ganglia.
Although the cerebellum and basal ganglia are important for planning, initiation, and performance of movements, their roles are different. The cerebellum creates the movement by trial-and-error learning and the final movement is optimal for the particular situation and the set goal. Basal ganglia release the movement from general tonic restraint, allow nonrival positions and movements to develop and prevent rival activity (Thach, Mink, Goodkin, & Keating, 2000). Warren, Smith, Denson, and Waddy (2000) have shown the importance of basal ganglia in speech planning, word recall, and short-term verbal memory. Their 53-year-old bilingual (English–German) patient, who had suffered a stroke in the left posterior nucleus lentiformis (a part of the subthalamus, i.e., basal ganglia), presented with apraxia of writing and speech in both languages. Detailed language tests revealed disorders in articulation, fluency, repetition of auditory stimuli, interpretation of complex semantic relations, definition forming, and short-term verbal memory. The connections between the cortex and subcortical areas are very important for the functioning of the brain, because the injuries in those pathways may cause behavioral disorders that are identical to those caused by injuries in particular functional areas (Kolb & Whishaw, 1996).
24
The Cognitive Neuroscience of Human Communication
The brain’s major foodstuffs are glucose and oxygen. Although it is only about 2% of the total body mass, it is responsible for about 20% of total oxygen and about 50% of total glucose consumption (Strange, 1995).
SENSATION AND PERCEPTION The prerequisites for the occurrence of sensation are (a) the existence of a stimulus; (b) the processes that convert the stimulus into bioelectrical signals suitable for neuronal transmission; and (c) the specific response of the organism to the coded message. All sensory systems are organized hierarchically, in a parallel manner, and topographically (Judaš & Kostovi´c, 1997). The first intermediary between the outside world and the body is the sensory receptor. It converts the received input energy, which may be mechanical, thermal, chemical or electromagnetic, into electrochemical energy, which fulfills one of the most important requirements of information transfer to the central nervous system (in the form of action potentials). Besides the general functioning principles common to all senses, each sensory modality has its peculiarities. For example, in the visual modality, different receptors are responsible for the blackand-white data/contrast (rod cells of the retina) from those that are responsible for colored input (cone cells). Within a single modality there may exist different receptors for specific stimulus features (different segments of a visual stimulus have been discussed already). In addition to being modality-specific (visual, auditory, gustatory, olfactory, somatosensory), all receptors may transmit the sense of pain in cases of excessive stimulation. The sensory transduction of the input stimulus into electrochemical energy is followed by neural coding of the stimulus in the primary sensory neurons that are in direct contact with the receptors. Neural coding includes information about stimulus modality, intensity, duration, and location. With the exception of olfaction, information from all sensory organs passes through the thalamus on its way to the cerebrum. The thalamus selects the bits of information to be passed on and directs them to the appropriate location in the cortex. The selected information is transmitted to the primary (modality-specific) areas in both hemispheres. Approximately at the level of the thalamus and the primary cortical areas information is grouped into wholes (gestalts). This process ensures the distinction of form from the background, in other words, drawing relevant information from all available stimulus characteristics. Information is then transferred from the primary to the secondary areas, where the wholes (i.e., their meaning) are recognized. Secondary cortical areas are also modality-specific. The term perception is used when there is sensory awareness, and that awareness is typically formed at the secondary cortical level. In other words, perception is conscious reception, adoption, and interpretation of external stimuli (exteroception) or those from one’s own body (interoception, proprioception). One of the prerequisites of perception is the existence of intracortical phenomena. Conscious sensation may be elicited only if stimulation lasts at least several hundreds of milliseconds (as indicated by electrical stimulation of the sensory cortex in
Central Nervous System
25
awake patients). Perception is the keenest in the areas of sharpest contrast (Judaš & Kostovi´c, 1997). Complex information is processed in the tertiary areas, so that the information arriving from the secondary areas is integrated and combined with affective data. In other words, this is where new information is interpreted in the light of previous experience and existing knowledge. That is part of the reason why these areas are also called association areas: a familiar odor may bring to life the image and the sound associated with it, because at some point in our life the stimuli from all three modalities were present at the same time. Pleasant and unpleasant emotions may also be associated with the event. Cerebral (mental) representation of an object, person, or anything else, for that matter, includes the corresponding physical properties, but also the unique affective and experiential components. Although it is impossible to draw a clear line between sensation and perception, it is generally accepted that sensation is subcortical whereas perception is cortical (Pinel, 2005). To put it more precisely, sensation is the outcome of the activity of receptors and their afferent pathways to appropriate sensory areas in the cortex, and perception is the result of the activity of the cerebral cells after the first synapse in the sensory cortex (Kolb & Whishaw, 1996). Attention plays a very important role in stimulus perception. In speech, it may be crucial for message comprehension. Some studies have shown that the subjects were able to extract very little information from the message presented to one ear while they were focusing on the other. Many of them had not even noticed that the stimuli presented to the unattended ear were in a foreign language or that it was speech reproduced backwards. Accordingly, very little of the information presented to the unattended ear was stored in memory. Contrary to that, if the other message is physically different or has been changed it will be easier to detect: for example, a male voice presented to one ear and a female voice to the other (Eysenck & Keane, 2000). Escera, Alho, Schröger, and Winkler (2000) found, using event-related potentials (ERP), that very small changes in the properties of an auditory stimulus activate the auditory cortex (temporal lobe) in both hemispheres, even in situations where the stimuli were not attended to. There is an ongoing controversy about the minimal perceptual unit of speech. There are equally abundant arguments for it to be the sound, the phoneme, the syllable, or the speech measure. The authors, who claim that the smallest unit is the sound or the phoneme, nonetheless believe that the decisions pertaining to sounds and phonemes are made in a wider context—at least to the syllable level. The criterion of processing efficiency does not help resolve the problem either. Namely, smaller units (e.g. phonemes) are less taxing on the memory, but processing is considerably slower than where larger units are involved. On the other hand, larger units, due to their great number of possible combinations, are more demanding on the memory. In any case, it is clear that prelexical processing implies some sort of transformation of acoustic–phonetic information into abstract representations. Since the main topic of this book is speech, it seems appropriate to shed some additional light on hearing as the primary path of its perception and on movement as the primary means of its production.
26
The Cognitive Neuroscience of Human Communication
NEURAL BASES OF SPEECH PERCEPTION AND PRODUCTION HEARING, LISTENING AND THE AUDITORY CORTEX In humans, the first reaction to sound, manifested as startle reflex, occurs approximately 16 weeks before birth. Such a reaction, albeit very primitive, already points to the active relationship between the auditory stimulus and motor reaction to it. More sophisticated reactions, such as orienting reflexes, take longer to develop. The auditory system is ready to receive sound and initiate action potentials at birth. It has been shown that electrical stimulation of the auditory pathway may cause impulses in the auditory cortex even before the final maturation of the synapse between receptor cells and the auditory nerve (Aitkin, 1990). Studies of congenitally deaf cats have revealed that electrical stimulation of the cochlea may result in normal spatial and temporal patterns in their auditory cortex (Klinke, Hartmann, Heid, & Kral, 2002). Keeping in mind the importance of the spectral pattern of auditory stimuli in determining the firing patterns of mature auditory cortical neurons, it is possible that temporal characteristics of surrounding sounds are important factors in synaptogenesis that occurs in the auditory cortex. The auditory pathway is the path from the peripheral organ (i.e., the ear), to the auditory cortex, structured as a very complex network of connections. Stimuli from each ear are transmitted ipsilaterally and contralaterally along several lines. Approximately one third of the information reaches the same side, and two thirds the opposite. The first neuron of the auditory pathway is located in the inner ear, specifically in the spiral ganglion of the cochlea, and it is in direct contact with the receptors in the organ of Corti. The cochlear nerve (nervus cochlearis) is a branch of the vestibulocochlear nerve (nervus vestibulocochlearis—the VIII cranial nerve, which is commonly referred to as the auditory nerve), and consists of some 30,000 to 40,000 axons. The vestibular branch (nervus vestibularis), which is responsible for balance, consists of some 8,000 to 10,000 axons (Judaš & Kostovi´c, 1997). Axons of the auditory nerve synapse in the ipsilateral cochlear nucleus (nucleus cochlearis). In addition to transmitting the tonotopic representation of the cochlea, considerable processing of the auditory signal takes place there. At this level there are at least six different neural responses, resulting from complex interactions and neural processing (Seikel, King, & Drumright, 1997). A number of projections from cochlear nuclei reach the superior olivary nuclei (nucleus olivaris superior) at the same level. Some go to the ipsilateral superior olivary nucleus, others to the contralateral side. Neurons in olivary nuclei have two large dendrites each: the right one receives information from the right ear and the right cochlear nucleus, the left one receives information from the left ear and the left cochlear nucleus. These dendrites can detect time differences in the activation from the left and right ear that are as short as several microseconds. This ability is partly responsible for sound localization (Gazzaniga et al., 2002; Thompson, 1993). Superior olivary nuclei are the first location where interaural differences in auditory signal intensity (particularly for high frequencies) and phase (particularly for low frequencies) differences are detected. The differences reflect different stimulation of the left and the right ear by the auditory stimulus
Central Nervous System
27
due to its position in space relative to the listener. In other words, there are neurons that react to specific stimuli features (the so-called feature detectors) and transmit the recognized and analyzed signal features to the cerebral cortex. In this way the complex sound signals are analyzed into their components. Inferior colliculi (colliculus inferior) receive bilateral signals from superior olives and indirectly from cochlear nuclei via the lateral lemniscus (lemniscus lateralis). Several responses occur here as well; it seems that at this level the intensity-difference data are combined with the phase-difference data again. Each inferior colliculus is an integration nucleus of the central auditory pathway, where about a dozen projections from the lower auditory nuclei in the brainstem, from the contralateral inferior colliculus, and from the auditory cortex come together (Syka, Popelar, Nwabueze-Ogbo, Kvasnak, & Suta, 2002). The medial geniculate body (corpus geniculatum mediale) is the last synapse in the diencephalon (in the thalamus). Here, the tonotopic organization is manifested as the projection of: (a) the ventral part of the medial geniculate ipsilaterally into the primary auditory cortex in the temporal lobe (Brodmann’s area 41); (b) the medial part into other regions of the temporal lobe (Brodmann’s areas 42 and 43); and (c) the dorsal part into the so-called cortical association areas. The diversity of neural responses has been preserved at this level as well. Some neurons are sensitive to very small intensity differences between signals reaching the two ears (Seikel, King, & Drumright, 1997). In summary, the cochlea performs the first analysis of complex sound stimuli into their components. Cochlear nuclei, superior olives, and lateral lemniscus nuclei code different aspects of the stimulus (frequency, intensity, timing) and transmit the processed information via six parallel paths into the medial nuclei of inferior colliculi. At the lateral lemniscus level, the left and the right auditory pathways are connected by Probst commissure. This commissure also has crossed projection fibers by means of which the connection with the contralateral inferior colliculus is established. All the information is integrated and synthesized here before being forwarded to the medial geniculate body in the thalamus and subsequently to the primary auditory cortex where the different aspects of the stimulus are to be recognized. The left and the right auditory pathways are connected (in addition to the level of lateral lemniscus) by the inferior colliculi commissure and by corpus callosum at the cortical level. Tonotopic organization is preserved in all auditory nuclei and in auditory cortices of both hemispheres. Cochlear representation is preserved through the auditory pathway, but it multiplies at the central levels. This redundancy is the key to our ability to analyze and understand very complex sounds. Obviously, the auditory pathway is a network of separate but intertwined neural mechanisms that are adapted for the transmission of highly specific, detailed information from periphery to the brain. Cells in the subcortical nuclei react to the wider frequency bands than the cells that are higher in the system. Other sensory systems have parallel ascending pathways, but in the auditory pathway there are more parallel paths than in the other modalities, there are more synapses in the brainstem nuclei, and, finally, after converging, they again go their separate ways into different parts of the auditory cortex (Webster, 1995). Giraud et al. (2000) have found, using the fMRI, that the human auditory
28
The Cognitive Neuroscience of Human Communication
system is organized as a hierarchical filter bank with parallel processing, so that each processing level has its preferred stimulus frequency but is able to respond to all others. Such structure makes possible separate transmission of groups of frequencies containing different bits of information, but at the same time preserves the possibility of integration of complementary features in the auditory cortex. Besides the dorsal surface of the superior temporal gyrus and the small transverse temporal gyri of Heschl, the majority of the human primary auditory cortex (Brodmann’s areas 41 and 42) and the neighboring areas of the secondary auditory cortex (Brodmann’s area 22) are located deep within the Sylvian (lateral) fissure. There are variations in the location and the size of cortical areas. This is especially true of the primary auditory area, mostly due to the great variability in the Heschl’s gyrus (Schönwiesner, Von Cramon, & Rübsamen, 2002). The primary auditory cortex receives input primarily from the opposite ear via the ipsilateral medial geniculate body. At that level the tonotopic representation is realized as columnar organization such that each column is made up of neurons that react to similar frequencies—much like the organization of hair cells in the cochlea. However, Schönwiesner et al. (2002) question the strict tonotopic organization, based on their fMRI studies of humans and other primates in which they found several locations characteristic for each frequency. Takahashi et al. (2002) reported that tonotopic organization is affected by the stimulus intensity: stimuli of higher intensity are less clearly tonotopically organized. Different neurons in the columns react to different elements of the stimulus (e.g., upward or downward frequency or intensity modulation). This is analogous to the already mentioned columnar organization of the primary visual cortex, with the neurons in different columns sensitive to different components of the visual stimulus such as movement, color, slant, and so forth (Scheich, Ohl, & Brechmann, 2002; Seikel et al., 1997). Neuroimaging techniques have revealed that different structures are involved in the processing of sound properties (the ventral “what” path) and location (the dorsal “where” path). In that respect, the auditory and the visual systems are comparable as well (Anourova et al., 2002; Kusmierek, Laszcz, Sadowska, & Kowalska, 2002). Although both pathways are present in both hemispheres, even a unilateral brain injury may cause difficulties in sound recognition and localization (Clarke et al., 2002). With respect to the perception of tactile information, however, Van Boven, Ingeholm, Beauchamp, Bikle, and Ungerleider (2005) found no dorsal/ventral dissociation. In their fMRI study of tactile form and localization processing, they found a hemispheric lateralization effect manifested as selective left intraparietal sulcus activation in the form detection task, and selective activation of the right temporoparietal junction in the location discrimination task, independent of the stimulated hand. By means of fMRI Binder et al. (2000) have found that parts of the auditory cortex in which spectral information is represented are separated from those where the intensity sensitivity is represented. Both are located on the dorso-lateral surface of the temporal lobe in both hemispheres, and although they involve neighboring regions, there is no overlap. Sound intensity is represented in the caudomedial part of Heschl’s gyri and the neighboring parts of the transverse
Central Nervous System
29
temporal sulcus, whereas spectral information is represented in the rostrolateral part of Heschl’s gyri and the neighboring parts of the transverse temporal sulcus. Due to its location deep within the Sylvian fissure, the auditory cortex is well protected from injury and even the most severe bilateral injuries to the temporal cortex do not result in deafness. Patients with damaged auditory cortex perceive simple auditory stimuli without problems, but they have difficulty in identifying very short stimuli, in discriminating sounds presented in short intervals and in judging the temporal sequence of sounds presented in short intervals. Consequently, patients with extensive auditory cortex damage often complain of difficulties in perceiving fast speech (Pinel, 2005). Damage to the auditory cortex in the left hemisphere, dominant for speech in most people, usually has more serious consequences. Although the auditory cortex is mostly located in the temporal lobe, several areas of the secondary auditory cortex (each of them a tonotopically organized map) extend into the parietal lobe. It has been shown that patients with extensive auditory cortex damage that extends into the parietal lobe have trouble locating the sound source, especially if the injury is in the right hemisphere. The right hemisphere was found to be crucial in the discrimination of the direction of frequency modulation—upward or downward (Scheich et al., 2002). Despite the fact that speech information has been processed to a great extent before reaching the auditory cortex (a large portion of information necessary to decode speech is processed in subcortical auditory nuclei), many aspects of speech (e.g., speaker variability, coarticulation) are attributed to perception that takes place in the cortex. Durif, Jouffrais, and Rouiller (2002) found that some cortical neurons process both physical properties of sounds and their relevance for behavior. The auditory cortex is not just a final point of bottom-up analysis, but also an important part of the top-down system, that is actively directed toward particular aspects of the sound stimulus, depending on the type of information that needs to be extracted from the input pattern (Scheich et al., 2002). Although there is no clear distinction between auditory projections and auditory processing on the one hand, and areas where speech and language are decoded, on the other, it is believed that conscious processing of speech takes place at the cortical level. Even very young (1-month-old) infants are able to distinguish sound categories, for example, /p/ versus /b/. Electrophysiological studies have shown that 3month-old infants have neuronal networks for phonological processing similar to adults (Dehaene-Lambertz & Baillet, 1998). These results used to be taken as evidence of the inborn speech ability to categorize phonemes perceptually. However, it has been found that children are just as successful in distinguishing among nonspeech sounds, and that the ability is present in other mammals as well. Apart from being able to categorize the sounds of their mother tongue, in their first six months of life children learn to normalize across speakers, sound contexts, and prosodic features. Cross-linguistic studies of children have revealed selective sensitivity to their mother’s voice and the sounds of the mother tongue, suggesting the influence of prenatal auditory experience. The recognition of the mother tongue and its discrimination from other spoken languages within the first
30
The Cognitive Neuroscience of Human Communication
six months relies primarily on prosodic features, only to be followed by the development of sensitivity to segmental features. This is attributed to the fact that, in infant-directed speech, the mothers (or other caregivers) talk more slowly, using higher pitch and more pronounced intonation variations. These suprasegmental features enable babies to distinguish comforting (long, smooth, falling intonation), from warning and disapproval (short, sharp intonation patterns), or calling and drawing attention (rising intonation; MacNeilage, 1997).
MOVEMENT AND SPEECH PRODUCTION Movement is a readily visible product of cerebral activity directed at acting on the world around us. Movement initiation and coordination are controlled by the brain. The entire course of the movement, from its planning to execution, is hierarchically organized. In this respect it resembles the reception and interpretation of stimuli (in reverse order). The highest wrung on the hierarchical ladder is occupied by groups of neurons whose main purpose is to plan movements. These neurons are located in the tertiary (premotor, association) cortical areas. Actions are planned on the basis of available perceptual data, experience and goals, with the help of the cerebellum and basal ganglia. Tertiary areas are responsible for very complex movements (e.g., speech) and they control primary and secondary motor areas. Secondary areas control and coordinate several muscle groups. The commands are relayed to lower levels until they reach the neurons in the primary motor cortex (Brodmann’s area 4) that will activate individual muscles or small muscle groups in order to fulfill the set goal—executing the desired movement. The execution is controlled by motor neurons in the brainstem and the spinal cord by means of commands that reach effector organs, that is, muscles. Simple movements require minimal processing and are limited mostly to the primary motor and sensory areas. Increasing complexity results in increased activated areas, in such a way that the activation spreads to the regions anterior to the primary motor area in both hemispheres. This bilateral activation can be explained by the activation of some abstract motor plan that is yet to be perfected for a specific activity, or it may reflect activation of several alternative motor plans with the same goal. The one finally selected will result in increased lateralized activity of the motor cortex, related to the activity of the contralateral effector organ, for instance, the hand. This selection process involves the supplementary motor area (Brodmann’s area 6) as well. It is particularly important for hand coordination in complex motor tasks. Obviously, the entire process has to be parallel and interactive at the same time. Feedback is present at all levels. Based on the sensory data the execution programs are modified with the purpose of fulfilling the task optimally and efficiently—yielding maximum results with minimal effort. In the process, the thalamus plays a very important role: it relays information from the sensory areas to the corresponding motor areas. However, the movement does not depend exclusively on feedback involving peripheral data. There are internal representations of movement patterns as well. Even profoundly hearing-impaired patients, who have
Central Nervous System
31
lost their hearing postlingually, can speak well despite the loss of auditory control. Obviously, kinesthetic feedback is at play here, but it would not be sufficient were it not for fixed movements the patient can rely on. The longer the period without the feedback, the less reliable the patterns will be. As it has already been mentioned above, the cerebellum and basal ganglia also play very important parts in planning, coordination, and execution of movements. It is believed that the cerebellum is particularly important for the functioning of the speech rhythm generator. This means that it is the seat of the neural center for syllable timing that is independent of afferent (sensory and proprioceptive) information or that it actually controls the center (Horga, 1996). Traditionally, it was believed that the motor cortex was the only starting point of the pyramidal tract (that was thought to be the highest level of motor control). It seems, however, that only about 60% of pyramidal tract motor neurons originate in the motor cortex, whereas the rest of them originate in Brodmann’s area 6 and in the parietal lobe. It also seems that the motor cortex is not the place where the movement originates, but rather it is a conduit for information about movement arriving from other cortical and subcortical areas. Studies have shown that the pyramidal tract comprises numerous fibers that originate in the cortex and that influence the sensory transmission at the subcortical level and at the spinal cord level (for a review see Kent & Tjaden, 1997). In short, the studies of humans and other primates indicate that motor planning involves parallel circuits. One of them—including the parietal lobe, lateral premotor pathways, and the cerebellum—is important for movements in space. The other circuit, that includes the supplementary motor area, basal ganglia and, possibly, the temporal lobe, is activated once the skill has been mastered (Gazzaniga et al., 2002). The type of functional disorder depends, naturally, on the location and extent of lesion. Lower-level injuries will impair the execution of specific movements of particular muscles or muscle groups. Higher-level injuries will impair planning and coordination, entirely or to a great extent, which will result in impaired execution of all but the simplest movements. In the case of speech production, which requires a very high degree of planning and coordination of numerous muscles, this means that an injury to the tertiary area will impair speech production, despite the preserved ability to move individual muscles involved in speech. Contrary to that, after damage to the area responsible for the movement of one of the articulators, speech may be difficult but not necessarily impossible. For example, damage to Brodmann’s areas 6 and 44 will result in speech apraxia that is manifested as nonfluent speech and uncoordinated complex speech movements; damage to Brodmann’s area 7 will cause conduction aphasia, manifested as phonological paraphasias, omissions, and transpositions, but without affecting fluency. Stuttering is explained by some authors as a consequence of inappropriate lateralization and/or uncoordinated functioning of both hemispheres. This may be realized as simultaneous and uncoordinated bilateral control or as uncoordinated callosal communication between the two hemispheres. Both explanations are based on the right hemisphere involvement in speech production to the
32
The Cognitive Neuroscience of Human Communication
extent that hinders efficient processing and control by the left hemisphere (generally dominant for speech). In other words, stuttering is a consequence of intraand interhemispheric interference between functionally related areas of cerebral cortex (Webster, as cited in Horga, 1996). Let’s say a few more words about movement in speech. All speech production models assume three phases, of which the second and the third are basically simultaneous: (a) organization of speech motor programs; (b) realization of these programs by converting the commands to appropriate muscles into a series of articulatory movements; and (c) conversion of articulatory movements into actual speech sound. The articulation program is a part of the speech motor program, not its synonym (Horga, 1996). The distinction between the articulation and speech program is clearer if we agree that motor program does not include predefined commands to the articulators, but that it is conditioned by context and the set goal. In harmony with the goal and the continuous feedback (auditory, tactile, proprioceptive) it adjusts actual efferent stimuli (i.e., execution of the articulation program) to the current context (Horga, 1998, 2002a). It is believed that some common mechanisms are at work in movement in general, and specifically in speech movement. There is evidence that regions around Broca’s area are not only involved in series of speech movements, but also in series of movements in general. There are numerous examples of brain injuries that result in parallel impairment of speech and general motor functioning. Saygin, Wilson, Dronkers, and Bates (2004) have found that not just speech production but speech comprehension as well may be impaired together with comprehension of action (pantomime interpretation). They concluded that brain areas important for the production of language and action are also active in their comprehension. Moreover, they suggest that brain is organized for action processing. This is not to be confused, however, with the composition of the muscles themselves—Kent (2004) reviewed a large body of literature and concluded that speech muscles (mandibular, lingual, palatal and laryngeal) are unique in their fiber composition, distinguishing them from other muscles in the human body. Hauk and Pulvermüller (2004) also studied the correlation between action words and specific cortical areas, and similarly to Saygin et al., suggest a possible role of mirror neurons in the premotor cortex in language processing. Another argument in favor of this view is the parallel development of hierarchical organization of speech (language) and movement. As opposed to automatic motor behavior patterns controlled by subcortical structures and reflexes, speech movements are controlled by higher levels. This may be described as a central neural feedback mechanism that is realized in the cerebral cortex due to the proximity of motor and sensory association areas; the effects of motor actions are recorded as patterns of activity in the association cortex, from whence they in turn control the muscles. Although traditionally, swallowing has been considered one of the automatic motor behavior patterns under subcortical control, there is increasing evidence that the cortex (particularly the cingulate cortex, the insula, and the inferior frontal gyrus) and the cerebellum are involved as well (Daniels, Corey, Fraychinaud, Depolo, & Foundas, 2006;
Central Nervous System
33
Stickler, Gilmore, Resenbek, & Donovan, 2003; Suzuki et al., 2003; Tasko, Kent, & Westbury, 2002; Watanabe, Abe, Ishikawa, Yamada, & Yamane, 2004). According to some authors, the center for production of speech patterns is located in the brainstem. This opinion is based on studies that have revealed mechanisms, the so-called pacemakers that keep the rhythm necessary for cyclic life-sustaining functions—for breathing, movement, and chewing—with the assumption that speech, as a higher complex function, has developed from these lower functions. There is, however, evidence to the contrary as well: patterns of muscle activity recorded during speech have been found to be different from those recorded during chewing or some other nonspeech tasks. Gracco (cited in Kent & Tjaden, 1997) suggested a compromise: lower rhythmic functions and speech differ in motor patterns, but share mechanisms for their implementation and principles of organization. For a critical review of related literature see Kent and Tjaden (1997). Just as in perception, there has been no consensus on the minimal units of the speech motor program and its execution, in other words, the articulation motor program. Distinctive features, individual sounds or phonemes, syllables, words and word groups or short sentences have been discussed as the likely candidates. Studies of both types of programs (speech and articulation alike) converge on the value of approximately seven syllables. This is the average capacity of working memory (the famous 7±2 number) and may be related to the motor speech program, but it is also the average syntagmatic length in speech, which makes it a plausible articulation unit (for review see Horga, 1996). In view of the efficiency and rate of speech, in spite of the enormous number of muscle activations (several hundreds in one second), some preset motor subroutines probably exist. On the basis of the degree of variance among speech segments Horga (1996) opts for phonetic word as this subroutine size. Besides dealing with the size of the minimal units of speech and articulation motor programs, most speech production models address the question of the relationship between execution—for example, the movement itself—and goal attainment—for example, speech. Here is a very simplified short list of examples: 1. Speech is a series of relatively constant context-independent articulatory gestures (Haskins Laboratories’ authors). 2. Movements of the articulators are determined by their relations to all other elements of the system (Chistovich and associates). 3. The physiological model oriented toward the mechanics of speech movements and its acoustic supplements (Peterson; Shoup). 4. Binary functional models that assume phonetic and phonological levels, based on distinctive features founded on the acoustic and articulation characteristics of speech (Jacobson; Fant; Halle; Chomsky). 5. Sound systems of natural languages are harmonized with human perceptual and articulation abilities, so that the sounds that are articulated in natural areas have particular quantal characteristics, in other words,
34
The Cognitive Neuroscience of Human Communication
there are areas where small articulatory changes result in considerable acoustic changes, and areas where large articulatory changes result in negligible acoustic changes (Stevens); distribution of vowels in acoustic space is such that it provides maximum perceptual contrast, regardless of the number of vowels in the system (Liljencrants; Lindblom). 6. Model of speech production based on a closed feedback involving the respiratory, phonatory and articulatory levels on the one hand, and the afferent sensory and proprioceptive feedback, on the other, with the appropriate comparator that continuously coordinates afferent stimuli with the final goal (Fairbanks). 7. The parametrical approach, according to which speech is a unique and complex system with continuous interaction between different levels, oriented toward neural mechanisms of speech production (Liberman). 8. According to several dynamic models that are based on the target theory, and that rely on neurophysiologic foundations of speech, all movements are oriented toward the set goal, without assuming fixed, preset articulatory movements (MacNeilage; Noteboom; Kelso; Saltzman; Tuller). Horga (1996), Kent and Tjaden (1997), Obler and Gjerlow (1999), Eysenck and Keane (2000), and Gazzaniga et al. (2002), among others, have reviewed and discussed such models. The language network, albeit physically close to the general sensorimotor one, is a separate system. Beatse, Sunaert, Wilm, Van Hecke, and Marchal (2000) have revealed by fMRI that the language network involves the superior temporal gyrus in both hemispheres, the left angular gyrus, the left middle and inferior frontal gyrus, and the cingulate gyrus in both hemispheres. Within the network, the receptive language regions are separate from the expressive ones. The hand, however, is represented in the sensory and motor cortex located in the precentral and postcentral gyrus, respectively.
RELATIONSHIP BETWEEN SPEECH PRODUCTION AND PERCEPTION The fundamental issue in discussing the neurolinguistic aspects of the relation between speech perception and production is the question of their cerebral representation. In other words, do speech production and perception share a common cerebral representation or are they two neurofunctionally different areas with stronger or weaker connections? Neurolinguistic literature most frequently deals with descriptions of the production (expressive) or the perception side of speech, and their relationship is commonly described by claims that production is more clearly lateralized to the language-dominant hemisphere (usually left) whereas perception is more widely distributed with greater involvement of the nondominant (usually right) hemisphere. Cortical stimulation studies have revealed that the mouth and face muscles are represented at the points that are responsible for phoneme identification. This supports the motor theory of speech perception—one of the best known and most
Central Nervous System
35
extensively discussed theories of speech perception (Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967) according to which speech is perceived by means of movements necessary to produce the heard sounds, not by means of the sounds themselves. (The theory is discussed in more detail in chapter 5, this volume.) Penfield and Roberts (as cited in Bradshaw & Nettleton, 1983) have named that region of the brain the voice control area. In other words, speech and language comprehension are parts of a single system. In their studies of chimpanzees and speech acquisition in children Premack and Premack (1994) have found that comprehension and production start off as separate systems, but become united on the basis of even very limited experience. Albeit limited, that experience is a necessary prerequisite to initiate the merging of the two systems. Hurford (1994a, 1994b), on the other hand, believes that there is no period in early childhood during which a child might know a new word or structure without being able to use it. Ojemann (1994) also concluded that language processing includes motor mechanisms even without actual production. However, the relationship is not as straightforward as it might seem. Recordings of neural activity have shown that different cells are responsible for speech perception and production, even when the word to be perceived and produced is the same (Creutzfeldt, Ojemann, & Lettich, 1989a, 1989b). The auditory speech perception system shuts down briefly during speech production. Ojemann (1994) believes that the proximity of the neurons responsible for perception and those involved in production may explain the stimulation results. They become parts of the network that has properties not inherent to individual cells. The major brain areas involved in production and comprehension of speech and language are shown in Figure 1.8. Cortical representation of speech and language is discussed in more detail in chapter 8. Jäncke and Shah (2000) and Specht, Shah, and Jäncke (2000b) also found in their fMRI studies that language production activates parts of the cortex involved in perception and vice versa; language recognition tasks activate classical speech production areas as well. Kent and Tjaden (1997) discussed possible points of contact and common representations. They may be summarized as three types of connections underlying the integration of speech production and perception: (a) neighboring location of cerebral and cerebellar motor and sensory neurons; (b) existence of multimodal neurons, or nerve cells that react to more than one sensory modality; and (c) neural structure that includes parallel and recurrent pathways. Fowler and Galantucci (2005) also provide some evidence of close correlation between mechanisms supporting action and those supporting its perception. Neighboring Location of Motor and Sensory Neurons A case in point is the location of motor and somatosensory cortex, in other words, cortical regions responsible for efferent (motor—motor neurons) and afferent (somatosensory—sensory neurons) activities of the body. Although the central sulcus separates the motor (precentral, anterior to the sulcus) from the somatosensory (postcentral, posterior to the sulcus) cortex (see Figure 1.8), some motor
36
The Cognitive Neuroscience of Human Communication
4 6
8
5
3
7
1 2
19
40 9
46
10
41
43
18
44
45
52
47 11
39
22
17
42 37
21
38
20
4
6
31 2
5
7
8 24
9
31 23
32 33
10
30
26
18
27 29
11
17
25 38
19
34 35 28
36
37
19
18
20
FIGURE 1.8. Major brain areas involved in the production and perception of speech and language.
neurons can be found immediately next to the sensory ones. In addition to the already mentioned cortical representations of mouth and face muscles involved in speech that share their location with the points responsible for phoneme identification (Ojemann, 1983), motor and sensory neurons are found in the vicinity of each other in the cerebellum as well. Such results, in combination with clinical observations, suggest that among the important roles of the cerebellum are temporal calculations that are used in many perceptual and motor functions, including speech. Multimodal Neurons These neurons respond to two or more types of stimuli. They can be found in different animal species: in macaques about 55% are bi- or multimodal (visual and
Central Nervous System
37
auditory, visual and somesthetic or visual, auditory and somesthetic); in songbirds the pathways that control song also respond to auditory stimuli. Separate sensory pathways allow stimulus distinction with respect to modality, and multisensory pathways make possible integration of stimuli from different modalities (Rosenblum, 2005). This is particularly important for speech because of its multimodal properties, and it has proven to be invaluable in speech and hearing rehabilitation (Guberina, 1992–1993; Pansini, 1995, 2002). Parallel and Recurrent Pathways Parallel pathways enable comparisons among various neural representations. In songbirds this explains their ability to learn their species-specific song—by continuously comparing their own song with that of adult birds. These comparisons are possible due to multiple neural mechanisms, modalities, multiple loops and representations of song patterns. They are based on two neural branches that mature at different times. One develops gradually during song acquisition and it is essential for song production in adult birds. The other reaches full maturity very early, quickly reacts to stimuli, and its purpose is quick and recurrent comparison of the bird’s own (motor) activity with the model (control, sensory activity). According to Williams (1989), when the species-specific song is acquired, this branch activity decreases (as cited in Kent & Tjaden, 1997). Despite the lack of comparable direct physiological data for humans, Kent and Tjaden (1997) propose the following analogy to speech: (a) different parts of the neural structure mature functionally at different ages, which explains the existence of the sensitive (critical) periods in speech acquisition; (b) different parts of the neural structure have different representations of speech sounds, such as auditory, tactile, motor, or kinesthetic; (c) there are branches that are used for recurrent comparisons of the different elements of sound representations, and as speech acquisition progresses, humans rely on their activity less and less.
2 Sex Differences
There are several aspects of differences between men and women with respect to brain structure, lateralization, and behavior. Before we go on discussing them, it should be stressed that the differences among individuals of the same sex are greater than the differences between sexes.
STRUCTURAL DIFFERENCES There is no evidence that neural connections or brain architecture are any different in men than in women, although at all ages the average brain mass of men is approximately 10% greater than in women. That difference can probably be attributed to the difference in neuron size rather than number, and it does not imply differences in performance (Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Differences in mass between the two hemispheres (right hemisphere has greater mass, whereas the left has greater density) are smaller in women than in men. Thicker cortex of the right hemisphere as compared to the left has been found in male, but not in female, fetuses. The shape of the left lateral (Sylvian) fissure corresponds to the right in more women than men. In women, the left planum temporale is not as frequently larger than the right as it has been found in men. Relative brain size is different at 2 years of age, and in the hypothalamus, sex differences occur around the fourth year of life (Swaab & Hofman, 1995). Verchinski et al. (2000) conducted an MRI study on 111 healthy subjects ranging in age between 17 and 54. They reported significantly greater gray matter density in most neocortical regions in women. The greatest sex differences were found in the left middle temporal gyrus (Brodmann’s area 22), in the right parietal lobulus, and in the left precuneus. None of the areas in male subjects had greater gray matter density. Although they reported no statistically significant differences, Amunts, Schleicher, Ditterich, and Zilles (2000) found a trend toward greater asymmetry in cytoarchitecture of Brodmann’s areas 44 and 45 (that correspond to Broca’s area) in men than in women. Callosal size has been the object of study by several groups of authors. Most have agreed that some parts of corpus callosum are larger in women than in men, but there is no general consensus. Cowell, Allen, Zalatimo, and Denenberg (1992) found that in men, the corpus callosum reaches its maximal width by age 20, and in women much later—between 41 and 50 years of age. These age-dependent differences may be one of the causes of discrepancies in the literature. Besides, the callosal surface size is not a reliable indicator of the actual number of commissural axons (Judaš & Kostovi´c, 1997). In any case, even if the size difference is accepted, it does not necessarily result in different cognitive functioning (Kimura, 1992). 39
40
The Cognitive Neuroscience of Human Communication
DIFFERENCES IN FUNCTIONAL ORGANIZATION OF THE BRAIN According to Kimura (1992), sex hormones begin exerting their influence on brain organization so early in ontogenetic development (prenatally) that from the very moment of birth the environment acts on the differently structured brains of girls and boys. She found that 3-year-old boys were better at aiming than girls of the same age and that sex differences in rotating objects in space were evident before puberty. Although the influence is less pronounced in humans than in other animals, early exposure to sex hormones has permanent effects on brain organization. Cohen and Forget (1995) described possible hormonally determined regulation of activities in brain hemispheres during cognitive tasks. According to Bradshaw and Nettleton (1983), left-hemisphere injuries affect language abilities to a lesser degree in women than in men, due to their less prominent left-lateralization of language functions. Kimura (1983, 1992) does not believe that these different consequences of left-hemisphere damage are due to more bilateral organization of language functions in women. According to her, unilateral focal damage occurs more frequently in posterior areas than in anterior ones, regardless of sex. The difference lies in the fact that in more women, speech and speech-related motor functions are represented in more anterior regions of the left hemisphere as opposed to men, who on average, more frequently have the same functions represented in posterior regions of the left hemisphere. Still, Kimura (1992) reports no evidence of sex differences in functional brain asymmetry with respect to the basic speech, motor, or spatial abilities. Kanasaku, Yamaura, and Kitazawa (2000) also found sex differences in lateralization to vary along the anterior–posterior axis. She did report, however, differences in more abstract verbal tasks. For instance, damage of any hemisphere caused poorer results on verbal tests in women, whereas the results were poorer only in men with left-hemisphere damage. Shaywitz et al.’s (1995) results of fMRI studies of rhyming tasks show that women exhibit more bilateral activation than men. Other authors also report task-specific sex differences (for example, Kitazawa & Kansaku [2005], and Rossell, Bullmore, Williams, & David [2002]). Developmental dyslexia is more frequent in boys than in girls. Moreover, it has been shown that dyslexic girls still function better than dyslexic boys, which suggests greater plasticity during development (Spreen et al., 1984). As it has been mentioned at the beginning of this chapter, male and female brains are organized differently from the first days of life. During prenatal development sex hormones control that differentiation. Similar mechanisms probably cause within-sex variation, as suggested by the relationship between levels of some hormones and adult cognitive styles (Kimura, 1992; Sherwin, 2003).
BEHAVIORAL AND COGNITIVE DIFFERENCES Sex differences in cognitive abilities are most frequently and most consistently manifested in the language and spatial domains (Spreen et al., 1984). In general, it is believed that the two hemispheres are more asymmetrically organized for
Sex Differences
41
speech and spatial functions in men than in women. The considerable difference in the size of Brodmann’s area 22 (by approximately 11%) is thought to underlie the sex differences in language abilities (Verchinski et al., 2000). On IQ tests, girls score consistently better on verbal tasks and poorer on visuospatial tasks than boys, especially after puberty. It seems that the most pronounced differences in cognitive functions are to be found in the types of abilities and not in the overall intelligence level as measured by IQ (Bradshaw & Nettleton, 1983). On average, men outperform women on some spatial tasks. They are particularly more successful on tests that require mental rotation of an object in space or some other type of manipulation. They are more successful on mathematical reasoning tests and in orienting on a given route. They are also more precise on tests of target-directed motor skills—in other words in aiming and intercepting. Women have greater perceptual speed (they are quicker to find objects that match some target object). Women display greater verbal fluency, including the ability to find words that begin with a given letter or that satisfy some other criterion. Women outperform men in mental arithmetic and remembering landmarks. On precise manual tasks (e.g., O’Connor’s dexterimeter) women are faster than men. These differences are manifested in gender distribution in certain occupations (composers, chess players), disorders (developmental dysphasia and dyslexia, stuttering, autism), in normal developmental stages (speech acquisition), and in overall behavior during childhood (Swaab & Hofman, 1995). Boys tend to investigate more, girls tend to be more social (Bradshaw & Nettleton, 1983). These traits may be encouraged by parents and society. Hereditary factors probably have a stronger impact on spatial skills whereas environmental factors seem to influence verbal abilities more. Sex differences seem to be more pronounced in more authoritarian cultures with rigidly defined gender roles (Bradshaw & Nettleton, 1983). Kimura (1992) believes that human brain organization has been determined by natural selection through generations. During the thousands of years of human brain development people have lived in relatively small groups of hunters and gatherers, with probably clear division of labor between men and women. Men engaged in hunting, which frequently involved covering great distances and developing strategies for finding and overpowering animals. They were responsible for the defense of the group against predators and enemies as well as for design, manufacture, and use of weapons. Women were most probably in charge of tending to home, which involved child care, gathering food nearby, preparing meals, and making clothes. Such specialization assumed different pressures on men and women. Men had to develop skills of aiming and finding their way on the basis of terrain configuration, and women had to develop fine motor skills in a limited space and sensitivity to small differences in the environment as well as in the appearance and behavior of children. These speculations aside, there is no firm evidence that would explain sex differences in cerebral representations of verbal and spatial functions as a consequence of environmental and/or social factors. Sex differences are not exhausted by the verbal–nonverbal aspects of processing. There are also differences in the degree of lateralization. At first it was
42
The Cognitive Neuroscience of Human Communication
believed, probably due to different rates of maturation, that women were more lateralized than men. At present the opposite is more commonly found and believed to be true. Smaller differences between the left and the right hemisphere in women compared with men have been found in studies of morphological asymmetries, clinical studies (women are less affected than men by left-hemisphere trauma), and behavioral studies on healthy subjects. Although, paradoxically, men are more frequently and strongly left-handed than women, they exhibit more pronounced right-visual-field and right-ear advantage for verbal material. Sex differences are as a rule easier to show on verbal material, just as verbal tasks generate stronger and more consistent asymmetries than nonverbal, visuospatial materials (Bradshaw & Nettleton, 1983). Women frequently exhibit left-visual-field advantage on lexical decision tasks (but not when vocal latencies are measured) early on in the task. On phonological tasks (as revealed by MRI) men exhibit more left-lateralized brain activity compared with women, whose activation is more diffuse (Shaywitz et al., 1995). Women resemble left-handed men by exhibiting the verbal contribution of the right hemisphere, suggesting that in women the areas otherwise reserved for visuospatial processing in the right hemisphere are taken over by the secondary speech mechanisms that facilitate comprehension of difficult and/or unfamiliar material. Moreover, women with right-hemisphere injuries score more poorly on tests of verbal intelligence, but show no signs of dysphasia. It seems that the right hemisphere in women is more active in speech comprehension than in production, which may explain the absence of obvious effects when it is suppressed by the unilateral sodium amytal injection (Bradshaw & Nettleton, 1983). However, it should be said that cerebral blood flow measurements obtained by PET studies have revealed that, both in men and in women, blood flow increases in the left hemisphere during verbal tasks and in the right hemisphere during spatial tasks (Kolb & Whishaw, 1996). McRoberts and Sanders (1992) reported that in the recognition of fundamental frequencies, men were significantly better than women when the stimuli were presented binaurally (i.e., to both hemispheres simultaneously). Dichotic listening tests have revealed no significant sex differences. They speculate that the reason for that may be the differences in relative callosal width. In a study of the relationship between active hand and ear advantage carried out on 72 male and 72 female subjects, Mildner, Stankovi´c, and Petkovi´c (2005) reported consistently lower right-ear advantage for male than for female subjects. Sex differences in verbal and spatial tasks may be a consequence of unequal maturation rates. Early maturing adolescents (typically women) achieve better results on verbal than on spatial tests and are less clearly lateralized. Those who mature more slowly have a chance to stabilize their hemispheric dominance later, which helps spatial abilities if language develops before them. Thus, late maturers (boys) can develop language abilities in the left hemisphere without interference with the right-hemisphere functions. Accordingly, the right hemisphere is free to develop its visuospatial abilities (Bradshaw & Nettleton, 1983). Besides resulting in more pronounced lateralization and separation of functions, an extended period of maturation typical of men might mean a longer critical period during which
Sex Differences
43
malfunctions in verbal ability development may occur. Developmental dyslexia in boys may be a consequence of too much reliance on bilaterally represented spatial functions that develop late, which does not happen in girls because they are protected, having had their language skills developed earlier. The greater degree of hemispheric specialization probably puts men at risk with respect to language disorders, because they have no alternative support system in the nondominant hemisphere. So, if the differences are summarized into the different maturation rates it may be concluded that, regardless of gender, early maturers have better developed verbal abilities, whereas late maturers develop better spatial skills. Contrary to this, authors such as Shucard (as cited in Spreen et al., 1984) have reported that sex differences in cognitive functioning are apparent in babies as young as three months. Sex differences have been attributed to hormonal influences as well. It is possible that sex hormones change male and female brains in such ways that they respond differently to social influences. Some genetic models have also addressed variations in spatial abilities (Kolb & Whishaw, 1996). Perceptual asymmetries may vary in men and women because of different approaches to the task. For instance, women might prefer verbal strategies during spatial tasks, so that the obtained lateralization effect would not be a measure of sex differences in cerebral lateralization of that particular function, but rather a manifestation of the preferred cognitive style in task solving (Kolb & Whishaw, 1996). Cahill (2003) found sex differences in memory for emotionally arousing events. Right hemispheric processing of global/central aspects of a situation is more pronounced in males, whereas left-hemispheric processing of more local/ fine detail aspects is more pronounced in females. Finally, it is possible that genetic, developmental, and environmental factors predispose men and women for a particular cognitive style, which influences their successfulness in solving verbal and spatial problems. Most probably, gender differences are at least partly under the influence of neurological factors that may be changed by the environment (Kolb & Whishaw, 1996). Sommer, Aleman, Bouma, and Kahn (2004) reviewed 14 studies involving 377 men and 442 women. Their meta-analysis indicated no significant difference in language lateralization between men and women (see also Sommer, Aleman, & Kahn, 2005). They proposed that general sex differences in cognitive performance and the neuronal basis for these cognitive sex differences are not to be found in different language lateralization.
History of 3 Brief Neurolinguistics from the Beginnings to the 20th Century First descriptions of language disorders date back to pharaonic medical texts recorded on papyrus, some 3000 years B.C. The outlines of the first theory of brain and nervous system functions appeared in the first centuries A.D. and were based on ideas originating in the last few centuries B.C. The so-called ventricular theory was based on fluids and their flow, because the scientists of the day believed that something had to move in the brain in order for it to work. That “something” flowed between the sensory and the effector organs and was called animal spirits. The theory held on until the 17th century and Descartes (1596– 1650) incorporated it into his hydraulic theory. Some principles of ventricular theory (later also referred to as the medieval cell doctrine) have been kept in the more recent models of brain function developed within cognitive neuroscience (e.g., modular structure according to which the functions are localized in various regions of the brain). Between the 11th century and Renaissance, written descriptions of medical cases included drawings from which it was obvious that speech production was located, together with memory, in the third ventricle. Moreover, language and memory disorders were commonly described as a single problem. In the 16th century, the first associations between language disorders and certain brain injuries were determined. Different aspects of impaired language function were observed, which may be seen from descriptions of alexia and agraphia, aphasia and even selective aphasia, with preserved internal speech and thought but with seriously impaired speech production. There were also records of aphasia without alexia or agraphia. This was accompanied by descriptions of differences between articulatory control and language production, because clearly, there were patients who had preserved motor functions but could not speak. In the 17th century, the increasing number of experimental studies of brain brought about abandonment of ventricular theory and acceptance of models that proposed localization of functions in particular brain structures. The Renaissance belief that the nerves relayed information to and from the brain was kept but the idea that nerves were hollow tubes through which energy flowed back and forth was abandoned. Instead, it was posited that information was coded so that different ideas were transmitted by means of different vibrations. The number of 45
46
The Cognitive Neuroscience of Human Communication
descriptions of simultaneous and related occurrences of language disorders and right-sided paralysis increased; descriptions of cognitive function impairments became more thorough, including discussions about acalculia, agrammatism and other disorders, and with discussions about differences between disorders accompanied by agraphia and/or alexia and those without them. Language was still closely associated with memory. The scientists could distinguish language disorders from general cognitive impairments, production from comprehension, lexical errors from sound or letter errors, as well as syntactic and morphological disorders. The brain was believed to function as a whole. The idea of localization of functions in the brain was adopted and further advanced in the 18th century. It was believed that each ability or trait was connected with a special nerve bundle in the brain. At the time, sensory, moral and intellectual traits and abilities were considered to be equal candidates for localization. Moreover, some authors believed that each word had its own neuronal fiber. Franz Joseph Gall (1758–1828) thought that the shape of the human brain with all the individual traits left a trace on the inside wall of the skull, which could be observed from the outside as bumps. This was called craniology. He also proposed two separate brain areas devoted to two language levels and marked them as Wortsinn (for the lexical component) on the left orbit, and immediately above that was the place for expressive speech—Sprachsinn. The relationship between abilities and skull shape established by Gall served as the foundation for phrenology, a very influential pseudoscience that had a significant impact on the 19th century, most closely connected with Johann Gaspar Spurzheim (1776–1832). In line with Gall’s descriptions, the phrenologists proposed specific locations in the left and the right hemisphere for different functions that were reflected on the external shape of the skull. There were areas responsible for hope, love, caution, hate, shyness, language and many other functions, abilities and traits; within the brain there were the organs of hope, caution, and so forth. For example, it was believed that people with bulging eyes had exceptionally developed language abilities, because the seat of language functions was thought to lie behind the eyes. One of the most heated opponents of phrenology was Pierre Jean Marie Flourens (1794–1867). On the basis of his tests on animals he concluded that all senses, perception, and voluntary activities had common brain substrates. In spite of numerous critics, many authors were looking for connections between impaired functions and brain damage as support for phrenology until well into the third decade of the 19th century. Although not a fervent supporter of phrenology, Jean Baptiste Bouillaud (1796–1881) supported the theory of brain localization, particularly localization of language function in frontal lobes. He offered ample clinical evidence of the connection between frontal lobe injury and loss of speech. Moreover, he described several cases of isolated speech disorders without other motor impairments and with intact nonspeech tongue movements, on the basis of which he concluded that the center for motor control of speech is separate from the center(s) for general motor control. He also distinguished between the organ of spoken language
Brief History of Neurolinguistics From the Beginnings to the 20th Century
47
(i.e., speech), located in the white matter, from the organ of memory for words (i.e., lexical representation), located in the gray matter, in both frontal lobes. Pierre Paul Broca (1824–1880) focused on disorders of speech production because until the sixth decade of the 19th century speech referred only to production, whereas disorders of speech comprehension were not discussed within language proper. At the time comprehension processes of any kind were thought to belong to the mind that was studied by philosophers, theologists, and psychiatrists. By the time his well-known papers were published in the 1860s, a 40year-long debate about the role of frontal lobes in speech production and about the associations between speech disorders and injuries, and stroke in that region had been raging in medical circles. It was clear from many published case studies that most physicians were aware of the hypothesis about the existence of a language center in frontal lobes. Cases of aphasia without frontal lobe damage were taken as counterarguments, and not as evidence of different types of aphasia. Before Broca’s papers, an unknown village doctor, Marc Dax (1770–1837), reported at a medical convention in Montpellier, France, his observations that of his 40 brain-damaged patients who presented with speech problems, none had a right-hemisphere injury. That report went unnoticed, and Dax died a year later, unaware of having anticipated research into lateralization of function. Whitaker (1995) believes that even when Broca’s results were published 25 years later, they might have gone unnoticed had not scientists and doctors already begun accepting clinical pathological associations between specific brain regions and particular functions. Broca presented postmortem results of two of his patients, followed by seven more aphasic patients, who had suffered left-hemisphere injuries, specifically in the frontal lobes, anterior to the motor area. Because of these results and his detailed case descriptions, this area—for speech production—is called Broca’s area. Soon after that, authors began characterizing neurologically acquired speech comprehension disorders. Several names were the most prominent: John Hughlings Jackson (1835–1911) and Carl Wernicke (1848–1905), as well as, according to Whitaker (1998a) the unjustly overlooked Theodor Meynert (1833–1892). It became widely accepted that the left hemisphere was responsible for language in general, not just for speech production. Studies showed that left-hemisphere damage caused difficulties in reading, writing, and speech comprehension, which was not the case with right-hemisphere damaged patients. Because of Wernicke’s descriptions of sensory aphasia occurring as a consequence of injuries to the posterior superior temporal gyrus (approximately at the point of contact of the temporal and the parietal lobe), the center for speech comprehension was named Wernicke’s area. Case studies revealed that apraxia was another consequence of left hemisphere damage. On the basis of these results the left hemisphere became known as the dominant hemisphere, due to its crucial role in the control of all complex behavioral and cognitive processes. Some authors, among whom Whitaker (1998a) particularly points out Henry Charlton Bastian (1837–1915), were aware of the fine differences between various aspects of language functioning and types of memory. Bastian found that function disorders (e.g., aphasia) may
48
The Cognitive Neuroscience of Human Communication
be caused by injury to the center itself, to its connections with periphery, or to some other connections. Obviously, his model had some properties of the neural networks models. According to him, words establish connections with percepts and concepts, and during the thought process brain activity is not limited only to strictly localized centers, but there is a widespread activity in various cortical areas in both hemispheres. He proposed that concepts were formed in cortical areas immediately next to sensory centers, which would correspond loosely to the idea of association areas. Whereas studies by Broca, Wernicke, Bastian, and many others dealt mostly with the taxonomy analysis of the observed phenomena at the word level (or lower), John Hughlings Jackson was the first to give any serious thought to agrammatism. He talked about the propositional nature of language and concluded that propositional meaning did not depend only on constituent words, but also on relationships among words, making the sentence the unit of meaning. Hughlings Jackson also claimed that the right hemisphere was involved in automatic (involuntary) expression and perception. His particular contribution is a description of the motor cortex (precentral gyrus), especially the observation that different parts of the contralateral side of the body are represented in specific locations of the cortex. He also noticed that complete loss of function was rare, and based on that, concluded that a particular type of behavior was managed by many regions of the brain. Another one of his conclusions was that localization of symptoms does not necessarily imply localization of function (Gazzaniga, Ivry, & Mangun, 2002). Arnold Pick (1851–1924) believed that sentence plan (i.e., syntactic structure) preceded the selection of lexical constituents and that content (lexical, open class) words were selected before functional (grammatical, closed class) ones, with surface grammatical form being determined last. He talked about two types of expressive agrammatism: motor (quantitative) and sensory (qualitative), and attributed both to the loss or decrease of control over grammatical devices. He related motor agrammatism to motor aphasia, frontal lobe injuries and telegraphic speech, and sensory agrammatism to temporal lobe injuries, but his emphasis was on expressive aspects of speech. After the relatively long and very active period of localizationism, a more holistic approach to brain study was adopted at the beginning of the 20th century. One of the most influential proponents of that view was Karl Lashley (1890–1958). In his studies of rat behavior he concluded that there was no single location in the brain whose injury might prevent maze learning. His most important finding was that the extent of the injury, more than its location, will affect a particular function. He defined two important principles: the principle of mass action and the principle of equipotentiality. According to the first, certain types of learning are controlled by the cerebral cortex as a whole (contrary to the view that each cognitive function has a special location in the cortex). Italian physiologist Angelo Mosso (1846–1910) noted a relationship between mental activity and increased number of pulsations in the cortex, in other words, between cerebral blood flow and neuronal activity (Gazzaniga et al., 2002). According to the second principle,
Brief History of Neurolinguistics From the Beginnings to the 20th Century 49
which mainly refers to sensory systems, some parts of the system can take over functions from the other parts. By the second decade of the 20th century it has become clear that the nervous system was built of dynamically polarized nerve cells, whose axons and dendrites were organized into groups, and made specific neural connections by means of synapses. In the 1930s the first motor and sensory maps were produced, and it became obvious that each sensory modality has more than one of those maps. After World War II different methods of imaging and quantifying these connections were designed (more on that in chapter 4, this volume). The obtained results and techniques ensured better understanding of brain physiology and changed the approach to speech and language. However, the key question—does the brain function as a whole or as a set of independent parts—continues to be debated. According to Kosslyn and Andersen (1992), it seems plausible that simple processes are localized, whereas complex functions such as perception, language and others are more widely distributed (as cited in Gazzaniga et al., 2002). This simple distinction does not solve the problem, of course. Rather, it moves it to a different plane: to the question of the level of complexity at which a process is localized, or at what point it becomes complex enough to warrant wider distribution.
4 Research Methods Methods of research into brain functioning have developed alongside advancements in knowledge and technology. The application of a particular method depends, naturally, on the function we wish to study, subjects, available resources and equipment, and frequently on what is in vogue, or for some other reason the focus of interest, at the particular point in time in the scientific community. Existing methods may be characterized in several ways: according to the subjects studied (e.g., clinical population vs. healthy subjects); according to the functions studied (e.g., language, motor, etc.); according to type of tasks used (e.g., detection, identification, or discrimination); chronologically (from the oldest to the most recent); or according to the techniques used. Obviously, some overlap is unavoidable and, regardless of the criterion, one method may fit into several categories. This is not an exhaustive list, but it includes brief descriptions of the methods most frequently used in neurolinguistic research on clinical and healthy subjects, as well as those that are, due to their invasiveness, costliness, or relative obsoleteness, used less frequently. Many methods used with healthy subjects are also applicable to clinical patients, such as neuroimaging methods. Since there are advantages and disadvantages to every single one of them, most authors agree that a combination of several different methods on the same subjects will yield the most reliable results. This chapter is limited mainly to the methods acceptable in work with humans, so different lesion methods, genetic and/or pharmacological manipulations, deprivation, and similar methods are not included. Their descriptions may be found in neuroscientific literature (Gazzaniga, Ivry, & Mangun, 2002; Judaš & Kostovi´c, 1997; Purves et al., 2001; Sternberg, 2003).
CLINICAL STUDIES This large group of subjects includes patients with brain damage, regardless of etiology—trauma (i.e., open or closed head injury), cerebrovascular problems (e.g., stroke), tumor or degenerative diseases and conditions (e.g., Alzheimer’s or Parkinson’s disease, epilepsy, etc.). The first documented studies of brain language functions were carried out on clinical patients, particularly in the 19th century. The authors, mainly physicians and surgeons, described observed speech disorders and tried to make connections between their observations and diagnosed brain injuries. Whenever possible they looked for confirmation of their findings in postmortem analyses. They are referred to as diagram makers, because they tried to establish unambiguous relations between injuries and function disorders. In other words, their goal was to 51
52
The Cognitive Neuroscience of Human Communication
draw inferences about the ways different parts of the brain contribute to behavior. The disadvantages of those studies were the strictly localizationist approach, insufficiently defined psychological terms, and unsatisfactory empirical descriptions. However, brain injuries and their consequences cannot be reduced to the loss of a single component. Damage to one function does not mean that other aspects of cognitive functioning are intact. For instance, formation of scar tissue or disorders in blood supply may affect functioning of neighboring regions. Finally, it is debatable how much we can infer about the functioning of individual components from information about the consequences of their removal. Even the most detailed description of difficulties will not answer the question of how the language system actually works. At best we will find out what aspects of the system have been damaged. Obviously, we cannot reliably make conclusions about functions of the removed area on the basis of facts about how behavior changes when the area has been removed. Furthermore, the key technical issues here are whether the structure in question has been removed entirely and what other structures may have been damaged (Gordon et al., 1994). It is difficult to infer about functioning of the intact brain on the basis of the injured one, because the exact extent of damage or recovery rate is usually not known. It has been shown that more reliable associations between injury and functional disorder may be seen approximately one year after injury. Recovery is most remarkable in the first few weeks and months, but to a certain extent it continues after that initial period (Dronkers & Ludy, 1998). It is also possible that methods to determine functioning are just not sensitive enough to register change. Obviously, one of the more serious drawbacks of forming any conclusions about intact functioning on the basis of clinical cases is that normal functions of the system are not likely to equal those lost by damage. Consequences of injury may be masked by positive symptoms, for instance, events occurring in the injured individual that would not have occurred in a healthy one (Legg, 1989). Moreover, it is usually impossible to control some other characteristics of the patient—the natural aging process, psychological status, other health problems, and so forth. Another important issue is which results are more reliable: those obtained from large groups of subjects or those based on single case studies? The average behavior of many patients who supposedly suffer from identical disorders or injuries to a particular brain region does not necessarily offer a better insight into cerebral representation of a cognitive function. In averaging, considerable anatomical and functional individual variation may be lost. Moreover, individuals who differ widely not only in the extent of their injuries but also in other characteristics that may significantly affect recovery, such as education, age, socioeconomic status, attitude toward therapy, motivation, and so forth, are frequently studied together as a homogenous group. This approach ignores individual compensatory strategies that also play an important role in recovery and tends to place too much emphasis on some areas of the brain, neglecting others that are seemingly inactive or less active during certain functions. Although group studies are carried out on patients who share some common characteristics of linguistic behavior, or that have been diagnosed with damage to the identical region of the
Research Methods
53
brain, the groups are still quite heterogeneous. Obviously, it is an advantage that individual case studies avoid unjustified overgeneralizations and do not involve large groups of patients with similar cognitive deficits. In this type of study individuals are subjected to large numbers of different tests, which can offer insights into their functional cerebral organization and help to test theories and models of cognitive functioning. One of the most obvious disadvantages of this kind of research, naturally, is the fact that one case cannot serve as a basis for general conclusions, because individual behavior is affected by many different factors (some of which have been mentioned previously) that may not be noticeable or directly measurable. Willmes (1998) and Eysenck and Keane (2000) propose a sensible compromise: start with individual cases and look for what they have in common with other individuals. Despite all the drawbacks, clinical case studies are a valuable source of data about cognitive and other functioning.
STUDIES OF SPLIT-BRAIN PATIENTS One type of brain injury is severing of connections between the two hemispheres (commissurotomy). In humans, the most frequent version is cutting the corpus callosum (callosotomy) with the purpose of preventing the spread of epileptic seizures from the affected hemisphere to the healthy one. Another small group of patients without callosal connections are the so-called acallosal patients, in whom the connection was never established. Commissurotomies became the method of choice between the 1940s and 1950s, but were subsequently replaced by increasingly more effective antiepileptic medications. At present the procedure is used only exceptionally, if all other attempts to stop epileptic seizures fail. In cases of nonexistent callosal connections between the two hemispheres, the centers in each of them are intact and each hemisphere can operate on its own. Studies have shown that, under experimental conditions, such patients function as if they had two separate brains, which is not noticeable in everyday life (Milner, Taylor, & Sperry, 1968; Sperry, 1981; Zaidel, Zaidel, & Bogen, 2005). As it was explained in chapter 1, the corpus callosum is the largest connection between the two hemispheres. It is more than just a vital communication line, because it coordinates hemispheric functioning and behavior. Although we have two hemispheres capable of neural activity and—as hemispherectomy patients show us, we can function with only one—we behave as one mind, not two. Two hemispheres between which the corpus callosum has been severed can learn identical tasks, whereas intact corpus callosum prevents that. In callosotomy patients, one hemisphere has no awareness of the existence or activities of the other. This was supported by observations of perception, learning, cognition, volition, and memory (Levy, Koeppen, & Stanton, 2005). However, the two hemispheres can exchange information, for example, by means of facial expressions that are bilaterally controlled, and via some subcortical connections (Kalat, 1995; Sternberg, 2003). A typical experiment carried out on split-brain patients looks like this: the patient is seated at a table with test objects hidden from view by a partition. The
54
The Cognitive Neuroscience of Human Communication
subject is required to pick up or feel the object within reach with one hand, and to (a) name it, or (b) feel for an identical object with the other hand. The tests are run for each hand separately. Due to crossed connections between the periphery and the brain the right hand has direct access to the left hemisphere, and the left hand to the right hemisphere; because the two hemispheres have been disconnected, the information that reaches the brain cannot be transferred from one hemisphere to the other. Hemispheric differences in language functioning are manifested as the inability of the patient to name the objects felt with the left hand (right hemisphere), with preserved ability to name those felt with the right hand (left hemisphere). The results are highly dependent on the type of commissurotomy—whether it was complete or partial, and if it was partial, how much of the connection was preserved. For example, an intact splenium will preserve the connection between occipital lobes, and the anterior commissure will enable transfer of semantic (but not sensory) information (Gazzaniga et al., 2002).
CORTICAL STIMULATION Cortical stimulations involve administering stimuli to a part of a neural circuit and measuring its consequences at some other point. Stimuli may be electrical or chemical. Although the first results of cortical stimulation in an awake person were reported as early as 1909, the method was used more systematically by Penfield and Roberts (1959) with the purpose of determining cortical locations crucial for language functioning during brain surgery in order to spare them. Since the brain has no pain receptors the procedure is performed while the patient is awake. Electrical stimulation of particular points in the cortex elicits various sensations, images, and experiences that are entered into the so-called stimulation map as the patient describes them. Hence the term brain mapping. Stimulation of speech areas causes interference in object naming. Patients may lose the ability to speak entirely or just the ability to name objects presented in pictures. Hesitations, slurred speech, repeated attempts to name the objects, and other effects have also been recorded. Such stimulation maps have been compiled for functions other than speech production and perception as well. There are several methodological problems to be dealt with here. One is the problem of whether the stimuli are physiologically real in terms of their intensity and/or duration as well as the issue of interpretation of localized stimulation of a system that under natural circumstances most probably receives complex stimulus structures. The main technical problem is how to apply a stimulus of intensity that faithfully reflects the level of activity occurring spontaneously in the brain, and how to determine which structures were affected by stimulation: the brain is structured so that information about the same event may be coded in the activity of a large number of not necessarily neighboring cells. Furthermore, the information on when a cell is inactive is just as important as knowing when it is active, because the brain’s electrical activity is temporally organized. If we do not know in advance how the information is represented in the particular brain region, stimulation will not be realistic, and the results will be worthless (Legg,
Research Methods
55
1989). Besides, it has been shown that human brains do not have identical functional organization of language functions (Whitaker, 1998b).
TRANSCRANIAL MAGNETIC STIMULATION (TMS) This method is used for experimental purposes on healthy subjects. It simulates injury of a desired brain region. An electrical coil is positioned on the skull over the location of interest. The resultant magnetic field stimulates neural activity on the area of about 1 to 1.5 cm2 around the coil position. For example, stimulation of the motor cortex activates wrist and finger muscles in about 20 ms; stimulation of the visual cortex causes inability to recognize letters. An obvious disadvantage of this method is that it may be used only to study cortical surface. Furthermore, stimulation effects are very brief, and the timing of the signal beginning and stimulation onset is crucial: in the previously mentioned example of letter recognition, the errors will be elicited only if stimulation is applied 70 to 170 ms after the letter has been presented (Gazzaniga et al., 2002). Another drawback of this method is a well-established risk of inducing seizures, and recently it’s been suggested that memory can be affected as well (Boatman, personal communication).
WADA TEST This method, first introduced by Huhn Wada in 1947, is used clinically to determine the language-dominant hemisphere in patients who are candidates for neurosurgery. The procedure involves injecting the anesthetic sodium amytal into the carotid artery on the side of the neck, which supplies blood to the brain. This temporarily blocks the function of the ipsilateral hemisphere, with practically no effect on the contralateral one. Within seconds of the injection, the hand opposite the injection side starts to drop, because the anesthetized hemisphere cannot control it any longer. During the 5 to 10 minutes the anesthetic is effective the patient is presented with various (mainly linguistic) tasks. For example, reciting the alphabet, days of the week, or months, and naming well-known objects. The procedure is carried out for one side at a time. The patient’s inability to perform the tasks successfully is taken as an indication that the anesthetized hemisphere is dominant for that particular activity. With the anesthetized left hemisphere about 95% of right-handers and approximately 70% of left-handers cannot talk at all for about 2 minutes. For several more minutes after that they behave as aphasics—making errors in order and/or naming. With the anesthetized right hemisphere there are typically some minor speech problems, but speech is never inhibited entirely (Kolb & Whishaw, 1996). About one half of the remaining 30% of left-handers behave just the opposite—total failure of speech functions occurs with anesthetized right hemisphere, with some inhibition with the anesthetized left hemisphere. The other half has apparently bilateral speech control because the consequences are the same regardless of the anesthetized hemisphere. The results of Wada tests are usually compared with behavioral tests performed on the same subjects. The main disadvantage of this method is its invasiveness and
56
The Cognitive Neuroscience of Human Communication
inappropriateness for healthy subjects. Apart from that, if used on its own, it can only help determine the dominant hemisphere, but not regions of increased activity during a particular function.
NEURORADIOLOGICAL METHODS A conventional X-ray can reveal, for example, skull fracture, but it is not appropriate for visualizing different brain structures. Somewhat better are contrast techniques. These techniques involve temporary substitution of some of the cerebrospinal fluid with air (nowadays considered obsolete) or injecting the cerebral artery with a dye impermeable to X-rays (angiography); such techniques enable visualization of ventricles and fissures, as well as of the vascular system of the brain. However, they are invasive and their use is only clinically justified. Besides that, they are not appropriate for functional neuroscientific research because they do not provide clear identification or differentiation of gray and white matter structures.
COMPUTERIZED (AXIAL) TOMOGRAPHY—C(A)T This technique involves computer-supported use of X-rays. It is based on the principle that different biological materials vary in their density, which affects absorption of X-rays. High-density tissues (e.g., bones) absorb a great deal of radiation, neural tissue somewhat less, and low-density materials such as blood or air the least. Due to this it is possible to obtain a three-dimensional image of the living brain by putting together a series of X-ray scans of its horizontal sections. By means of contrast, this procedure offers good view of injury location, with somewhat poorer images of more recent injuries and those closer to the skull. Due to their similar density, it is difficult to distinguish white matter from the gray. Spatial resolution is between 0.5 and 1 cm of tissue (Gazzaniga et al., 2002).
MAGNETIC RESONANCE IMAGING (MRI) The images are obtained by measuring the energy of waves emitted by hydrogen protons after being activated in the magnetic field. Depending on the model, the magnetic field of an MR machine is more powerful than the earth’s magnetic field by about 500 times or more (0.001 vs. 0.5 tesla, or more, even up to 7 tesla). The advantage of this technique is its noninvasiveness—the brain is not exposed to any radiation—but it is relatively slow (it takes about 15 minutes to complete) and noisy. Spatial resolution is less than 1 mm (Gazzaniga et al., 2002). This method provides good images of the brain structure and structural changes (gyri and sulci, white and gray matter, injuries, tumors, or morphological changes such as one finds in Alzheimer’s disease patients), but poor functional images (Eysenck & Keane, 2000; Kalat, 1995). A better method for studying brain activity during cognitive functions is functional magnetic resonance imaging, which is based on this technique.
Research Methods
57
FUNCTIONAL MAGNETIC RESONANCE IMAGING (FMRI) This method provides a series of functional images of the brain while engaged in a particular activity. It seems that our brains resort to anaerobic metabolism to supply energy necessary for temporary increases in neuronal activity. As a result, the amount of blood oxygen in the activated area increases and the image of brain activity is based on the magnetic properties of hemoglobin, the principal oxygen carrier in the bloodstream. When the oxygen is absorbed, what remains is deoxygenated hemoglobin that is more sensitive and has more powerful magnetic properties. The ratio of oxygenated and deoxygenated hemoglobin can be measured, and this is called the blood oxygenation level dependent (BOLD) effect. During increased activity, the amount of oxygen is greater than the possibility of its absorption, so the ratio is usually in favor of oxygenated hemoglobin. This method poses minimal biological risk to the subject, but is counter indicated in patients who wear pacemakers, cochlear implants and/or other metal implants. The signal is obtained by functionally provoking changes in brain tissue (there is no injection or inhalation of radioactive or other substances); it provides anatomic and functional information about each subject, and its spatial resolution is good (1 to 2 mm, i.e., 3 mm3 of volume). The disadvantages of the technique are that it is an indirect measure of neural activity, and it cannot keep up with the pace of most cognitive processes. Temporal resolution is several seconds: the first changes are visible several seconds after the start of activity, with the greatest increase occurring about 6 to 10 seconds later (Eysenck & Keane, 2000; Gazzaniga et al., 2002; Kalat, 1995; Posner & Raichle, 1997). fMRI is the method of choice for most cognitive neuroscientists. Similarly to PET (see the following for a more detailed description), it is based on subtraction of activity patterns at rest from those recorded during a task. Its great advantage is the practically unlimited number of measurements (Demonet, 1998). Due to this, the BOLD effect may be temporally aligned with a particular event, and combined with multiple repetitions of the same event and averaging (similarly to event-related potentials, see following discussion) it may provide a clearer image of activity in particular brain regions (Gazzaniga et al., 2002). The method is extremely useful in longitudinal studies that monitor recovery from aphasia and the plasticity of language functions (Papke, Hillebrand, Knecht, Deppe, & Heindel, 2000).
RECORDING OF ACTIVITY These methods are based on the following assumption: if a particular brain region is crucial for a process (e.g., cognitive, motor, or sensory) there will be increased activity in that region when the process is initiated. The technology and techniques of recording brain electrical activity have been developing for approximately 100 years. During this time it has become possible to measure more than just electrical activity, for example, oxygen and glucose consumption, or blood flow in a particular brain region while the subject is performing a task.
58
The Cognitive Neuroscience of Human Communication
Measuring regional cerebral blood flow (rCBF) underlies many techniques, and it is based on the fact that blood flow increases in the activated parts of the nervous system. Several blood flow measurement techniques are in use. One of them involves injecting radioactive, chemically inert gas, xenon (133Xe) into the carotid artery (or inhaling it), and measuring radioactivity accumulated in different parts of ipsilateral cortex. Blood flow may be measured by (functional) infrared spectroscopy (f)NIRS that is based on the brain’s permeability to infrared rays. This method actually measures changes in the concentration of deoxygenated and oxygenated hemoglobin. It is noninvasive, fast, and relatively cheap, but its spatial resolution is poor (Cooper & Delpy, 1995). The problem common to all these techniques is that increased blood flow is present when excitatory and inhibitory neurons are active, which makes the results unreliable with respect to the nature of activity (Colier, Jaquet, van der Hoeven, Hagoort, & Oeseburg, 2000; Posner, 1995). It is also interesting, and should not be neglected when interpreting findings, that people with high blood pressure present a different picture from those whose blood pressure is normal. van der Veen et al. (2000) found their hypertonic patients showed increased bilateral blood flow during cognitive tasks that commonly cause increased activity in only one hemisphere in control subjects. In their patients, even the classical dichotomy between verbal (left-hemisphere controlled) and spatial (right-hemisphere controlled) working memory tasks was substituted by increased blood flow in the left hemisphere regardless of task.
ELECTROPHYSIOLOGICAL METHODS Some methods of measuring electrical activity register activity of large groups of cells, whereas others record action potentials of single cells. Single-Unit or Single-Cell Recording This method was developed some 40 years ago with the aim of studying the activity of single neurons. Because of its invasiveness it is typically used in laboratories on experimental animals, which is its major drawback. A tiny electrode, approximately 1/10000 mm in diameter, is inserted into the animal’s brain. This enables measurements of extracellular and intracellular potentials. It is a very sensitive method that can record electrical charges of as little as one millionth of a volt. It has helped provide invaluable information about the functioning of the primary visual cortex (Eysenck & Keane, 2000) as well as data on the auditory cortex in experimental animals. On the basis of obtained results it was possible to make retinotopic and tonotopic maps of primate brain (Gazzaniga et al., 2002; Ivry & Robertson, 1999). Its main advantage is that it can make very precise recordings of neural activity during different time periods—from very short (one second) to as long as several days. Besides its invasiveness, the fact that it offers only information about the single-cell level makes it unsuitable for studying larger brain
Research Methods
59
areas. More recent techniques make possible the recording of activity of several neurons at a time (multiple-cell recording), which takes care of at least one of the disadvantages. Electroencephalography (EEG) This method is used to continuously measure brain activity through the skull. It reflects the totality of electrical events in the head, including action potentials and postsynaptic potentials generated by neurons, as well as electrical signals from the skin, muscles, blood, and the eyes. It is suitable for determining the functional state of the brain, but the results can hardly be associated with the activity of individual areas. Event-Related Potentials (ERP) In this context we talk of late, so-called cognitive, event-related potentials. The procedure is based on EEG, but the potentials are evoked by a specific stimulus (hence the alternative term evoked potentials) and extracted from the EEG record by signal averaging. Electrodes are attached to the scalp before the procedure. The stimuli are presented and ensuing electrical activity is recorded. Underlying the procedure is the fact that EEG contains all the changes in potential that are time-locked with the specific event, such as stimulus presentation, or movement, that become visible by signal averaging over a large number of these events. In the averaging process, the noise (the uninformative part of the signal) is cancelled out and the relevant signal emerges. Ten to 100 repetitions of the same stimulus are usually sufficient to produce a reliable potential that reflects characteristics of the studied brain and stimulus. Each wave is determined by latency and polarity (positivity or negativity). Evoked potentials recorded in the hemisphere contralateral to the auditory stimulus are generally stronger than those recorded in the ipsilateral hemisphere. It has also provided insights into the time course of processing, as well as detailed analysis of the influence of experimental variables. It is possible to compare the effects of two experimental conditions on early or late components of the response: the early component is taken to represent sensory events and the late component is attributed to cognitive ones. For example, P300 is a positive peak that occurs about 300 ms after the stimulus only if the stimulus has some relevance for the subject. Studies with language stimuli have revealed that N400 (a negative peak occurring approximately 400 ms after the stimulus onset) reflects semantic processes (Hagoort & Brown, 1995). It is possible, however, that a large portion of cellular activity relevant for behavior is not manifested as action potentials. A serious disadvantage of this method is its poor spatial resolution (i.e., impossibility of precise spatial localization of the studied activity), and it is also unsuitable for basic cognitive processes. Its principal advantage is good temporal resolution.
60
The Cognitive Neuroscience of Human Communication
Compared with the early evoked potentials (visual, auditory, somatosensory), cognitive potentials are to a certain extent stimulus-independent. This means: (a) they do not necessarily occur every time a stimulus reaches the threshold, even with intact sensory pathways; and (b) they may occur without stimulation if the stimulus is expected. Different modalities can elicit almost identical cognitive evoked potentials. The response is affected by several factors: probability of stimulus occurrence, psychiatric status, selective attention and arousal, habituation, maturity of the central nervous system, aging, and task relevance, to name just a few (Dabi´c-Jefti´c & Mikula, 1994; Praamstra & Stegeman, 1993; Stenberg, Lindgren, Johansson, Olsson, & Rosen, 2000). The response amplitude may be affected by type of stimulus, and even by such fine differences as noun categories. Müller and Kutas (1996) have found that personal names have the same latency (N1 and P2) but greater response amplitude than common nouns, and that the subject’s own name, compared with other names elicits an additional response with a peak latency of about 400 ms over parieto-central electrodes and another even later response between 500 and 800 ms over left lateral frontal points. A word of caution concerning some inconsistencies in presenting the evoked response test results is necessary. Response latency can be expressed in hundreds or single digits, so P1 designates the first positive peak, and P100 a positive peak of 100 ms latency. They may correspond to one another, but not necessarily. Sometimes, both alternatives are present in the same figure. Single digits are used to mark the peaks of same polarity from 1 on, which means that we can have N1 followed by P1, N2, P2, N3, and so on. Additional confusion may be caused by the fact that some authors turn the negative peaks upward and positive downward, whereas the others do not. Hagoort and Brown (1995) point out, as well as many other authors, that a complete understanding of the neurobiological foundations of language requires the combination of several different methods. Cortical Cartography Recently, new combinations of methods have been introduced. One of these techniques is cortical cartography that combines mapping, event-related potentials and fMRI. It provides three-dimensional images of the skull and brain. Temporal aspects may be recorded by 3D movies. Magnetoencephalography (MEG) Neurons maintain electrical potentials along the external membrane, and when electrical potentials of many neurons are summed up, it is possible to record them encephalographically. These currents generate magnetic fields that may be recorded on the skull surface providing us with a magnetoencephalogram. This method allows precise (to the order of 1 ms) monitoring of the temporal structure of cognitive processes. It is considered a good direct measure of neural activity, as opposed to fMRI or PET, as the latter two methods reflect blood flow, which is an
Research Methods
61
indirect measure of neural activity (Wang, Sakuma, & Kakigi, 2000). The advantage of MEG over encephalography is the insensitivity of magnetic potentials to the cranial bones and other tissues. Although its spatial resolution is superior to that of encephalography, it still provides relatively rough, hence unsatisfactory, anatomic and structural data. Another disadvantage of MEG is possible interference of other, task-independent sources of magnetism. Although MEG is very sensitive to tangential projections from sulcal structures, it is not as sensitive to radial projections from gyral structures (Boatman, personal communication).
RADIOISOTOPIC METHODS Positron Emission Tomography (PET) This technique provides information about metabolic activity of the brain. Metabolism-relevant substances (glucose or oxygen) are marked by a low dose of radiation. The patient is injected with radioactive isotope 15O (tracer), or the isotope is inhaled. Approximately one minute later the radioactive water accumulates in the brain proportional to the local blood flow. During radioactive decay, the tracer emits positrons. As a consequence, an activity performed by the patient (e.g., reading) will reveal parts of the brain that are the most active at that moment. A special apparatus records different levels of radioactivity (i.e., metabolic activity), which results in brain images. The final record is not the absolute level of activity at the moment of task performance, but the result of subtraction of activity patterns at rest from those recorded during a task (similarly to fMRI). Regions with the greatest difference after subtraction are interpreted as areas of highest activity during a function. After 10 minutes no radioactivity can be detected any longer. One advantage of this method is its applicability to large numbers of different tasks, and also its relatively good spatial resolution (3 to 4 mm, or 5 to 10 mm3 of volume). Its disadvantages are technical difficulties in application, high price, and invasiveness in exposing the subjects to some (albeit small) doses of radioactivity, in addition to limited temporal resolution, that can be up to 60 s (Eysenck & Keane, 2000; Posner, 1993, 1995). Cognitive processes take one second or less, so it is obvious that this technique will not reveal everything that goes on during testing. Demonet (1998) suggests that it be combined with the results of responsetime measurements. Interpretation of results is additionally complicated by the fact that even the subjects’ anticipation of what they are about to say changes the final picture of which parts of the nervous system are involved or particularly active. It is also debatable to what extent increased isotope concentration reflects faithfully changes in the neural activity: there are indications that it might be true only for early stages of processing (Eysenck & Keane, 2000). Great intersubject variability precludes attributing specific language processes to limited brain regions (Fromkin, 1995). Again, it needs to be pointed out that, just as with fMRI, for example, increased activity in a certain area during a task does not necessarily mean that the area plays a crucial role in the task at hand.
62
The Cognitive Neuroscience of Human Communication
Single-Photon Emission Computed Tomography (SPECT) This scanning technique uses computerized tomography technology, but instead of gamma rays, individual photons emitted by an externally applied substance are recorded. Its resolution is worse than PETs, but due to its lower price it is more suitable for wider use in clinical practice.
ULTRASOUND METHODS Functional Transcranial Doppler Ultrasonography (fTCD) This method is used to measure increased blood flow related to an event. It has been proven reliable and valid in lateralization studies. The results are comparable to fMRI and Wada test results, and they are replicable (Anneken et al., 2000; Buchinger et al., 2000; Lohmann et al., 2000; Zavoreo & Demarin, 2004).
SUMMARY When evaluating methods of brain activity imaging, one must consider whether the information is represented in the brain at the level of the individual cell or at the level of cell groups. If the former is true, changes in the electrical activity of each individual cell may be associated with the function of the system that is based on it. In the latter case, the situation is much more complex. Another issue is the relevance of the recorded change to the brain. In other words, are the other parts of the brain “aware” of the activity? If they are, how do they interpret this increased activity: as relevant for further action or as a sign of some random and/ or functionally irrelevant activity. Combining several methods may reveal whether the recorded connections are causal or just correlational, possible differences in the timing of activations in different locations in the brain as well as their temporal organization. In tests where the subjects were asked to recognize the objects by feeling them, fMRI revealed that even with their eyes closed their visual areas were activated. Combination with TMS showed that activation of visual representations is necessary for successful recognition by touch (Gazzaniga et al., 2002). Most neuroimaging studies report average results of group studies, and they have to be interpreted in the light of the discussion at the beginning of this chapter (on the advantages and drawbacks of single case studies and group studies). Although general principles of organization and functioning of the nervous system are common to humans, individual variations are more likely than uniformity. Methods involving recording of brain activity may lead to results that are different from the ones obtained by cortical stimulation methods, or by observation of clinical cases. Eysenck and Keane (2000) describe amnesiacs whose hippocampi were key structures for declarative or explicit memory. On the other hand, most PET and fMRI studies failed to replicate the findings of high activation in the same structure in healthy subjects during tasks that rely on declarative memory. One of the reasons may be that PET and fMRI show all areas that are active during a task,
Research Methods
63
not just the crucial ones. Furthermore, damage to a part of the brain will result in functional disorder when the affected region is relevant for a particular function, and it may be that functional reorganization of neighboring areas that might take over has not yet taken place. Furthermore, the subjects’ ages vary widely across studies: PET and fMRI methods involve mostly young healthy subjects, whereas in brain-injured groups there are older subjects. Increased sophistication and sensitivity of activity recording methods is a mixed blessing: on the one hand, it is their great advantage, and on the other, it makes interpretation much more complicated because registering even the smallest levels of activity frequently makes it difficult to decide which of the recorded active locations is really crucial for the function studied. Another problem is the fact that various techniques may be differently sensitive to the many features of the same function and its representation (Eysenck & Keane, 2000; Gordon et al., 1994). In addition to the recommendation to combine different methods, it seems increasingly necessary to include different tasks that examine the same function (e.g., different language tasks studied by several methods). This is known as combined tasks analysis (CTA). Obviously, if a certain structure proves to be relevant for the same function on several different tasks, it will enable us to make more valid inferences about the relationship between structure and function (Ramsey, Sommer, & Kahn, 2000; Rutten, van Rijen, van Veelen, & Ramsey, 2000; Sternberg, 2003).
BEHAVIORAL METHODS Since the brain is the organ of behavior, its functions can be studied by studying behavior, as it reflects psychological processes (Legg, 1989). The subjects are presented with different cognitive, sensory, or motor tasks, their solution is associated with physiological manipulations, and on the basis of the obtained results, functions of particular brain areas are defined. Complex psychological processes may be broken down into stages, and researchers can look for associations between individual stages and corresponding brain areas. Numerous psychological tests include general IQ tests, memory tests, tests of perceptual and motor functions, and various language tests, such as Wechsler’s intelligence scale (WAIS), Raven’s matrices, and so forth. They can be implemented with a healthy population as well as clinical patients. It is important to have neuropsychological assessments for those who have suffered brain injury— regardless of etiology—for the purpose of choosing a therapy and monitoring recovery. However, clinical assessments are limited by the type of damage, in other words, the patients’ ability to understand directions and perform the task. Obviously, someone with a paralyzed right arm as a consequence of stroke will not be able to perform motor tasks that require the use of the right hand. Response time and the percentage of correct answers (or errors) are the most frequently used indirect measures of task difficulty. However, we must keep in mind the limitations of these measures. For example, the time it takes to give a motor response (e.g., raising one’s hand or pressing a button) to a certain cognitive task (e.g., finding the target letter in visual tests, or determining rhyming status in auditory tests),
64
The Cognitive Neuroscience of Human Communication
includes time needed to make the decision and the time needed for the actual motor action. Technical limitations of equipment are, of course, additional factors. The following section includes brief descriptions of the most frequent behavioral experimental methods used in studying language functions. Paper-and-Pencil Tests These are all the tests (including language) in which the tasks are presented graphically (as text or images), and the subjects are expected to respond by writing, drawing, or marking. Word Association Tests These tests are usually designed in a way that the subjects are presented with one word at a time, and are expected to provide the first word that comes to mind. Variations of the test include providing superordinate terms, synonyms, antonyms, additional members of the same category, and so forth. Another subtype of this test is the so-called word fluency test in which the subjects are asked to provide as many words that begin with the target letter or sound as they can in a limited period of time. Stroop Test The original Stroop test involved naming the colors in two ways: as blots of different colors and as words designating colors that were printed in letters of (in)congruous colors (e.g., the word blue printed in blue or red ink). Response times were longer to incongruous letter–color combinations due to the disagreement between the two bits of information. In a wider sense, the Stroop effect may be elicited by other stimulus combinations (Preston & Lambert, 1969). The Wisconsin Card Sorting Test (WCST) Different numbers of symbols varying in color and form are presented on cards. The subject picks up one card at a time and places it on one of the several piles in front of him/her. On the basis of the examiner’s reactions, the subject concludes, by trial-and-error, whether the cards should be sorted according to color, shape, or number of symbols. After 10 successful consecutive attempts at correctly sorting cards, the examiner changes the sorting criterion without warning. The subject is supposed to discover the new criterion. Failure on this test—most frequently manifested as perseveration, or insistence on sorting according to the first adopted criterion—reveals disorders of frontal lobe functions. Priming and Interference In these tests one stimulus is used to facilitate or interfere with another that is related to the first phonologically, syntactically, lexically, or semantically. The
Research Methods
65
degree of facilitation or interference (measured by response times to the second stimulus) offers information about the organization and relationship between the first stimulus (prime) and the second stimulus (target). For example, in a visual task in which a subject is asked whether the presented letter was A or V, the response will be faster if before the target A, an a (prime) rather than v is presented briefly. These experiments may reveal temporal sequence and degree of interference between the two competing cognitive processes or representations of different stimuli, but do not tell us much about the functioning of these processes and representations, or about their interference or facilitation (Westbury, 1998). Shadowing In auditory tests, the subjects are presented with different stimuli to the left and the right earphone. They are asked to listen carefully and repeat simultaneously with the speaker the message heard in one of the ears. When a sudden change is made in the message presented to the unattended ear (an unexpected sound or switch of the speaker from male to female), the subjects typically start to hesitate or lose continuity. This procedure may reveal language deficits even when they are not noticeable in everyday communication. The test may also be visual, in which case two stimuli are presented in different locations in space, or in a bimodal manner (e.g., one stimulus is auditory and the other visual). It has been shown that the success of shadowing may be affected by several factors. For example, meaningful stimuli are more easily shadowed than meaningless ones. Shadowing is more successful if the stimuli are further apart in space, if they are presented at different pitch (e.g., male and female voices), and if they are presented at different rates, and so forth (Coren & Ward, 1989). Gating This method was introduced by Grosjean (1980). A recorded speech segment (typically one sentence) is spliced into shorter segments (usually about 20 ms) by sound-editing computer software. The subjects are presented with stimuli of progressively longer duration (each time one 20-ms segment is added, yielding stimuli that are, for example, 60, 80, 100, and 120 ms long). The subjects’ task is identical at each presentation, or gate: “Say what you heard and/or what you think the entire sentence was.” The responses of all subjects constitute a cohort. The subjects are frequently required to grade the certainty of their response (Erdeljac & Mildner, 1999). The procedure is repeated until the subjects guess the entire sentence, or the recorded material comes to an end. Comparison of the subjects’ responses and the stimuli, in addition to an analysis of their progression, allows inferences about how much and what kind of information is needed to reach the correct response. The number of activated members of the cohort, and the frequency of their occurrence among subjects, as well as the reduction of their number and the final single response are considered indicators of organization, activation, and functioning of the mental lexicon.
66
The Cognitive Neuroscience of Human Communication
Dichotic Listening This is one of the most frequently used behavioral lateralization tests with auditory stimuli. In the original version of the test (Kimura, 1961a, 1961b; 1967) the subject hears through the headphones three numbers presented one after another to the left ear, and simultaneously, three different numbers presented to the right ear. During a very brief pause the subject is asked to report as many of the six numbers as possible. Then another triad is presented and responded to, and so on. Some versions of the test include manipulating attention to one ear or the other (Hugdahl & Anderson, 1986; Hugdahl, Anderson, Asbjornsen, & Dalen, 1990; Mildner, 1999; Mildner & Golubi´c, 2003; Mondor & Bryden, 1992; Obrzut, Boliek, & Obrzut, 1986). Laterality indices, calculated from the subjects’ responses, are taken as indicators of ear advantage, i.e., contralateral hemisphere dominance. Dichotic tests in a wider sense include all auditory tests in which two different stimuli are presented to the two ears at the same time—these can be two signals, or a signal can be presented to one ear and noise to the other. If the untested ear is continuously presented with white noise or murmur, the attention is directed toward the tested ear. The more the distracting stimulus differs from the test stimulus, the greater attention is directed toward the tested ear (Sparks & Geschwind, 1968). For example, if the test stimulus is speech, white noise will direct attention to the tested ear more than if it were speech-based noise (murmur or backward speech). As one channel is loaded more, the response to stimuli in the other one is improved, so the overall response is actually constant (Berlin, 1977). This is equally applicable to healthy subjects and commissurotomized patients. Such tests have shown that the ear contralateral to the speech-dominant hemisphere is superior to the other ear. In a great majority of people the superior ear is right, and the speech-dominant hemisphere is left—phenomena supported by independent sodium amytal (Wada) tests. Different varieties of the dichotic listening tests have revealed right-ear advantage (REA), with a corresponding lefthemisphere dominance for digits, words, nonsense syllables, formant transitions, backward speech, Morse code, difficult rhythms, phonological tone, and many other speech and language-related stimuli. However, test–retest procedures reveal that only about 50% of the subjects exhibit consistent statistically significant REA for dichotically presented speech sounds. Left-ear advantage (LEA), with a corresponding right-hemisphere dominance, was reported for tunes, music chords, environmental sounds, emotional sounds, tones that are processed independently of the language content, pitch perception, and similar stimuli. No ear advantage, or inconsistent ear advantage, was reported for vowels, isolated fricatives, rhythms, nonmelodic humming (Kolb & Whishaw, 1996), and also for linguistic prosody (Mildner, 2004). Ivry and Robertson (1999) report results of studies that claim auditory information is indeed lateralized; however, it is not with respect to ear, but with respect to the side of space from whence the sound comes. Accordingly, each ear may have projections in both hemispheres, but sound input from one side of the space will project to the contralateral hemisphere: the sound coming from the right will
Research Methods
67
be projected to the left hemisphere, and the sound coming from the left will be projected to the right hemisphere, along both ipsi- and contralateral pathways. Divided Visual Field This is one of the most frequently used behavioral lateralization tests with visual stimuli. It is based on the anatomic and physiological specificity of the visual pathways: all stimuli from the left visual field reach the right cerebral hemisphere, and all stimuli from the right visual field reach the left hemisphere. Since the hemispheres communicate via commissural connections, primarily the corpus callosum, visual information from the two hemispheres is combined in a matter of tens of milliseconds. Nevertheless, it is possible to design the experiment so that very brief stimulation of one or the other visual field by instantaneous presentation of stimuli to the left or to the right of the fixation point results in response-time differences depending on whether the stimuli were verbal or nonverbal and whether they were presented to the left or the right visual field. Some 60 to 70% of right-handers exhibit right-visual-field advantage (RVFA), in other words, left-hemisphere dominance for language material but not for nonlanguage (Pinel, 2005; Tadinac-Babi´c, 1994, 1999). Since it takes the eye about 150 to 200 ms to shift from the fixation point, in order to get the lateralization effect the stimuli must be kept shorter than that (Gazzaniga et al., 2002). Dual Tasks These tasks may tell us something about possible common neural substrates of speech and language functioning and motor activity. The classical experiments of Kinsbourne and others (Chang & Hammong, 1987; Cherry & Kee, 1991; Hicks, 1975; Kinsbourne & Cook, 1971; Singh, 1990; Tsunoda, 1975) have shown that the results of simultaneous tasks that rely on the same neural structures differ from those where processing is controlled from different brain regions. The test typically includes two tasks that have to be carried out at the same time, and it attempts to test the relationship between the two tasks (i.e., between their cerebral representations). It is expected that success on one or both tasks will be poorer if performance requires the activation of identical areas. For example, it has been shown in right-handers that language processing has a significantly greater negative effect on the accuracy and speed of simultaneous motor activity of the right hand than of the left, and that the effect becomes larger as the language processing complexity increases (Mildner, 2000). The same study revealed that even designating one of the two tasks as primary, and purposefully directing attention to it, does not necessarily result in its unhindered performance compared with the secondary one. The two tasks may facilitate or inhibit each other—simultaneity does not necessarily have to cause interference. It is not quite clear to what extent the results of dual tasks are affected by attention mechanisms and the fact that they share a common neural substrate. Dubost, Beauchet, Najafi, Aminian, and Mourey (2002) studied the relationship
68
The Cognitive Neuroscience of Human Communication
between walking and two different cognitive tasks—mental arithmetic and speech production. When performed simultaneously, both cognitive tasks slow down the person’s pace and shorten the step, but the walking efficacy is preserved (the subjects’ pace is still even). On the other hand, walking affects each of the cognitive tasks differently. Whereas mental arithmetic (working memory) was not affected by walking, word generation (semantic memory) was facilitated by it (expressed as the greater number of words produced while walking than in the speech production task alone).
SUMMARY Some other lateralization tests look at gesticulations as motor accompaniment and reflection of language functions. Kimura (1973b, 1973c) found that people gesticulate more with the hand contralateral to the language-dominant hemisphere, and that hand gestures accompany linguistic expression. Video recordings of people talking revealed that right-handers make bigger movements with the right side of their mouth than with the left. This was true not only for speech production but also of all complex nonverbal mouth movements. Careful observations have also revealed that right-handers turn their heads and/or eyes to the right while solving verbal problems, but upward and/or to the left when solving numerical or spatial problems (Pinel, 2005). According to some authors, writing posture is indicative of the language-dominant hemisphere, regardless of whether the person is left-handed or right-handed (Levy & Reid, 1976). Namely, if the writing hand wrist is above the finger level, with the hand pointing downward, the language-dominant hemisphere is ipsilateral to the writing hand. If the person writes with the fingers pointing upward (as most people do) the contralateral hemisphere is language-dominant. The problem of studying lateralization by means of behavioral methods is that in healthy subjects, as opposed to the split-brain patients, we cannot be certain whether the response to the lateralized stimulus is the result of processing in the contralateral hemisphere or whether it reflects joint efforts of both hemispheres after callosal transfer. The most reliable results are obtained if different methods are combined, for example, behavioral methods, with noninvasive activity recording methods, especially if they are compared with disordered functions in clinical patients and the functioning of split-brain patients.
APHASIA TEST BATTERIES One large subset of neuropsychological methods for studying speech and language functions are tests for aphasics. These are used to test different levels of language functioning—from the most general division into production and perception to tests that tap into specific levels of processing: phonological, syntactic, semantic, and so forth. One of the most widely known and used is the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983). This battery comprises 34 subtests for 12 language areas, two additional tests of music abilities and seven tests for the assessment of parietal lobes functions. Grammatical-syntactic tests,
Research Methods
69
tests of nonverbal functions, and profile assessment scales are usually added as well. Many other aphasia tests are in use too, but they all assess four basic language skills, all or some of which may be disordered in aphasia: spontaneous speech, repetition, comprehension, and naming (Juncos-Rabadan, 1994; Turdiu, 1990; Vuleti´c, 1987).
Central 5 The Nervous System Principles, Theories and Models of Structure, Development and Functioning On the basis of systematic characteristics, the principles of structure, development, and functioning of the nervous system are established. These principles serve as starting point for the design of theories and models. Empirical and experimental tests and evaluations of theories and models complement existing knowledge. This, in turn, enables us to develop new principles or modify them in such a way that we come to have new models or new versions of the old ones. Due to this obvious interactivity, it seems reasonable to discuss the principles of structure and functioning of the central nervous system in the same chapter with the theories and models that refer to them. The four principles that are discussed in this chapter are hierarchical organization, parallel processing, plasticity, and lateralization of functions.
PRINCIPLES HIERARCHICAL ORGANIZATION One of the most important principles of the nervous system structure and functioning is its hierarchical organization (Fischbach, 1992; Kalat, 1995; Kolb & Whishaw, 1996; Webster, 1995). Higher levels provide greater precision. Neurons respond to increasingly abstract aspects of complex stimuli as the distance, measured by the number of synapses from the source, increases. Examples of this principle are reflexes. They are quick precisely because the reflex arc, also measured by the number of synapses, is short. For instance, the cochleo-stapedial reflex (i.e., the response of the stapedius muscle in the middle ear to high sound intensity) is initiated at the level of the superior olive rather than completing the entire auditory pathway that ends in the auditory cortex. On the other hand, phoneme discrimination takes place in the medial geniculate and cortex after the stimulus has passed through a much longer part of the auditory pathway (cochlear nuclei, olivary complex, inferior colliculi). 71
72
The Cognitive Neuroscience of Human Communication
PARALLEL PROCESSING Another important principle of the functioning of the nervous system is that bits of information do not travel along a single pathway. Various aspects of the same sensation are processed in parallel ways (Fischbach, 1992; Kalat, 1995; Kolb & Whishaw, 1996; Webster, 1995). Zeki (1992) described examples of visual stimuli. He found that in the brain there is a division of labor manifested in anatomically discrete cortical regions and subregions specialized for different visual functions. There are four parallel systems in charge of different visual properties: one for movement, one for color, and two for shape, of which one is closely related to color and the other is independent. These specialized areas are interrelated directly or through other areas. Signal integration in the parietal or temporal lobe proceeds through local circuits. Some of these areas are fully developed at birth (the so-called V1 region—where the entire retinal map is located—is synonymous with the primary visual area in traditional terminology). The development of other areas is completed after birth, which indicates that their maturation depends on experience. Auditory stimuli behave in a similar manner. Sound localization depends on interaural differences in phase and amplitude, and these differences are processed along several paths in three synaptic relays in the brain. Phase differences provide information about the location on the horizontal plane, whereas amplitude differences provide information about the location on the vertical plane. In humans, as little as 3º shift on the horizontal plane (i.e., a temporal difference of only several μs in the activation of the dendrites of the left and right cochlear nuclei) is sufficient to identify the difference in sound source location (Kalat, 1995; Webster, 1995). This type of parallel processing is probably characteristic of other sensory systems, association areas, and motor pathways as well. Primates have about a dozen different cortical maps for visual stimuli, about six for auditory (Lieberman, 1991), and four for the somatosensory ones (Kalat, 1995).
PLASTICITY The third principle of the structure, development, and functioning of the central nervous system—primarily the brain—is its plasticity. This is the ability of the central nervous system to adapt or change under the influence of exogenous or endogenous factors. There are three main types of plasticity: (a) changes in connections that occur after birth, as a consequence of interactions with the environment; (b) changes in connections that occur after lesions; and (c) changes induced by learning and experience. The term developmental plasticity that is often found in literature refers to the first, and partly the third type. Plasticity is closely related to the notion of functional reorganization—a phenomenon referring to the change in (cerebral) representation of a function or sensory modality. The change may be caused by lesions or natural aging processes, but also by practice and learning. Functional reorganization may be manifested as a change in the size of the cortical area where the particular function or sensory organ is represented, or as a different location of representations. For example, if one finger is amputated, the neighboring fingers will be represented on a greater area than before, at the
The Central Nervous System
73
expense of the amputated finger. Shifting of speech representation to the opposite hemisphere is sometimes found in patients with brain injuries in the speech-dominant hemisphere. Without plasticity, functional reorganization would be impossible. This topic is discussed in more detail in chapter 7, this volume.
LATERALIZATION OF FUNCTIONS The fourth important principle of brain functioning is lateralization of functions. Since the brain appears to be anatomically symmetrical, early works on the biological foundations of behavior assumed that functions were symmetrically organized as well. However, more detailed studies have revealed that the brain is not exactly symmetrical and that the two hemispheres function differently; in other words, they are both anatomically and functionally asymmetrical. This was supported by findings that lesions to homotopic regions in the left and the right hemisphere have different consequences for behavior and functioning in general. Hence the term lateralization of functions. After many repeated unsuccessful attempts to associate the site of lesion with a function disorder, the localizationist approach of the late 18th century was replaced by a holistic one—understanding and describing cerebral functions as a process in which each behavior requires interaction of many different structures as a whole. Therefore, the first attempts to relate language deficits to left-hemisphere damage went largely unnoticed. Broca emphasized the difference between anterior and posterior injuries, but only as Wernicke’s findings on the importance of the left hemisphere for language functioning emerged. As a consequence, the left hemisphere was called the dominant or principal hemisphere, whereas the right hemisphere was treated as less important or minor. In this volume, as indeed in most contemporary literature, the term dominant may refer equally to both hemispheres. It is used in the sense of being primarily involved in the processing of a function, not in the sense of being more important in general. The few authors who studied right-hemisphere functions at the turn of the 19th century (e.g., Hughlings-Jackson and Balint) found that damage to that hemisphere most frequently resulted in deficits in spatial functioning (perception of objects in space and orientation in space). This was the foundation of the propositions that language and spatial functioning are two basic human cognitive abilities, each represented in a hemisphere of its own. Primary sensory processing and motor functioning are bilaterally represented. Lateralization occurs in higher, more complex functions, such as language, spatial processing, music, and so forth, so that at the highest levels of processing the two hemispheres have different roles. Lateralization of function is discussed in more detail in a separate chapter.
THEORIES AND MODELS The brain consists of numerous cortical areas and subcortical nuclei that differ in afferent and efferent connections, appearance under the microscope, and
74
The Cognitive Neuroscience of Human Communication
characteristics of their physiological reactions. It is, therefore, logical that their contributions to psychological processes are different as well. Although phrenology turned out to be unacceptable, anatomical studies have shown that some areas have direct connections with the motor system, others with the sensory, and still others are to be found between motor and sensory areas. Accordingly, some areas are closely connected with a particular sensory modality, others with movements, whereas others exhibit complex response properties that are neither exclusively motor, nor solely sensory. The parts that are most closely connected with specific functions are those having the closest connections with the sensory organs. Experimental and clinical data have shown that damage to different parts of the brain may have considerable consequences for behavior and cognitive functions. Theories and models of central nervous system functioning, especially of the brain, attempt to address several key issues: (a) where does a particular function take place? (b) What is its course? and (c) how autonomous is it? The processing may proceed in parallel (more aspects or levels are processed at the same time) or serially (one at a time); processing may be one-way (bottom-up or top-down) or there may be more or less interaction (between levels or within one level), with total or partial autonomy of processing at each level. Processing may go on in separate and specific brain structures or it may be more diffuse. Active theories posit the existence of very complex transformations of neural information and programs that may be compared, and on the basis of that comparison perceptive decisions may be made. Each level is connected to other levels and processing proceeds in a parallel manner in all of them—there is no necessary order of levels of processing (e.g., acoustic material serves as a basis for certain hypotheses, but at the same time as a test of the existing ones). Passive theories are as a rule one-way—processing is only bottom-up or top-down, with no interaction among levels. Obviously, each model may belong to several categories (e.g., localistic and serial). Furthermore, not all models deal with all or the same components of language functioning—some are limited to language production, others to perception or word recognition. In this chapter, I discuss some of the most influential and most widely quoted general theories and models of structure and functioning of the central nervous system. This includes the application to language functioning, as well as some specifically linguistic theories.
PARALLEL OR SERIAL PROCESSING? The first models of sensory system functioning were serial. According to them, the information flowed between components, but only through a single channel. Later, it became acceptable to talk about parallel models that were believed to represent sensory functions better than the serial ones. In parallel models, the information may be relayed among components through different channels. Although there is ample evidence for parallel processing in sensory systems, the nature of language processing is not so easily agreed upon (Gazzaniga, Ivry, & Mangun, 2002). Examples of serial processing are Levelt’s (1989) model of
The Central Nervous System
75
speech production and the Wernicke–Geschwind model of language functioning. These models are discussed in more detail later. Serial or cascade models were supported and discussed, among others, by Bentin, Mouchetant-Rostaing, Giard, Echallier, and Pernier (1999) and Schmitt, Muente, and Kutas (2000). On the other hand, Starreveld (2000) does not agree with the serial model of speech production.
LOCALISTIC MODELS A principal characteristic of localistic models is establishing direct connections between structure, condition, and functioning of a particular brain region with behavior. According to these models, focal injury to some brain region will result in specific patterns of cognitive deficits, because brain functions are assumed to be localized. An obvious drawback of these models is the inability to deal with cases of disordered functions without damage to “responsible” brain regions, and with cases of preserved function in spite of damage to corresponding regions. The models are acceptable to the extent that it has been shown that particular functions are in fact localized within specific brain regions, with the higher functions (e.g., attention and thought) spread over greater areas of the cortex and including subcortical parts. Some advocates of these models were Broca, who localized the speech production center in the left frontal lobe; Wernicke, who localized the speech perception center in the left temporal lobe; and Hughlings–Jackson, who associated motor activity with the posterior parts of the frontal lobe, on the basis of postmortem examinations of his epileptic patients. One of the first models was Lichtheim’s in 1885. It consisted of three main parts and connections among them. They were: (a) Wernicke’s area with the phonological store, (b) Broca’s area with programs for planning and execution of speech, and (c) an area where concepts were stored. Lichtheim’s was a classic localizationist approach, elaborated later by Geschwind and Galaburda (Galaburda, 1994; Gazzaniga et al., 2002). On the basis of the properties of the Hebbian synapse, Pulvermüller (1992) calls functional neuronal groups cell assemblies. Turner and Frackowiak (1995) call these restricted areas cortical fields (each field comprises about 106 to 107 neurons) and consider them to be the substrates of specific processes. However, these cannot be counted as true representatives of localistic theories because, in essence, their approach relies on neural networks. For some 30 years now, all discussions about localization of language functions have included the Wernicke–Geschwind model. In spite of its inconsistencies and imperfections, over the years the model has instigated research and discussions about cortical representation of language. Before the model was proposed, there had been attempts to describe localization and lateralization of language functions on the basis of deficits in language functions (chap. 3 on the history of neurolinguistics). These localistic theories lost ground at the beginning of the 20th century to holistic theories that see the brain functions as a whole. Geschwind incorporated these ideas into the so-called Wernicke–Geschwind model based on a theory stemming from the belief that damage to language areas in the left hemisphere and/or connections between them caused aphasia and other language disorders.
76
The Cognitive Neuroscience of Human Communication Primary motor area
4 8
3 1 2
6
5
7
Angular gyrus 19
Broca’s area
10
41
9
46 45
18
44
47
22
11
21 38 Fasciculus arcuatus
FIGURE 5.1.
39
17
42
Primary visual area
37 Wernicke’s area
20 Primary auditory area
Wernicke–Geschwind model of speech and language.
Wernicke–Geschwind Model The Wernicke–Geschwind model consists of seven components: primary visual cortex, primary auditory cortex, primary motor cortex, angular gyrus, arcuate fasciculus, Wernicke’s area, and Broca’s area. All the components are located in the left hemisphere (primarily in the precentral and the postcentral areas, immediately anterior and posterior to the central sulcus, where face representations are located). The model is shown in Figure 5.1. When we read aloud, the signal arriving in the primary visual cortex is relayed to the angular gyrus, which transforms the visual form of the word into its auditory code and transfers it to Wernicke’s area where comprehension takes place. Wernicke’s area initiates appropriate responses in the arcuate fasciculus. In Broca’s area the signal activates the appropriate articulation program that activates neurons of face and mouth in the primary motor cortex, and finally, muscles that participate in word articulation. This model belongs to the group of localistic and serial models, because it involves a series of responses that occur in linear succession, and this is one of its most serious drawbacks. Any model that involves any number of highly localized cortical centers connected linearly, is not a true description of actual brain functioning and cannot be applied to complex cognitive processes. Broca’s and Wernicke’s aphasias are, in fact, primarily but not exclusively expressive and sensory, respectively. Moreover, it has been found that these types of aphasia do not necessarily ensue from damage to Broca’s and Wernicke’s areas. The model was not supported by the examples of patients who had, for medical reasons, undergone surgical removal of parts of the cortex that are supposedly
The Central Nervous System
77
components of the model. For example, if the entire Broca’s area was removed, but with little or no surrounding tissue, the patients had no speech difficulties. Similar examples from clinical practice may be found for other components of the model as well. The advocates of the model explain this phenomenon by the fact that the patients had longstanding neurological disorders before the surgical procedure, which might have resulted in some functional reorganization prior to surgery. However, the model is not supported by cases of sudden brain injury caused by trauma or illness. Only extensive damage involving three lobes caused difficulties proposed by the model (Pinel, 2005). It has also been consistently found that extensive damage to the anterior regions is associated with articulation problems as opposed to equally extensive damage to posterior regions. CAT scans have shown that damage limited to Broca’s and Wernicke’s areas does not cause language problems, and that in all patients presenting with language difficulties there had been some subcortical damage. All studies confirmed a greater probability that extensive damage to left anterior parts would cause difficulties in language expression, as opposed to damage to posterior parts, and that comprehension difficulties would be caused by extensive posterior damage, rather than by anterior damage. Additional explanations were supplied by Kimura (1983; see also chapter 2, this volume) and Hier, Yoon, Mohr, Price, and Wolf (1994). Cortical stimulation of large (widely distributed) numbers of sites in the left hemisphere cause speech arrest or selective speech deficits (inability to name objects, incorrect naming, counting, etc.). Aphasic symptoms occurred very rarely in response to right-hemisphere stimulation. Contrary to the model’s predictions, cortical areas that were affected by stimulation in terms of language functions were found well outside the language areas defined by the model. Another important result emerged from these studies: individual differences were substantial among subjects in the organization of language abilities. Ojemann (as cited in Pinel, 2005) proposed that the language cortex is organized like a mosaic, in which columns of tissue responsible for a certain function are widely distributed throughout the entire cortical language area. He also concluded, on the basis of results from a series of recording and stimulation studies (Ojemann, 1994), that there were separate systems of neurons and basic areas for additional language dimensions. Different components of each system obviously work in parallel. The neurobiological substrate of language probably involves (mainly) separate processing of each language dimension, but with the systems related to each dimension active at the same time. Human language, like so many other functions in the primate cortex, is probably processed in multiple parallel distributed systems. Scanning techniques have revealed increased activity during speech in the left frontal, temporal and parietal lobes, as well as in the left thalamus and basal ganglia (Kalat, 1995). Gordon et al. (1994) present evidence that confirm the existence of fine subdivisions in representation of language and related functions and conclude that a considerable number of these subdivisions may be controlled by higher mechanisms. Evidence may be found in the speech perception system, as well as in verbal and visual semantic systems and their connections to other language subsystems,
78
The Cognitive Neuroscience of Human Communication
including phonology and its connections. Global regulation and coordination is supported by the same data about semantic processing difficulties. With respect to neuroanatomic connections, cortical stimulation maps differ from the classical language function maps. First, Broca’s and Wernicke’s areas are probably spatially more circumscribed for any individual than is usually shown. Second, other areas crucial for language may be identified more reliably, for example, those related to the posterior connections between the temporal and occipital lobe. As expected, resection of the speech cortex can cause acute deficits, but recovery is quick—within several days or weeks. Such speedy recovery of even those skills that had taken longer to acquire (e.g., reading) suggests that, in normal situations, there are multiple but latent copies of language functions and representations (Gordon et al., 1994). The serial Wernicke–Geschwind model is challenged by the cases of deficits in reading aloud. According to the so-called dual-route models proposing that the reading process is in fact mediated by two different pathways of neural activity, there are two different systems of reading aloud. One is the lexical procedure, which is based on the information about pronunciation of written words stored in our vocabulary. The other one is the nonlexical procedure, which is based on general rules of pronunciation that enable us to read unfamiliar and nonsense words. Patients who have trouble with one procedure but not with the other are taken as strong support for these models. For example, in surface dyslexia, the nonlexical procedure is intact, but the patients have considerable difficulty reading words that require unusual pronunciation (impaired lexical procedure) and most frequently pronounce words in accordance with the rules (e.g., reading /ei/ instead of /æ/ in have as if the word rhymed with cave). With impaired nonlexical procedure but intact lexical procedure, the patients are typically unable to pronounce nonwords, but their ability to read real words is spared. PET scans have shown that looking at words activates only the occipital lobe with no additional activation in Wernicke’s area or in the angular gyrus; the frontal lobe processes spoken and written language in the same way (Pinel, 2005). Furthermore, the activity recorded in the right hemisphere during these studies is not predicted by the model. Obviously, these results cast serious doubt over any serial model of language processing. The problems with the Wernicke–Gerschwind model, as indeed with any localistic model, may be summarized in the following points: (a) localization of language system parts is identical in all healthy subjects; (b) specific functions of subregions of the language cortex are derived from the connections of these subregions with the motor and sensory cortical regions. These features are problematic, because they do not take into consideration the nature of many components of language processing, and are in disagreement with the evidence of variability in language function localization (Caplan, 1988). Despite the problems we have discussed with the Wernicke–Geschwind model, in combination with the observed language difficulties in brain-damaged people, it can provide sufficient information about the general location of damage.
The Central Nervous System
79
HIERARCHICAL MODELS These models are based on the principle of hierarchical organization primarily of sensory systems, such as visual, tactile, auditory, olfactory, and gustatory. The main path goes from each sensory organ to the thalamus, a group of nuclei located at the top of the brainstem. Nuclei responsible for individual sensory modalities relay the majority of the arriving information from the sensory organ to the limited part of the neocortex that is the primary sensory cortex for that particular sensory system. The complexity of coding increases with proximity to the neocortex. A large portion of outputs from each area of the primary sensory cortex are relayed to the neighboring cortical area—the secondary sensory cortex for that sensory modality. According to the traditional model, the association cortex is an area of cortical tissue not yet entirely understood that is the final destination of the sensations, loosely defined. It is believed that association cortex connects activities of different sensory systems, that it transforms the sensory input into the programs necessary for motor output, and that it mediates complex cognitive activities, such as reasoning and memory. There are multiple representations of sensory information in visual, auditory and somatosensory systems. At the highest levels, the functions are lateralized. Injuries to primary visual, auditory, and somatosensory systems cause serious deficits in sensory discrimination, which are manifested as significantly reduced sensory acuity. When secondary regions are damaged, acuity is frequently preserved, but not the ability to recognize objects, shapes, or patterns. It is assumed that the hierarchical stages are closely connected. A generally accepted view is that structures at increasingly higher levels of the sensory system have a less and less sensory and more and more perceptive role. This is based on the notion that each level of the system analyzes the input from the previous level, adding to the complexity of analysis. The Triune Brain According to MacLean (as cited in Spreen et al., 1984), the mammalian brain may be anatomically and conceptually divided into three hierarchical systems that make up a whole. The first system comprises upper parts of the spinal cord, the midbrain, the diencephalon, and the basal ganglia. On the evolutionary scale this system corresponds to the lowest developmental level. This part plays a crucial role in instinctive activities necessary for survival of the individual and the species. The second part corresponds to the limbic system and is on a higher evolutionary level. It is involved in the integration of emotional expressions and self-awareness, and it may inhibit and suppress the first part. The third part corresponds to the cortex, and according to MacLean, it is responsible for nonemotional, integrative, detailed analysis of the external world. In humans, these properties are represented by language in the left hemisphere, enabling us to reason and think about future actions. The third part may override the first two and, in terms of evolution, it is on the highest level. Such a division corresponds to the ontogenetic development of the central nervous system. Van der Vlugt (1979;
80
The Cognitive Neuroscience of Human Communication
cited in Spreen, 1984) divides this ontogenetic development into three stages as well, with each developmental stage depending on previous ones, and each subsequent structure having increasingly more refined and more complex adaptive and integrative functions. Luria’s Model of Functional Systems Luria (1973, 1982) worked out three functional units of the brain that are responsible for reception, interpretation, and use of sensory information. These units act serially, producing behavior. Any weakness in any of the structures may affect processing in another. The First Functional Unit The arousal unit (upper and lower parts of the brainstem, reticular formation and hippocampus) is primarily responsible for arousal level, attention, motivation, and reasoning or impulsiveness. Inappropriate arousal or attention may have a negative impact on information processing that takes place in the second unit, which in turn may influence planning performed by the third unit. The Second Functional Unit The sensory (input) unit (posterior cortical areas, including the occipital, temporal, and parietal lobes with the accompanying projection zones) is responsible for reception, processing, and storing of information. In this unit the information is processed (in parallel or serially) and coded; it is also responsible for integration of information from different sensory modalities. This unit is, to some extent, involved in every task. The Third Functional Unit The motor unit (for output and planning, located in the frontal lobe and associated areas) is responsible for planning and programming of behavior, as well as for actual performance. This unit depends on the efficient functioning of other units and is probably in highest correlation with intelligence. It is the highest functional level of the brain. Sensory and motor units may be further divided into primary, secondary, and tertiary areas, which represent levels of increasing complexity and integration in information processing. The first functional unit may be identified with selective attention; the second with sensory data, information processing, motor responses, and cerebral dominance; and the third unit may be identified with planning, organization, and higher-order language functioning. It should be kept in mind that several functional units in the brain participate in production of behavior; there is no direct one-to-one correspondence between a cerebral area and particular behavior. Luria related his theory to ontogenetic development of brain and behavior. Secondary zones cannot develop before the primary zones have fully developed. Only after the secondary (gnostic) zones have been developed can higher cortical zones become operative. The tertiary zones become dominant in adults.
The Central Nervous System
81
This hierarchical development of the three zones is mirrored in the development from the maximally modal specificity of the primary zones to the supramodal organizational and interpretative functions of the tertiary zones. This means that the primary areas are maximally modality-specific, whereas the tertiary areas are associative and integrative across sensory modalities. The development entails progressive lateralization of functions as well: left and the right primary areas have identical roles in an individual’s functioning, but hand dominance and language development require functional organization and specificities that develop concurrently with the secondary and tertiary areas and are radically different in the two hemispheres. Performance at any level does not predict the quality of performance at subsequent levels. Since psychological functions dependent on tertiary zones develop in late childhood or adolescence, clear differences between children and adults in this respect can be expected. Luria reserved a special place for the role of language in a child’s development. Newborns have only the basic ability to express themselves by movement, crying or facial expressions (the first signal system), but older children begin to complement their actions with verbal expressions and eventually develop the second signal system. With the maturation of the tertiary cerebral zones (according to Luria, occurring around age 6), language becomes more sophisticated, and accompanying motor actions are no longer necessary (this does not pertain to normal hand gesturing accompanying speech). Language does not have to be overt any longer—it becomes internalized, and it mediates and directs behavior. Kolb and Whishaw (1996) address several drawbacks of Luria’s model. First, it is practically impossible to set a clear distinction between sensory and motor functions at any level of the nervous system, particularly when it comes to the cerebral cortex. Besides, the model does not predict the possibility of different perceptions of the same object under changed circumstances. Another difficulty is that the model does not allow the possibility of information bypassing the frontal lobe or primary motor cortex. Finally, it does not predict parallel sensorymotor systems, for the existence of which there is anatomical evidence. One of the hierarchical models of perceptual speech analysis was proposed by Chistovich (1984). Her model posits the existence of hierarchically organized stages of perceptual analysis—auditory, phonetic, morphological, syntactic, and semantic. The result of analysis at one level is not final—at the higher level the ambiguous decision may be resolved. Each level operates on its units: at the morphological level decisions cannot be made about phonemes, for example, but only about morphemes, yet the higher levels may influence the outcome of the lower levels. Processing is parallel (at several levels simultaneously: morphological, phonological) and interactive (mutual influence of different levels). Some sort of normalization mechanism is assumed that takes care of stimuli arriving from different speakers. The model also posits that perceptual decisions are formed on the basis of (auditory) feature detectors. Perception starts at the subphonemic level because a decision is reached before the entire phonemic segment has been produced.
82
The Cognitive Neuroscience of Human Communication
Jurgens’ Model of Neural Vocalization Control Jurgens (as cited in Kent & Tjaden, 1997) proposed a three-subsystem model of neural vocalization control, based on the studies of human and animal vocalizations. Subsystem I This is primarily responsible for coordination of breathing, movements of the larynx and upper respiratory pathways. It includes reticular formation, and motor and sensory nuclei of the brainstem and spinal cord responsible for phonation. Apart from coordinating muscle activity, this subsystem may have subroutines necessary for genetically programmed vocalization patterns. Subsystem II This initiates the vocal expression and selects among the existing vocalization patterns. It is hierarchically organized into three levels: • Lowest level—periaqueductal gray and the surrounding tegmentum in the midbrain. This structure plays an important role in vocal expression of emotions. • Middle level—subcortical limbic structures including the hypothalamus, medial part of the thalamus, amygdala, etc. These same structures are involved in regulating emotions. Their role is integration of external stimuli with the internal motivational ones. • Highest level—anterior cingulate—the role of this structure is to maintain control of emotional states. Subsystem III This comprises the motor cortex and its primary input–output structures, primary somatosensory cortex, cerebellum, ventrolateral part of the thalamus, putamen, nucleus ruber, and the pyramidal tract. This subsystem executes fine voluntary control of vocalization patterns. Obviously, this is also a hierarchical model. Kent and Tjaden (1997) proposed a somewhat more elaborate variant of the model. Their model also has three main subsystems, but they have modified Jurgens’ subsystem III so that it reflects a greater number of connections and specific functions of some structures. It includes more inputs to the motor cortex, particularly from premotor areas. Basically, their model predicts parallel connections and processing.
MODULAR MODELS Legg (1989) claims that the brain owes its resistance to damage to its modular structure. As opposed to the interactive principle of functioning, in which damage to any part causes collapse of the entire system (because the subregions are connected to each other), modular structure is such that groups of components are
The Central Nervous System
83
functionally isolated from each other. Accordingly, if one group is damaged, the remaining ones will continue to function normally. Modularity can be achieved in two ways: (a) functionally independent modules are created so that each of them performs a different function (complementary system); or (b) several identical modules exist, each of which is capable of performing equal functions (redundant system). Processing is exclusively bottom-up. A higher level cannot be reached before completing the lower one. Modularity has found support in the fact that brain damage results in very specific consequences limited to the damaged area. Specificity does not only imply that a function is lost or weakened by damage, but also that the part survives lesions to other parts that result in different consequences. Legg (1989) concludes that the brain functions on the principle of the redundant modular system (because if it were interactive, it would be much more susceptible to damage), and that the only operative difference between the interactive and the redundant modular system is the degree of dysfunction resulting from damage to individual components. According to Fodor’s (1983) modular model of the mind language does not belong to the central system. It is an input system, and input systems have modular structure. This modular structure is determined by several features. First, the input system receives information from several sensory systems, but it processes them in particular ways. For example, during reading the language input system translates information from the visual modality to the phonological or speech representation modality. Second, processing is one-way, bottom-up, which means that higher cognitive modules do not affect the lower ones (e.g., sentence context does not affect phonemic processing). In cases of ambiguities, both (or all) word meanings are activated and forwarded to higher levels, and ambiguity is resolved by context only after both (or all) meanings have reached the level of context. Third, each module has its place in a particular region of the brain. Hence, modular models are at the same time predominantly localistic. During speech production, actual speech output is preceded by several preparatory stages that are superordinate in terms of cognitive level. One of the most frequently quoted models in the past 15 years is Levelt’s (1989; Levelt, Roelofs, & Meyer, 1999). According to his theory, word preparation is accomplished through the stages of conceptual preparation, lexical selection, morphological and phonological encoding, as well as phonetic encoding, which is followed by articulation. In the process, the result (i.e., the language product—speech) is monitored by means of the speaker’s speech comprehension mechanism. The model includes macroplanning (a decision about what we want to convey to the listener) and microplanning (choosing words and defining their roles). The output of the planning stage is the conceptual message that is forwarded to the formulator, which supplies the message with appropriate grammatical and phonological form (grammatical encoding). During the process, the surface structure of the message is computed. In this surface structure, lemmas are the lowest-level elements. Lemmas are stored in the mental lexicon, and they contain all syntactic and semantic characteristics of the word. Since the concepts are organized with
84
The Cognitive Neuroscience of Human Communication
respect to meaning, in a task where the subjects are required to name an object in the picture, activation of one concept will activate semantically related concepts, and the lemma eventually selected will belong to the term that corresponds to the picture (lexical selection). The lemma then activates the lexeme (word form) with its phonological and phonotactic data, followed by appropriate phonetic encoding based on the data, resulting in the activation of the sound form of the word. Articulation is the final process that results in the spoken word by mapping phonetic gestures to articulators. In summary, the model essentially consists of the following six stages: conceptual preparation (nlexical concepts), lexical selection (nlemma), morphological encoding (nbasic word form), phonological encoding (n syllabification), phonetic encoding (n preparation of speech sounds), articulation (nspoken word) (Eysenck & Keane, 2000). Since this model is modular and serial, phonological processing cannot take place before the lemma has been chosen. Therefore, phonological information has no effect on lemma selection. Auditory-perceptive self-monitoring is the only form, albeit very limited, of feedback (Levelt et al., 1999). The model is illustrated in Figure 5.2. Some ERP studies have confirmed that lemma selection precedes phonological activation at the level of the lexeme (Gazzaniga et al., 2002). Different approaches to studying language functions and models of language function organization more or less confirm that language is modular in the sense that it is autonomous with respect to other higher cortical functions, and that it involves submodules (e.g., syntax, lexicon) that are functionally autonomous. This autonomy, however, cannot be interpreted as exclusively oneway processing or the existence of separate neurological structures for individual submodules, but only as relatively independent functioning (Blumstein, 1988).
CASCADE MODELS These models are similar to modular ones. They include different, but mutually connected neural modules, each of which is responsible for a different level of processing of word information or some other data. In terms of word recognition, they propose that lexical access is autonomous, independent of higher levels, but that lexical selection may be equally affected by the sensory (bottom-up) and by contextual (top-down) levels. Examples of serial or cascade models are supplied by Bentin et al. (1999) and Schmitt et al. (2000). They report data from studies in which subjects were required to name pictures in two concurrent tasks. The semantic task was to decide whether the picture depicted an object or an animal; the phonological task was to decide whether the name of the object starts with a vowel or a consonant. They claim that semantic processing starts before phonological.
The Central Nervous System
Conceptualizer
85
Concepts Lexical concept Lexical selection Lemma Morphological encoding
Formulator
Mental lexicon
Morpheme Self-monitoring Phonological encoding syllabification Phonological word Phonetic encoding
Syllabary
Phonetic gesture
Articulator
Articulation Sound wave
FIGURE 5.2.
Levelt’s model of speech production.
INTERACTIVE MODELS Contrary to modular models, interactive models propose some degree of influence of one processing level on another (both toward the higher and toward the lower levels), and a combination of top-down and bottom-up processing. According to interactive models, phonological activation starts immediately after the beginning of semantic and syntactic activation and proceeds simultaneously. Not only can all types of information participate in word recognition, but context may affect sensory data even before they become available, by changing the activation status of the word form. According to these models ambiguity is not possible, because only those words will be activated that are in harmony with the familiar
86
The Cognitive Neuroscience of Human Communication
context. Most of these models do not predict the existence of words as readymade forms, but a particular word is recognized if within the neural network circuit certain nodes are activated simultaneously (Eysenck & Keane, 2000; Gazzaniga et al., 2002; Pinel, 2005). The models posit some feedback among levels, but also among elements within each level. Two types of processing are at work here: (a) data-driven processing (bottom-up) that starts with input data; (b) concept-driven processing (top-down) that starts with thought processes. These are active models, because they predict active cooperation between different levels taking into consideration context, expectations and attention, as opposed to the passive models in which processing is one way—only bottom-up. The interactivity assumes, naturally, simultaneous processing at several levels, but also some degree of hierarchy. A large number of models involve some measure of distinctive feature analysis at lower levels, which may be reflected in the name of the theory or model or just in its description. In auditory perception, the process starts from the bottom and proceeds upward. At the auditory level, speech segments are not separated, but acoustic analysis takes place (for more details see chap. 1). Processing continues through the phonetic level (where decisions are made about the pronounced sound), and on to the phonological level (where the results of phonetic analysis are compared and coordinated with the specificities of the language at hand). Models and their diagrams vary in the amount of detail within the basic levels, but common to them all, even when it is not explicitly specified, is comparison of the input signal with some already existing patterns (template matching).
CONNECTIONIST MODELS Classical connectionist models are those in which one processing continues after another. In that sense the components are related, but they are still predominantly localistic and serial models. An example is the Wernicke–Geschwind model discussed previously, but also the much older model by Lichtheim (discussed in this chapter). More recently, the connectionist approach assumes interactivity and parallel processing, as well as wide spatial distribution, due to which these models are said to be based on parallel distributed processing. Contrary to the localistic models, the connectionist models assume widely represented functions and connections among all segments of processing. The first computeraided connectionist models were designed by McClelland and Rumelhart (1986). Since they include basic units or nodes mutually connected into networks, they are most frequently referred to as network models or neural network models. The first discussions in which neural networks were mentioned date back to the 1940s (Lon´cari´c, 2002).
NEURAL NETWORKS Many studies have shown that hardly any function can be reliably localized in a particular spot, that performance of a cognitive task (e.g., speech) activates
The Central Nervous System
87 Output layer (units)
Hidden layer (units)
Input layer (units)
FIGURE 5.3.
Example of a simple neural network.
simultaneously several separate cerebral areas, and that stimulation of a limited area may elicit any number of different responses. Language and speech functioning can therefore be interpreted as coordinated activity of more different assemblies within a single unitary network (cortical or neural). As it has already been mentioned previously, it is believed that basic functions are localized in specific locations in the cerebral cortex, whereas complex functions involve parallel processing of information in widely distributed networks that spread over cortical and subcortical structures. Neural network models consist of three layers: the input layer (units), the hidden layer (units), and the output layer (units). The units (that actually represent neurons) are connected and these connections facilitate or inhibit (increase or decrease) activation levels of the specific reception unit. Each unit receives input from the preceding layer and relays its signals to the next layer (Figure 5.3). Each input has a value—weight. The activation level of each unit depends on the weighted sum of all inputs. Hidden units provide information necessary for correct mapping of the input layer to the output layer by changing the strength of connections between units. This is accomplished by developing a set of rules for learning, in other words, a quantitative description of necessary changes in processing, depending on the final outcome. During the practice phase, neural networks “learn” to respond to given inputs with a particular output, by modifying
88
The Cognitive Neuroscience of Human Communication
the connections between units, so that each actual output is compared with the target output, and in case of discrepancies the weights are changed. The rule for weight adjustment is called back propagation algorithm (back prop), because the error message is relayed through the network in the opposite direction—from output units (back) to input units. Rumelhart (1986) called this procedure supervised learning (as cited in Ainsworth, 1997; Gazzaniga et al., 2002). A trained network, in fact, resembles natural systems: it generalizes responses to the patterns that had not been explicitly used during practice, and its efficacy gradually decreases as individual units are eliminated. When we say that many areas function as a network, it means that they consist of mutually highly interconnected sets of neurons that function as units during a higher cortical activity. Neural networks are adaptable; they can learn to differentiate among patterns of input stimuli and to respond to each one of them in an appropriate manner. Information in the neural network is distributed, and each cell contributes to the recognition of every pattern. Particular cells do, however, respond selectively to specific patterns, but there are many of them for each type, and they are distributed throughout the network. This means that complex functions are represented within mutually interconnected areas that together make an assembly. Several conclusions may be drawn: (a) each individual cerebral area has the neural basis for several different types of behavior, and it can be part of several assemblies; (b) damage to one area will probably result in multiple deficits of some function; and (c) different aspects of the same complex function may be impaired depending on which of the many cortical areas and their connections have been injured. It can also be said that the brain consists of a number of smaller areas that function as overlapping neural networks. Network models that include programmed cell death (apoptosis) are even closer to the actual functioning of the brain. Such models have shown that artificial neural networks may successfully solve difficult problems if there are more neurons at the beginning than will be eventually needed. The least useful neurons die away gradually after learning has taken place, and the learned solution can be preserved even with a smaller number of units than was necessary for initial learning—this is obviously similar to the living brain (Brown, Hulme, Hyland, & Mitchell, 1994). Although there are recurrent networks as well as feed-forward networks, speech and language functioning requires a model that predicts a combination of bottom-up and top-down processing. Damage to the network causes overall inefficacy without specific deficits. The actual behavior of the network depends on the extent of damage: after mild damage the remaining cells may take over some of the function affected by damage; more serious damage makes the connections within the network sparse. The cells within the network recognize and respond to input patterns by mutual excitation and inhibition. As a consequence, damaged networks react more slowly, and it is possible that the normal excitation threshold is never reached, which results in malfunction (Deane, 1992). It seems that impulses relayed by neural pathways to the higher cortical areas code information digitally: During each functional state some neuronal fibers are
The Central Nervous System
89
active while others are at rest. Neural networks are more resistant to localized damage than single neuronal tracts. According to the model of neural networks connected by nerve tracts, the brain functions as a series of parallel processors that interact by means of information channels that integrate them into a whole (Deane, 1992). In short, networks have great data processing capabilities. They are resistant to damage, quick and adaptable, which makes them a good model of brain functioning (Lieberman, 1991). Neural network models can engage in some highly sophisticated cognitive processes. Many consequences of brain damage resemble closely what one would get by damaging a network system (be it individual neurons, or connections among them). Seidenberg and McClelland (1989) constructed a language acquisition network. By switching off individual units of the network they simulated brain damage, which resulted in their network behaving as a dyslexic. In studies of speech production, neural network models have been used to simulate articulatory movements on the basis of the data obtained by electromyographic recordings (e.g., the relationships among different articulator positions, speed, acceleration, etc.), or on the basis of experimental results about the influence of changes in the rate of speech and phonetic environment on the kinematics of speech movements (Löfqvist, 1997). On the basis of their PET results, Friston et al. (as cited in Kent & Tjaden, 1997) proposed a model of word production based on the neural network models. Their model comprises three neural systems: (a) a distributed word representation store; (b) an afferent system that relays information to the store; and (c) a modulating system that changes the sensitivity of the neurons in the store. The store is located in the left hemisphere, in Wernicke’s area, and the modulation system is located in the left dorsolateral prefrontal cortex. Simpler versions of the neural network models cannot include temporal variations in speech and are therefore not suitable for speech recognition, but more complex models, with more than one hidden layer, achieve better results because they can learn to recognize different acoustic properties, such as second-formant transitions, vowel onsets, and so forth—independent of temporal organization (Ainsworth, 1997). It seems that we have yet to design a model that would solve all the problems of speech recognition or that would integrate all aspects of speech and language functioning into a whole. The theory of neuronal group selection functions similarly to neural or cortical networks. According to this theory, different populations of neurons are structured by the interaction between genetic and experiential factors. The theory predicts exchange of signals by means of parallel and reciprocal connections among groups of neurons (reentrant signaling) and it attempts to explain such a wide selection of phenomena as perception, motor responses, language, and consciousness (Kent & Tjaden, 1997). Spatialization of form hypothesis (parietal hypothesis) predicts that language processing (grammatical competence) is represented in the part(s) of the brain whose primary function is the representation of body image (schema), in other
90
The Cognitive Neuroscience of Human Communication
words, processing of spatial structures, especially with respect to one’s own body. This is a direct isomorphism between spatial and grammatical knowledge. The cerebral area responsible for this function is located in the region where auditory, visual, kinesthetic and inputs from other sensory modalities converge and are integrated (Deane, 1992). The requirements are best met by the inferior parietal lobe. In other words, according to this hypothesis, the seat of grammatical (syntactic) competence should be in the inferior parietal lobe, and not, as traditionally assumed, in the frontal lobe, particularly Broca’s area. According to this hypothesis, aphasia can be explained by the loss of awareness about one’s own body. Pansini’s (1988) notion of spatioception (spatial perception) as a necessary prerequisite for speech acquisition includes five sensory systems: tactile, proprioceptive, vestibular, auditory, and visual. Stimuli from these five modalities converge in the part of the brain predicted by parietal hypothesis—on the border between the parietal, temporal and occipital lobes. Regardless of whether they are taken as five sensory modalities that take part in perception of space or as a new modality—spatioceptive—the underlying idea is that without them speech acquisition would be impossible just as spatial and language relations (grammar, primarily syntax, is based on these rules and relations) would be unthinkable. This idea is in agreement with the parietal hypothesis about the representation of language functions in the brain. A study of the relationship between space and native versus foreign language revealed a very strong effect of language on the preference of the left or the right hemispace (Mildner, 2002): When the processing was in the native language the subjects preferred reaching into the left hemisphere, and when the stimuli were in a foreign language they preferred the right hemispace. Here is some evidence in support of the parietal hypothesis (Deane, 1992): 1. The inferior parietal lobe is the seat of bodily awareness and orientation toward extrapersonal space. It integrates somatosensory information. Moreover, this is the only cortical area in which information from many modalities is available at the same time. There is ample clinical evidence and evidence from primate experiments to support this claim. 2. The abilities to represent and manipulate complex spatial relations are located in the inferior parietal lobe. This includes the ability to recognize complex objects on the basis of the nature and arrangement of their components (right hemisphere in humans). 3. The inferior parietal lobe seems to be necessary for constructing sequential plans of action—for manipulation of a series of actions as mental entities (left hemisphere in right-handers). The left hemisphere is responsible for mental manipulation of abstract structures. Damage to the parietal lobe causes various symptoms: global aphasia, agrammatism, reading difficulties, and disorders of grammar and logic, and so forth. 4. The right inferior parietal lobe has only visuospatial and somatic functions and is considerably more efficient in spatial processing than the corresponding area in the left hemisphere. Homotopic areas in the left
The Central Nervous System
91
hemisphere have a key role in hand–eye coordination and in general in hand movements. The hand is basically a parietal lobe organ (as extremities in general, it is the part of the body whose main function is interaction with the environment). The parietal hypothesis predicts that the greatest damage to structural thought (ngrammar) will occur as a consequence of injury to the margins of the inferior parietal lobe, and that the nature of the deficit will depend on connections that have been severed. This, in turn, is defined by functional brain geometry (Deane, 1992). According to the hypothesis, aphasia will occur after injury to any segment of the network, not only as a consequence of direct damage in the injured area, but also because the injured area can no longer be activated, or the result of its activity is not a desirable input to other parts of the network. This may explain why there are so few cases of aphasia that can be clearly categorized, as well as cases of, for example, Broca’s aphasia without actual damage to Broca’s area. Damasio and Damasio (1992) represent language processing in the brain through three sets of interacting structures. Their description is similar to Luria’s model of three functional units (see previous discussion), but it is interactive and to some extent it relies on neural networks as well. The first set consists of a large group of neural structures in both hemispheres that represent nonlanguage interactions between body and environment, realized by means of various sensory and motor systems; in other words, everything that a person does, perceives, or feels. The brain categorizes these nonlanguage representations (based on shape, color, order, emotional state, etc.) and creates a new level of representation for the results of this categorization. This is how humans organize objects, events, and relations. Subsequent layers of categories and symbolic representations are the basis of abstraction and metaphor. The second set comprises a smaller number of neural structures, mainly located in the left hemisphere, that represent phonemes, their combinations and syntactic rules for combining words. When stimulated from within the brain, they produce words and sentences that will be said or written. When stimulated by the environment, by means of speech or writing, they engage in initial processing of auditory or visual language signals. The third set (also mainly in the left hemisphere), acts as a mediator between the first two sets. This set may start with the concept and initiate word production, or it may receive words and stimulate the brain to retrieve appropriate concepts. Besides selecting appropriate words to denote a particular concept, these intermediary systems control the production of sentence structures that express relations among concepts. Neural structures that serve as mediators between concepts and words are located along the occipitotemporal axis. It seems that mediation for the many general terms occurs in the posterior part of the axis, whereas the most specific ones are mediated by the most anterior part, near the left temporal pole. Damasio and Damasio (1992) conclude this on the basis of clinical cases of focal injuries
92
The Cognitive Neuroscience of Human Communication
in different parts of the left temporal pole (see also Gazzaniga, 1994). Depending on the location of the injury, there may be selective damage to common or proper nouns. Injury to some parts of the temporal lobe may impair access to verbs and function words, but also the grammatical structure of the sentences produced by the patients. In general, Damasio and Damasio (1992) redefine the notion of “images” in the brain—physical representations of different concepts and experiences. They claim that the brain preserves a trace of neural activity that was elicited in the sensory and motor cortex during interaction with an object or a person, rather than a permanent record of actual “pictorial” representations of objects or persons, as was traditionally believed. These traces are patterns of synaptic connections that may repeatedly combine separate sets of activities defining an object or an event. Each trace may initiate related traces as well. In other words, besides preserving the representations of the external world, the brain records how the body reacts to the outside world. This interaction is represented in simultaneous microperceptions and microactivities in separate functional regions. These separate activities are united in neural assemblies in many areas of “convergence” in the brain, which would correspond to the traditional notion of association areas, as well as Turner and Frackowiak’s (1995) notion of cortical fields, or the Hebbian cell assemblies, as suggested by Pulvermüller (1992). The brain also categorizes information so that related events and concepts may be reactivated together. This possibility is based on the physiological principle that the neurons that are active together tend to group together (see the “fire together–wire together” principle in chap. 1, this volume). Activity within such a network may serve both expression and comprehension. It is precisely due to categorizing percepts and actions simultaneously along many different dimensions that symbolic representations (e.g., metaphors) are possible (