The Cognitive Neuroscience of Human Communication
The Cognitive Neuroscience of Human Communication
Vesna Mildner
L...
62 downloads
1859 Views
4MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
The Cognitive Neuroscience of Human Communication
The Cognitive Neuroscience of Human Communication
Vesna Mildner
Lawrence Erlbaum Associates New York London
Lawrence Erlbaum Associates Taylor & Francis Group 270 Madison Avenue New York, NY 10016
Lawrence Erlbaum Associates Taylor & Francis Group 2 Park Square Milton Park, Abingdon Oxon OX14 4RN
© 2008 by Taylor & Francis Group, LLC Lawrence Erlbaum Associates is an imprint of Taylor & Francis Group, an Informa business Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number-13: 978-0-8058-5436-7 (Softcover) 978-0-8058-5435-0 (Hardcover) No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Mildner, V (Vesna) The cognitive neuroscience of human communication / Vesna Mildner. p. cm. Includes bibliographical references and index. ISBN 0-8058-5435-5 (alk. paper) -- ISBN 0-8058-5436-3 (pbk.) 1. Cognitive neuroscience. 2. Communication--Psychological aspects. 3. Communication--Physiological aspects. I. Title. QP360.5.M53 2006 612.8’233--dc22 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com
2005049529
For Boris, without whom none of this would be possible or even matter.
Contents Foreword .............................................................................................................. xi Raymond D. Kent Preface ...............................................................................................................xiii Chapter 1 Central Nervous System .................................................................. 1 The Development of the Central Nervous System ................................................ 1 Structure and Organization of the Central Nervous System ................................ 5 Sensation and Perception .................................................................................... 24 Neural Bases of Speech Perception and Production ........................................... 26 Hearing, Listening and the Auditory Cortex ................................................. 26 Movement and Speech Production................................................................. 30 Relationship Between Speech Production and Perception ............................ 34 Neighboring Location of Motor and Sensory Neurons ............................. 35 Multimodal Neurons ................................................................................. 36 Parallel and Recurrent Pathways ............................................................... 37 Chapter 2 Sex Differences ............................................................................. 39 Structural Differences......................................................................................... 39 Differences in Functional Organization of the Brain ......................................... 40 Behavioral and Cognitive Differences ................................................................ 40 Chapter 3
Brief History of Neurolinguistics from the Beginnings to the 20th Century ....................................................................... 45
Chapter 4 Research Methods ......................................................................... 51 Clinical Studies ................................................................................................... 51 Studies of Split-Brain Patients ....................................................................... 53 Cortical Stimulation ............................................................................................ 54 Transcranial Magnetic Stimulation (TMS).................................................... 55 Wada Test ............................................................................................................ 55 Neuroradiological Methods ................................................................................ 56 Computerized (Axial) Tomography—C(A)T ................................................. 56 Magnetic Resonance Imaging (MRI) ............................................................ 56 Functional Magnetic Resonance Imaging (fMRI)......................................... 57 Recording of Activity ......................................................................................... 57 Electrophysiological Methods ........................................................................ 58 Single-Unit or Single-Cell Recording ....................................................... 58 Electroencephalography (EEG) ................................................................ 59 vii
viii
The Cognitive Neuroscience of Human Communication
Event-Related Potentials (ERP) ................................................................ 59 Cortical Cartography................................................................................. 60 Magnetoencephalography (MEG) ............................................................. 60 Radioisotopic Methods .................................................................................. 61 Positron Emission Tomography (PET)...................................................... 61 Single-Photon Emission Computed Tomography (SPECT) ...................... 62 Ultrasound Methods ....................................................................................... 62 Functional Transcranial Doppler Ultrasonography (fTCD)...................... 62 Summary ........................................................................................................ 62 Behavioral Methods ....................................................................................... 63 Paper-and-Pencil Tests .............................................................................. 64 Word Association Tests ............................................................................. 64 Stroop Test................................................................................................. 64 The Wisconsin Card Sorting Test (WCST)............................................... 64 Priming and Interference .......................................................................... 64 Shadowing ................................................................................................. 65 Gating ........................................................................................................ 65 Dichotic Listening ..................................................................................... 66 Divided Visual Field.................................................................................. 67 Dual Tasks ................................................................................................. 67 Summary ........................................................................................................ 68 Aphasia Test Batteries......................................................................................... 68 Chapter 5
The Central Nervous System: Principles, Theories and Models of Structure, Development and Functioning .................... 71 Principles ............................................................................................................ 71 Hierarchical Organization.............................................................................. 71 Parallel Processing ......................................................................................... 72 Plasticity ......................................................................................................... 72 Lateralization of Functions ............................................................................ 73 Theories and Models........................................................................................... 73 Parallel or Serial Processing? ........................................................................ 74 Localistic Models ........................................................................................... 75 Wernicke–Geschwind Model .................................................................... 76 Hierarchical Models ....................................................................................... 79 The Triune Brain ....................................................................................... 79 Luria’s Model of Functional Systems ........................................................ 80 Jurgens’ Model of Neural Vocalization Control ....................................... 82 Modular Models ............................................................................................. 82 Cascade Models ............................................................................................. 84 Interactive Models .......................................................................................... 85 Connectionist Models .................................................................................... 86 Neural Networks ............................................................................................ 86 Other Theories and Models............................................................................ 92
Contents
ix
Motor Theory of Speech Perception ......................................................... 93 Analysis by Synthesis ................................................................................ 94 Auditory Theory ........................................................................................ 95 Neural (Phonetic, Linguistic) Feature Detectors....................................... 95 Theory of Acoustic Invariance.................................................................. 95 The Cohort Theory.................................................................................... 96 Trace Model............................................................................................... 96 The Neighborhood Activation Model (NAM) .......................................... 96 PARSYN ................................................................................................... 97 The Mirror–Neuron System ...................................................................... 97 Chapter 6 Lateralization and Localization of Functions ............................... 99 Lateralization of Functions ................................................................................. 99 Verbal Versus Nonverbal and Language Versus Spatial Information ......... 103 Analytic Versus Holistic Approach to Processing ....................................... 107 Serial or Sequential Versus Parallel Processing .......................................... 108 Local Versus Global Data Representation ................................................... 109 High Frequencies Versus Low Frequencies ..................................................110 Categorical Versus Coordinate .....................................................................113 Developmental Aspects of Lateralization..........................................................113 Neuroanatomic Asymmetries .......................................................................119 Sensory Asymmetries .................................................................................. 120 Motor Asymmetries ..................................................................................... 121 Asymmetries in Other Species .................................................................... 122 Factors Influencing Functional Cerebral Asymmetry ...................................... 123 Localization of Functions ................................................................................. 127 Lateralization and Localization of Emotions .............................................. 134 Summary........................................................................................................... 137 Chapter 7 Learning and Memory ................................................................ 139 Plasticity............................................................................................................ 139 Critical Periods ................................................................................................. 144 Types of Memory .............................................................................................. 149 Sensory Memory .......................................................................................... 149 Short-Term/Working Memory ..................................................................... 150 Long-Term Memory ..................................................................................... 154 Neural Substrates of Memory ........................................................................... 155 Chapter 8 Speech and Language...................................................................161 Speech and Language Functions and Their Location in the Brain .................. 163 Anatomic Asymmetries and Lateralization of Speech and Language ..............167 Split-Brain Patients ........................................................................................... 168 Healthy Subjects ............................................................................................... 170
x
The Cognitive Neuroscience of Human Communication
Speech Production and Perception ................................................................... 172 Speech Production ....................................................................................... 172 Speech Perception .........................................................................................176 Phonetics and Phonology .................................................................................. 179 Tone and Prosody.............................................................................................. 185 Lexical Level and Mental Lexicon ................................................................... 190 Word Recognition ........................................................................................ 195 Perceptual Analysis of Linguistic Input....................................................... 197 Word Categories ........................................................................................... 199 Sentence Level: Semantics and Syntax............................................................. 205 Discourse and Pragmatics..................................................................................210 Reading ............................................................................................................. 212 Writing .............................................................................................................. 215 Calculation .........................................................................................................216 Is Speech Special?..............................................................................................217 Language Specificities ...................................................................................... 220 Bilingualism...................................................................................................... 222 Speech and Language Disorders....................................................................... 229 Aphasia ......................................................................................................... 231 Recovery of Language Functions: Functional Cerebral Reorganization ..... 234 Agraphia and Alexia .................................................................................... 237 Motor Speech Disorders ................................................................................... 241 Dysarthria .................................................................................................... 241 Apraxia of Speech ........................................................................................ 242 Stuttering ...................................................................................................... 243 Other Causes of Speech and Language Disorders ............................................ 244 Schizophrenia............................................................................................... 244 Epilepsy and Tumors .................................................................................... 245 Right-Hemisphere Damage .......................................................................... 246 Epilogue ........................................................................................................... 249 Glossary ........................................................................................................... 251 Appendix .......................................................................................................... 295 References ........................................................................................................ 299 Author Index ....................................................................................................331 Subject Index ................................................................................................... 343
Foreword Raymond D. Kent As humans try to understand themselves, one of the greatest fascinations—and most challenging problems—is to know how our brains create and use language. After decades of earnest study in a variety of disciplines (e.g., neurology, psychology, psycholinguistics, neurolinguistics, to name a few), the problem of the brain and language is now addressed especially by the vigorous interdisciplinary specialty of cognitive neuroscience. This specialty seeks to understand the neural systems that underlie cognitive processes, thereby taking into its intellectual grasp the dual complexities of neuroscience and cognition. In her extraordinary book, Vesna Mildner gives the reader a panoramic view of the progress that cognitive neuroscience has made in solving the brain–language problem. Mildner covers her topic in eight chapters that can be read in any order. Each chapter is a tightly organized universe of knowledge; taken together, the chapters are complementary in their contribution to the overall goal of the book. The first chapter addresses basic aspects of the development, structure, and functioning of the human central nervous system (CNS), arguably the most complexly organized system humans have ever tried to fathom. The author systematically identifies and describes the tissues and connections of the CNS, thereby laying the foundation for the succeeding chapters that consider the topics of sex differences, the history of neurolinguistics, research methods, models and theories of the central nervous system, lateralization and localization of functions, learning and memory, and—the culminating chapter—speech and language. The sweep of information is vast, but Mildner succeeds in locking the pieces together to give a unified view of the brain mechanisms of language. Science is a procession of technology, experiment, and theory. Mildner’s comprehensive review shows how these three facets of scientific progress have shaped the way we comprehend the neurological and cognitive bases of language. From early work that relied on “accidents of nature” (brain damage resulting in language disorders) to modern investigations using sophisticated imaging methods, the path to knowledge has been diligently pursued. The unveiling of the brain through methods such as functional magnetic resonance imaging and positron emission tomography has satisfied a scientific quest to depict the neural activity associated with specific types of language processing. Today we stand at a remarkable confluence of information, including behavioral experiments on normal language functioning, clinical descriptions of neurogenic speech and language disorders, and neuroimaging of language processes in the intact living brain. But the profound potential of this synthesis is difficult to realize because the knowledge is spread across a huge number of journals and books. Vesna Mildner offers us a precious gift of scholarship, as she distills the information from more than 600 references to capture the science of brain and language. xi
Preface This book is intended for those interested in speech and its neurophysiological basis: phoneticians, linguists, educators, speech therapists, psychologists, and any combination of cognitive and/or neuro- descriptions added. In order to get a comprehensive picture of speech production and perception, or representation of speech and language functions in the brain, it is usually necessary to go through page after page, actual and virtual, of texts on linguistics, psychology, anatomy, physiology, neuroscience, information theory, and other related areas. In most of them language is covered in one or at best a few very general chapters, with speech as a specific, but the most uniquely human means of communication, receiving even less attention and space. On the other hand, the books that focus on language do not have enough information on the neurophysiological bases of speech and language either with respect to production or perception. My intention was to make speech the central topic, and yet provide sufficient up-to-date information about the cortical representation of speech and language, and related topics (e.g., research methods, theories and models of speech production and perception, learning and memory). Data on clinical populations are given in parallel to studies of healthy subjects, because such comparisons can give a better understanding of intact and disordered speech and language functions. The book is organized into eight chapters. They do not have to be read in the order they are written. Each of them is independent and may be read at any time or skipped entirely if the reader feels that he or she is not interested in the particular topic or knows enough about it. However, to those who are just getting acquainted with the topic of the neurophysiological bases of speech and language I recommend starting with chapter 1 and reading on through to the last chapter. The first chapter is an overview of the development, structure, and functioning of the human central nervous system, particularly the brain. It is perhaps the most complex chapter with respect to terminology and the wealth of facts, but the information contained therein is necessary for a better understanding of the neurophysiological bases of speech and language. When introduced for the first time, each technical term (anatomical, physiological, evolutional, etc.) is given in English and Greek/Latin. Besides the sections on the development, structure, and organization of the central nervous system, the chapter includes sections on sensation and perception and on the neural bases of perception and production of speech. The latter section deals with hearing and the auditory cortex, with movement and speech production, and addresses the various ways in which speech perception and production are related. Chapter 2 is a brief account of differences between the sexes in neuroanatomy, development, and behavior. Awareness of these differences is important for a better understanding of the linguistic development and functioning of males and
xiii
xiv
The Cognitive Neuroscience of Human Communication
females, since these differences frequently become apparent in various aspects of speech/language disorders (e.g., aphasias and developmental dyslexia). In chapter 3, I present chronologically the major ideas, theories, and historical milestones in research on the mind–brain relationship (particularly with respect to speech and language). In addition to the well-known names (e.g., Broca and Wernicke), the chapter includes persons who have been frequently unjustly neglected in neurolinguistic literature in spite of their important contributions. The chapter sets the stage for the results of research that are discussed throughout the rest of the book, and that span the second half of the 20th century to the present. Chapter 4 is a review of research methods. It includes the descriptions, with the advantages and the drawbacks, of the techniques that are at present the methods of choice in clinical and behavioral studies (e.g., fMRI), as well as those that are for various reasons used less frequently but their results are available in the literature (e.g., cortical stimulation). The chapter includes a review of the studies of split-brain patients, cortical stimulation studies, radiological methods, electrophysiological methods, ultrasound and radioisotopic techniques, and the most frequent behavioral methods (e.g., dichotic listening, divided visual field, gating, priming, and Stroop). In chapter 5, I examine different models and theories—from the older, but still influential ones (e.g., Wernicke–Geschwind model) to the most recent that are based on modern technologies (e.g., neural networks). The chapter starts with the short description of the most important principles of the central nervous system functioning (e.g., hierarchical organization, parallel processing, plasticity, and localization of functions), which theories and models explain. Chapter 6 explains the key terms and dichotomies related to functional cerebral asymmetry (e.g., verbal–spatial, local–global, analytic–holistic), and also some less frequently mentioned ones (e.g., high vs. low frequencies, categorical vs. coordinate). It includes a section on developmental aspects of lateralization, within which the various aspects of asymmetry are considered: neuroanatomical asymmetry, motor asymmetry, asymmetry of the senses, and asymmetry in other species. There is also a section on the factors that affect functional asymmetry of the two hemispheres, and a section on the lateralization of functions, including cerebral representation of various functions. Chapter 7 deals with the different types of learning and memory, with particular emphasis on speech and language. The existing classifications of learning and memory types are discussed and are related to their neural substrates. There are sections on nervous system plasticity and critical periods, as important factors underlying the acquisition and learning of the first and all subsequent languages. Finally, chapter 8, albeit the last, is the main chapter of the book, and is as long as the rest of the book. It is subdivided into sections corresponding to different levels of speech and language functions, and includes sections on bilingualism and speech and language disorders. Here are some of the section titles: • Speech and Language Functions and Their Locations in the Brain • Anatomic Asymmetries and Lateralization of Speech and Language • Speech Production and Perception
Preface
• • • • • • • • • • • •
xv
Phonetics and Phonology Tone and Prosody Lexical Level and the Mental Lexicon Sentence Level—Semantics and Syntax Discourse and Pragmatics Reading, Writing, Calculating Is Speech Special? Language Specificities Bilingualism Speech and Language Disorders (e.g., Aphasia, Dyslexia) Motor Speech Disorders (e.g., Apraxia of Speech, Stuttering) Other Causes of Speech and Language Disorders (e.g., Epilepsy, RightHemisphere Damage).
The reference list contains more than 600 items and includes the most recent research as well as seminal titles. The glossary has almost 600 terms, which will be particularly helpful to the readers who wish to find more information on topics that are covered in the test. I felt that the book would read more easily if extensive definitions and additional explanations were included in the glossary rather than making frequent digressions in the text. Also, some terms are defined differently in different fields, and in those cases the discrepancies are pointed out. A comprehensive subject index and author index are included at the end. Relevant figures can be found throughout the text, but there is an added feature that makes the book more reader-friendly. In the appendix there are figures depicting the brain “geography” for easier navigation along the medial–lateral, dorsal–ventral, and other axes (Figure A.1). Brodmann’s areas with the cerebral lobes (Figure A.2), the lateral view of the brain with the most important gyri, sulci, and fissures (Figure A.3), the midsagittal view, including the most important brainstem and subcortical structures (Figure A.4), the limbic system (Figure A.5), and the coronal view with the basal ganglia (Figure A.6). Since many brain areas are mentioned in several places and contexts throughout the book, rather than leafing back and forth looking for the fixed page where the area was mentioned for the first time, or repeating the illustrations, the figures may be referred to at any point by turning to the appendix. Many friends and colleagues have contributed to the making of this book. First of all I’d like to thank Bill Hardcastle for getting me started and Ray Kent for thought-provoking questions. Special thanks go to Damir Horga, Nadja Runji´c;, and Meri Tadinac for carefully reading individual chapters and providing helpful suggestions and comments. I am immeasurably grateful to Dana Boatman for being with me every step of the way and paying attention to every little detail—from chapter organization to relevant references and choice of terms—as well as to the substance. She helped solve many dilemmas and suggested numerous improvements. Her words of encouragement have meant a lot. Jordan Bi´cani´c was in charge of all the figures. He even put his vacation on hold until they were all completed, and I am thankful that he could include work on this book in
xvi
The Cognitive Neuroscience of Human Communication
his busy schedule. Many thanks to Ivana Bedekovi´c, Irena Martinovi´c, Tamara ˘ Šveljo, and Marica Zivko for technical and moral support. I am grateful to Lawrence Erlbaum Associates and Taylor & Francis Group for giving me the opportunity to write about the topic that has intrigued me for more than a decade. Emily Wilkinson provided guidance and encouragement, and promptly responded to all my queries. Her help is greatly appreciated. I also with to thank Joy Simpson and Nadine Simms for their assistance and patience. Michele Dimont helped bring the manuscript to the final stage with much enthusiasm. Finally, I wish to thank my husband, Boris, for all his help, patience, support, and love. Naturally, it would be too pretentious to believe that this book has answers to all questions regarding speech and language. I hope that it will provide the curious with enough information to want to go on searching. Those who stumble upon this text by accident I hope will become interested. Most of all, I encourage readers to share my fascination with the brain, as well as with speech and language, as unique forms of human communication. —Vesna Mildner
1 Central Nervous System This chapter is an overview of the development, structure, and functioning of the central nervous system, with special emphasis on the brain. All areas that are discussed later, in the chapter on speech and language, are described and explained here, in addition to the structures that are essential for the understanding of the neurobiological basis of speech and language. More information and details, accompanied by excellent illustrations, may be found in a number of other sources (Drubach, 2000; Gazzaniga, Ivry, & Mangun, 2002; Kalat, 1995; Kolb & Whishaw, 1996; Pinel, 2005; Purves et al., 2001; Thompson, 1993; Webster, 1995). For easier reference and navigation through these descriptions, several figures are provided in the appendix. In Figure A.1 there are the major directions (axes): lateral—medial, dorsal—ventral, caudal—rostral, superior—inferior, and anterior—posterior. Brodmann’s areas and cortical lobes are shown in Figure A.2. The most frequently mentioned cortical structures are shown in Figures A.3 through A.6. These and other relevant figures are included in the text itself. At the end of this chapter there is a section on the neural bases of speech production and perception and their interrelatedness.
THE DEVELOPMENT OF THE CENTRAL NERVOUS SYSTEM Immediately after conception a multicellular blastula is formed, with three cell types: ectoderm, mesoderm, and endoderm. Bones and voluntary muscles will subsequently develop from mesodermal cells, and intestinal organs will develop from endodermal cells. The ectoderm will develop into the nervous system, skin, hair, eye lenses, and the inner ears. Two to 3 weeks after conception the neural plate develops on the dorsal side of the embryo, starting as an oval thickening within the ectoderm. The neural plate gradually elongates, with its sides rising and folding inward. Thus the neural groove is formed, developing eventually, when the folds merge, into the neural tube. By the end of the 4th week, three bubbles may be seen at the anterior end of the tube: the forebrain (prosencephalon), the midbrain (mesencephalon), and the hindbrain (rhombencephalon). The rest of the tube is elongated further and, keeping the same diameter, becomes the spinal cord (medulla spinalis). The forebrain will eventually become the cerebral cortex (cortex cerebri). During the 5th week the forebrain is divided into the diencephalon and the telencephalon. At the same time the hindbrain is divided into the metencephalon and myelencephalon. In approximately the 7th gestation week the telencephalon is transformed into cerebral hemispheres, the diencephalon into the thalamus and related structures, while the metencephalon develops 1
2
The Cognitive Neuroscience of Human Communication
into the cerebellum and the pons, and the myelencephalon becomes the medulla (medulla oblongata). During the transformation of the neural plate into the neural tube, the number of cells that will eventually develop into the nervous system is relatively constant—approximately 125,000. However, as soon as the neural tube is formed, their number rises quickly (proliferation). In humans that rate is about 250,000 neurons per minute. Proliferation varies in different parts of the neural tube with respect to timing and rate. In each species the cells in different parts of the tube proliferate in unique ways that are responsible for the species-specific folding patterns. The immature neurons that are formed during this process move to other areas (migration) in which they will undergo further differentiation. The process of migration determines the final destination of each neuron. The axons start to grow during migration and their growth progresses at the rate of 7 to 170 μm per hour (Kolb & Whishaw, 1996). Between the eighth and tenth week after conception the cortical plate is formed; it will eventually develop into the cortex. Major cortical areas can be distinguished as early as the end of the first trimester. At the beginning of the third month, the first primary fissures are distinguishable, for example, the one separating the cerebellum from the cerebrum. Between the 12th and the 15th week the so-called subplate zone is developed, which is important for the development of the cortex. At the peak of its development (between the 22nd and the 34th week) the subplate zone is responsible for the temporary organization and functioning. During that time the first regional distinctions appear in the cortex: around the 24th week the lateral (Sylvian) fissure and the central sulcus can be identified; secondary fissures appear around the 28th week; tertiary fissures start to form in the third trimester and their development extends into the postnatal period (Judaš & Kostovi´c, 1997; Kostovi´c, 1979; Pinel, 2005; Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Further migration is done in the insideout manner: the first cortical layer to be completed is the deepest one (sixth), followed by the fifth, and so on, to the first layer, or the one nearest to the surface. This means that the neurons that start migrating later have to pass through all the existing layers. During migration the neurons are grouped selectively (aggregation) and form principal cell masses, or layers, in the nervous system. In other words, aggregation is the phase in which the neurons, having completed the migration phase and reached the general area in which they will eventually function in the adult neural system, take their final positions with respect to other neurons, thus forming larger structures of the nervous system. The subsequent phase (differentiation) includes the development of the cell body, its axon and dendrites. In this phase, neurotransmitter specificity is established and synapses are formed (synaptogenesis). Although the first synapses occur as early as the end of the 8th week of pregnancy, the periods of intensive synaptogenesis fall between the 13th and the 16th week and between the 22nd and the 26th week (Judaš & Kostovi´c, 1997). The greatest synaptic density is reached in the first 15 months of life (Gazzaniga et al., 2002). In the normal nervous system development these processes are interconnected and are affected by intrinsic and environmental factors (Kostovi´c, 1979; Pinel, 2005; Spreen et al., 1984). In most cases the axons
Central Nervous System
3
immediately recognize the path they are supposed to take and select their targets precisely. It is believed that some kind of a molecular sense guides the axons. It is possible that the target releases the necessary molecular signals (Shatz, 1992). Some neurons emit chemical substances that attract particular axons, whereas others emit substances that reject them. Some neurons extend one fiber toward the surface and when the fiber ceases to grow, having reached the existing outer layer, the cell body travels along the fiber to the surface, thus participating in the formation of the cortex. The fiber then becomes the axon, projecting from the cell body (now in the cortex) back to the original place from which the neuron started. This results in the neuron eventually transmitting the information in the direction opposite to that of its growth (Thompson, 1993). The neurons whose axons do not establish synapses degenerate and die. The period of mass cell death (apoptosis) and the elimination of unnecessary neurons is a natural developmental process (Kalat, 1995). Owing to great redundancy, pathology may ensue only if the cell death exceeds the normal rate (Strange, 1995). The number of synapses that occur in the early postnatal period (up to the second year of life) gradually decreases (pruning) and the adult values are reached after puberty. Since these processes are the most pronounced in the association areas of the cortex, they are attributed to fine-tuning of associative and commissural connections in the subsequent period of intensive cognitive functions development (Judaš & Kostovi´c, 1997). Postmortem histological analyses of the human brain, as well as glucose metabolism measurements in vivo, have shown that in humans, the development and elimination of synapses peak earlier in the sensory and motor areas of the cortex than in the association cortex (Gazzaniga, Ivry, & Mangun, 2002). For example, the greatest synaptic density in the auditory cortex (in the temporal lobe) is reached around the third month of life as opposed to the frontal lobe association cortex, where it is reached about the 15th month (Huttenlocher & Dabholkar, 1997; after Gazzaniga et al., 2002). In newborns, glucose metabolism is highest in the sensory and motor cortical areas, in the hippocampus and in subcortical areas (thalamus, brainstem, and vermis of the cerebellum). Between the second and third month of life it is higher in the occipital and temporal lobes, in the primary visual cortex, and in basal ganglia and the cerebellum. Between the 6th and the 12th month it increases in the frontal lobes. Total glucose level rises continuously until the fourth year, when it evens out and remains practically unchanged until age 10. From then until approximately age 18 it gradually reaches the adult levels (Chugani, Phelps, & Mazziotta, 1987). Myelination starts in the fetal period and in most species goes on until well after birth. From the eighth to the ninth month of pregnancy brain mass increases rapidly from approximately 1.5 g to about 350 g, which is the average mass at birth (about 10% of total newborn’s weight). At the end of the first year, the brain mass is about 1,000 g. During the first 4 years of life it reaches about 80% of the adult brain mass—between 1,250 and 1,500 g. This increase is a result of the increase in size, complexity, and myelination—and not of a greater number of neurons (Kalat, 1995; Kostovi´c, 1979; Spreen, Tupper, Risser, Tuokko, & Edgell, 1984; Strange, 1995). Due to myelination and proliferation of glial cells, the brain
4
The Cognitive Neuroscience of Human Communication
volume increases considerably during the first 6 years of life. Although the white matter volume increases linearly with age and evenly in all areas, the gray matter volume increases nonlinearly and its rate varies from area to area (Gazzaniga et al., 2002). Brain growth is accompanied by the functional organization of the nervous system, which reflects its greater sensitivity and ability to react to environmental stimuli. One of the principal indicators of this greater sensitivity is the development of associative fibers and tracts; for example, increasing and more complex interconnectedness is considered a manifestation of information storage and processing. Neurophysiological changes occurring during the 1st year of life are manifested as greater electrical activity of the brain that can be detected by EEG and by measuring event-related potentials (ERPs; Kalat, 1995). Positron emission tomography (PET) has revealed that the thalamus and the brainstem are quite active by the fifth week postnatally, and that most of the cerebral cortex and the lateral part of the cerebellum are much more mature at 3 months than at 5 weeks. Very little activity has been recorded in the frontal lobes until the age of about 7.5 months (Kalat, 1995). Concurrent with many morphological and neurophysiological changes is the development of a number of abilities, such as language (Aitkin, 1990). In most general terms, all people have identical brain structure, but detailed organization is very different from one individual to the next due to genetic factors, developmental factors, and experience. Genetic material in the form of the DNA in the cell nucleus establishes the basis for the structural organization of the brain and the rules of cell functioning, but development and experience will give each individual brain its final form. Even the earliest experiences that we may not consciously remember leave a trace in our brain (Kolb & Whishaw, 1996). Changes in cortical layers are closely related to changes in connections, especially between the hemispheres. Their growth is slow and dependent on the maturation of the association cortex. Interhemispheric or neocortical connections (commissures) are large bundles of fibers that connect the major cortical parts of the two hemispheres. The largest commissure is the corpus callosum, which connects most cortical (homologous) areas of the two hemispheres. It is made up of about 200 million neurons. Its four major parts are the trunk, splenium (posterior part), genu (anterior part), and the rostrum (extending from the genu to the anterior commissure). The smaller anterior commissure connects the anterior parts of temporal lobes, and the hippocampal commissure connects the left and the right hippocampus. The hemispheres are also connected via massa intermedia, posterior commissure and the optic chiasm (Pinel, 2005). Most interhemispheric connections link the homotopic areas (the corresponding points in the two hemispheres; Spreen et al., 1984), but there are some heterotopic connections as well (Gazzaniga et al., 2002). Cortical areas where the medial part of the body is represented are the most densely connected (Kolb & Whishaw, 1996). It is believed that neocortical commissures transfer very subtle information from one hemisphere to the other and have an integrative function for the two halves of the body and the perceptual space. According to Kalat (1995), information reaching one hemisphere takes about 7 to 13 ms to cross over to the opposite one. Ringo, Doty, Demeter, &
Central Nervous System
5
Simard (1994), on the other hand, estimate the time of the transcortical transfer to be about 30 ms. Ivry and Robertson (1999) talk about several milliseconds. In their experiments on cats, Myers and Sperry (as cited in Pinel, 2005) have shown that the task of the corpus callosum is to transfer the learned information from one hemisphere to the other. The first commissures are established around the 50th day of gestation (anterior commissure). Callosal fibers establish the interhemispheric connections later and the process continues after birth until as late as age 10 (Kalat, 1995; Lassonde, Sauerwein, Chicoine, & Geoffroy, 1991). Corpus callosum of left-handers was found to be about 11% thicker than that of the right-handers, which was attributed to greater bilateral representation of functions (Kalat, 1995; Kolb & Whishaw, 1996). There is disagreement among authors considering the sex differences in callosal size (for more information, see chap. 2, this volume). Myelination of corpus callosum proceeds during postnatal development and it is one of the parts of the nervous system whose myelination begins and ends last. It is thought that the callosal evolution has an impact on hemispheric specialization (Gazzaniga et al., 2002). In Alzheimer’s patients, the area of corpus callosum, especially of its medial part (splenium), is significantly smaller than in healthy individuals (Lobaugh, McIntosh, Roy, Caldwell, & Black, 2000). After the age of 30 the brain mass gradually decreases and by the age of 75 it is approximately 100 g smaller (Kolb & Whishaw, 1996). Although the brains of people in their seventies have fewer neurons than the brains of younger people, in healthy elderly individuals the decrease is compensated for by the dendrites of the remaining neurons becoming longer and branching more (Kalat, 1995). Some recent studies have revealed that in rare cases and in a very limited way in some parts of the brain, particularly in the hippocampus and the olfactory bulb (bulbus olfactorius), a small number of neurons may develop after birth and during lifetime (Purves et al., 2001). However, whether these newly formed neurons have any function in the adult nervous system remains to be determined (Drubach, 2000; Gage, 2002; Gazzaniga et al., 2002; Gould, 2002).
STRUCTURE AND ORGANIZATION OF THE CENTRAL NERVOUS SYSTEM The nervous system consists of nerve cells—neurons—and glial cells (that will be discussed later). The neuron is a functional and structural unit of the brain. It consists of the cell body (soma) with the nucleus built from DNA, and other structures characteristic of cells in general, one or more dendrites, and one axon that ends with the presynaptic axon terminals (Figure 1.1). The neuron transmits information to other cells and receives information from them. The dendrites and the body receive information while the axon transmits information to other neurons. The space between neurons is filled with extracellular fluid so that in general they are not in direct contact. The size of the smallest neurons is approximately 7 to 8 μm, whereas the largest ones range in size between 120 and 150 μm (Judaš & Kostovi´c, 1997). Axons of some human neurons may be one meter or longer, whereas others do not
6
The Cognitive Neuroscience of Human Communication Dendrite
Cell membrane Soma (cell body) Nucleus
Myelin sheath Node of Ranvier
Dendritic spines
Axon hillock
Axon
Axon terminals
FIGURE 1.1. A stylized neuron.
exceed several tens or hundreds of micrometers. Most axons are between several millimeters and several centimeters long. The diameter of the thinnest axons is about 0.1 μm. At the end of each axon there are usually several smaller fibers that end with the terminal node. Each node synapses with another cell. A neuron may have a few short fibers or a huge number. The greater the number of dendrites, the greater the receptive ability of the neuron. In the cerebral cortex, many neurons’ dendrites are covered by literally thousands of little processes—dendritic spines. Since each one of them is a postsynaptic part of the synapse, the number of connections is greatly increased. These synapses are most probably excitatory. All neurons, from the simplest to the most complex organisms, rely on identical electrochemical mechanisms for information transmission. The considerable differences in neuronal organization, for example, in the patterns of their interconnections, are responsible for the functional differences that distinguish, for instance, the humans from other species. Neurons may be grouped into pathways or tracts (a simple series of neurons)—for example, the auditory pathway; into neuronal circles or networks; and into neuronal systems—for example, the auditory system. Each neuron may have connections to thousands of other neurons, which means that it may affect their activity. It can in turn be influenced by thousands of other neurons with excitatory or inhibitory results. There are no unnecessary or reserve neurons—each one has a function. Neuronal populations differ in size, shape, manner of information processing, and transmitters that they use to communicate with other neurons. Those that occupy neighboring positions in the brain and share common functions usually belong to the same population and have identical physical and functional properties. This principle is metaphorically referred to as “Neurons that fire together wire together.” This means that some
Central Nervous System
7
are specialized for visual information, others for auditory stimuli, and still others for emotional expressions. These functions are not interchangeable. However, in spite of such highly specialized properties, there are limited possibilities for the neurons neighboring those that have been injured to take over and assume the new function, different from their original one, which results in the neurofunctional reorganization of the entire affected area. This issue will be addressed in more detail in the context of plasticity in chapter 7, this volume. Motor (efferent) neurons have richly arborized dendrites, a large body, and a long myelinated axon. They send out their fibers from the nervous system toward the body (parts) and at their ends synapse with muscle fibers and gland cells. They control the activity of skeletal muscles, smooth muscles, and glands. They are controlled by several systems in the brain that are called motor systems. Sensory (afferent) neurons extend from the body to the brain. Their cell bodies are located along the spinal cord in groups. They are the so-called ganglia. One of their most important properties is selective detection and enhancement of particular stimulus features. The cerebral cortex is made up of several hundred different types of neurons. They are either pyramidal neurons (principal neurons) or extrapyramidal (interneurons). The principal neurons of a particular area are responsible for the transmission of the final information into other cerebral areas after the processing of incoming information. They are excitatory neurons and make up about 70% of all cortical neurons. They are rich in dendritic spines that contribute to the richness of connections; their axons are long and make projection, association, or commissural fibers. Numerous collateral branches (collaterals) of these axons are the greatest sources of excitatory postsynaptic potentials in the cerebral cortex (Judaš & Kostovi´c, 1997). Interneurons are mainly inhibitory and make up to 30% of all cortical neurons. Their dendrites have no dendritic spines; their axons are short and establish local connections. Neuronal feedback is essential for optimal brain functioning; the brain adjusts its activity on the basis of it. The number of neurons is greatly (perhaps tenfold) exceeded by the number of glial cells (glia). They make up about 50% of the total brain tissue volume (Judaš & Kostovi´c, 1997). They are not neural cells because they do not transmit information. Their function is not entirely clear, but their roles include absorbing substances in the brain that are not necessary or are excessive (e.g., at the synapses they often absorb excess neurotransmitters); after a brain injury proliferating in the location of neuron damage and removing cell debris, making the so-called glial scars; forming myelin sheaths; establishing the blood–brain barrier; guiding migrating neurons on the path to their final destinations, and so forth (Thompson, 1993; Judaš & Kostovi´c, 1997). Many types of glial cells communicate among themselves and with neurons as well. The neuron–glia–neuron loop is therefore considered to be a more precise description of a communication unit, rather than the simple neuron–neuron connection (Gazzaniga et al., 2002). The myelin sheath is basically fat. It enables faster propagation of the action potential along axons. There are interruptions in the sheath where the axon is in direct contact with the extracellular fluid, enabling the occurrence of action
8
The Cognitive Neuroscience of Human Communication
potentials. These interruptions are the so-called nodes of Ranvier (nodi Ranvieri). Myelinated axon segments between the two unmyelinated points are between 200 μm and 1 mm long. Their length depends on the axon type: the greater the axon diameter, the longer the myelinated segments. Consequently, longer myelinated segments, in other words, longer distances between the two nodes, will result in faster impulse conduction. The width of each node is about one μm. A synapse is a functional connection between two neurons or between a neuron and another target cell (e.g., muscle). It is the point at which the information in the form of a nerve impulse (signal) is transmitted. A tiny space, between 10 to 100 nanometers wide, separates the axon terminal of one cell from the body or a dendrite of another cell with which it communicates. That space is called the synaptic cleft. Synapses can be found only in nerve tissue, because they are formed only between neurons and their target cells. Synapses are functionally asymmetrical (polarized), which means that signal transmission is one-way only. Having said that, it is important to bear in mind the existence of the neuronal feedback—a process that enables communication among nerve cells. There are two kinds of synapses—chemical and electrical. Most synapses in the brain of mammals are chemical (Figure 1.2; the illustration shows synaptic transmission at a chemical synapse; adapted from Purves et al., 2001). They may be excitatory or inhibitory. Excitatory synapses increase the activity of the target cell, in other words, the probability of occurrence of action potential. Inhibitory synapses decrease target cell activity. The signals are transmitted by means of neurotransmitters that are released (provided that the threshold of activation has been reached) from the presynaptic neuron into the synaptic cleft, from which they are taken up by the corresponding receptors in the postsynaptic membrane. This transfer is very precise and takes less than 1 millisecond. Neurotransmitters are chemical substances that are produced in the presynaptic neuron and stored in the vesicles in the presynaptic axon terminal. About a hundred different kinds of neurotransmitters are known at present. Different neuronal populations produce and react to only one (or a very limited number) type of neurotransmitter. Apart from the excitatory and inhibitory neurotransmitters there are the so-called conditional neurotransmitters, whose activity is affected by the existence of another neurotransmitter or by the neuronal circuit activity (Gazzaniga et al., 2002). Activation of a single excitatory synapse in the neuron is not sufficient for it to fire: several excitatory synapses have to be activated simultaneously (spatial summation) in order to reach the threshold of action potential. Even simultaneous activation of several synapses may not always result in firing. In such cases these groups of synapses must be activated several times in a row, in short intervals (temporal summation). The same principles of spatial and temporal summation apply to inhibitory synapses. A normal neuron constantly integrates temporal and spatial pieces of information and “makes decisions” on whether to fire or not (neuronal integration). The moment of making a positive decision is the point of reaching the action potential threshold at the axon hillock, which is the result of domination of excitatory over inhibitory effects. It is also possible that a subliminal stimulus (i.e., the one that is not sufficient in itself to reach the action potential thresholds)
Central Nervous System
9
Presynaptic neuron Synaptic vesicle
Synaptic cleft
Postsynaptic neuron
Synaptic vesicle fusing Presynaptic membrane Synaptic cleft
Neurotransmitter
Postsynaptic membrane Postsynaptic neurotransmitter receptor
FIGURE 1.2. Chemical synapse; synaptic transmission. (From Purves et al., 2001. With permission.)
prepares the postsynaptic neuron for the arrival of another subliminal stimulus, which will (owing to that earlier stimulus) reach the action potential threshold. This phenomenon is called facilitation. It may be achieved by direct activation of excitatory synapses or by inactivation of inhibitory synapses (Judaš & Kostovi´c, 1997). Experience has a huge effect on the strengthening or weakening of synapses. As opposed to chemical synapses that exhibit a great deal of plasticity (i.e., may be changed in various ways with increased or decreased activity), electrical synapses are rigid and unchangeable. In electrical synapses there are no synaptic clefts between neurons: their membranes are in direct contact and their cytoplasms are connected through transmembranous channels (gap junction). As a consequence, such neurons have identical potentials, and electrical changes in one of them are immediately reflected on the other. At electrical synapses the transmission is very fast and it may be bidirectional (Purves et al., 2001). Learning and memory would be impossible in a nervous system that had only electrical synapses. However, some electrical synapses have been found in mammalian brains—mainly between cell bodies of neighboring neurons that share the same function.
10
The Cognitive Neuroscience of Human Communication
Nerve impulses are transmitted at the rate of about 1 to 100 meters per second. As it has been mentioned earlier, the rate at which the impulse travels through the axon depends on its diameter (the greater the diameter, the faster the transmission) and on whether it is myelinated or not (the impulse travels faster through myelinated fibers; Sternberg, 2003). The firing rate generally does not exceed 100 times in a second, although some neurons may fire at the very high rate of 1,000 times in a second. A common feature of all neurons is that they function on the all-or-none principle. This means that the action potential will either occur or not. The messages differ in the action potential frequency and the timing of impulses (Ferster & Spruston, 1995; Kalat, 1995). However, neurons may also determine the content of the message that is being transmitted by varying, among other things, the type, quantity, and rate of neurotransmitter release (Drubach, 2000). Nerves are bundles of nerve fibers. The nerves transmitting the information toward the central nervous system and from it are peripheral nerves. They transmit impulses from the periphery toward the center (sensory fibers, afferent pathways), from the center toward the periphery (motor fibers, efferent pathways), or are positioned between sensory and motor fibers (interneurons). Somatic nerves establish connections with the voluntary skeletal muscles and sense organs. Autonomic nerves are connected with internal organs and glands involved in autonomic aspects of reactions, usually related to emotional behavior (crying, perspiration, some activities of the heart and stomach). All organs are controlled by the sympathetic part of the autonomic nervous system, and some are controlled both by the sympathetic and parasympathetic segment. Peripheral nerves are cranial or spinal. Cranial nerves (nervi craniales) transmit sensory information from the face and head, and commands for motor control over face and head movements. This means that their functions are sensory, motor, or combined. There are 12 pairs of cranial nerves, seven of which are essential for speech production (V through XII). They are numerated from the anterior to the posterior part of the brain: I II
The olfactory nerve (fila olfactoria) transmits olfactory information from the nose to the brain (into the telencephalon). The optic nerve (nervus opticus), actually a part of the visual pathway, transmits visual information from the eyes (it is directly connected to the diencephalon). Nerves III through XII are directly connected to the brainstem:
III IV V
The occulomotor nerve (nervus occulomotorius). The trochlear nerve (nervus trochlearis) and VI—the abducens nerve (nervus abducens) are responsible for eye muscles. The trigeminal nerve (nervus trigeminus) transmits sensory information from the skin of the face and head and innervates the jaw muscles and tensor tympani muscle in the middle ear.
Central Nervous System
11
VII The facial nerve (nervus facialis) has afferent and efferent connections with parts of the face, ear (outer ear, stapedial muscle), tongue, and larynx. VIII The vestibulocochlear nerve (nervus statoacusticus or nervus vestibulocochleris) transmits information about sounds from the ears and about vestibular sense from vestibular part of the inner ear. Some efferent pathways related to that nerve have been found as well (Kent & Tjaden, 1997). IX The glossopharyngeal nerve (nervus glossopharyngeus) innervates laryngeal muscles that play an important role in the process of swallowing; it also transmits the sense of taste and sensory information from the outer ear. X The vagus nerve (nervus vagus) is important for the autonomic (parasympathetic) control of the heart and other internal organs; it innervates laryngeal and pharyngeal muscles important for phonation; and it transmits the sensations from the outer ear and the sense of taste from the area around epiglottis. XI The accessory nerve (nervus accessorius) innervates neck and shoulder muscles as well as the muscles controlling the soft palate. XII The hypoglossal nerve (nervus hypoglossus) innervates tongue muscles. There are 31 pairs of spinal nerves (nervi spinales). They protrude from the spinal column and innervate the muscles with their motor parts in the anterior portions (they come out from ventral roots). Or they receive afferent information with their sensory parts in the posterior portions (these enter the dorsal roots), to forward it through the spinal cord to the thalamus and the cerebral cortex. The spinal cord (medulla spinalis) is a tube-like structure that extends downward from the brainstem and connects the brain with the parts of the body below the neckline. It is a major thruway for (a) information from the cerebral cortex and other brain structures that control body movements going toward motor neurons (and indirectly through them to the muscles) and toward all the organs in the body (autonomic nervous system), and for (b) information from all receptors, including proprioceptive, and information from some peripheral organs to the brain. The spinal cord is also responsible for reflex muscle and autonomic responses to stimuli. On their way from the brain to the periphery the axons cross over from one side of the tract to the other. Consequently, the motor control of the right side of the body is situated in the left cerebral hemisphere, whereas the right hemisphere controls the left side of the body. Information from the periphery also projects to the hemisphere contralateral to the stimulated side (Drubach, 2000). The brain (encephalon) is a part of the central nervous system (together with the spinal cord). The principal parts of the brain are the following: brainstem, cerebellum, and cerebrum (Figure 1.3). The brainstem (truncus cerebri or truncus encephalicus) consists of the medulla (medulla oblongata; myelencephalon), pons (pons; metencephalon) and the midbrain (mesencephalon). It connects the cerebrum and the cerebellum with the spinal cord, but it also has functions of its own (Drubach, 2000; Judaš & Kostovi´c, 1997; Purves et al., 2001; Webster, 1995). In it there are centers, the
12
The Cognitive Neuroscience of Human Communication Cingulate sulcus Central sulcus Cingulate gyrus
Precuneus
Fornix
Parieto-occipital sulcus Corpus callosum
Thalamus
Superior colliculus Midbrain
Hypothalamus Optic chiasm
Pons
Pituitary Cerebellum
Medulla
Brainstem
Inferior colliculus
Anterior commissure
Spinal cord
FIGURE 1.3. Midsagittal view of the brain.
so-called nuclei, made up of groups of neurons, which regulate body temperature; cardiac, respiratory, and gastrointestinal systems; blood vessels; and consciousness. Among other things, they also control the responses to visual and auditory stimuli, movement and, to some extent, wakefulness and sleep. It is believed that motor control of speech stems directly from the basic centers for swallowing and breathing control. Afferent (ascending, sensory) and efferent (descending, motor) pathways go through the brainstem. Reticular formation (formatio reticularis) is an important part of the brainstem. It is a heterogeneous set of functionally very different structures (Judaš & Kostovi´c, 1997) through which pass almost all sensory and motor pathways. It is connected with other nuclei in the brainstem, cerebellum, diencephalon and the cerebrum. Injuries to the brainstem may cause paralysis, loss of sensation and/or control of corresponding functions and consciousness, coma, and even death. The cerebellum is located in the posterior part of the cranial cavity. Its main parts are the cortex, subcortical white matter, and subcortical nuclei. It is one of the philogenetically oldest brain structures. Its surface is convoluted, with fissures, sulci, and gyri much more densely folded than in the cerebrum. It has two hemispheres and a medial part (vermis). The cerebellar cortex is functionally organized in three parts. Various functions are differentiated by specific input– output neuronal connections. On average, its mass is about 145 g. Its cortex is about 1 to 1.5 mm thick and consists of three cellular layers. Despite its relatively small size in proportion to the total brain volume (10%), it contains more than 50% of all neurons of the brain (Judaš & Kostovi´c, 1997). There is a threepronged connection (cerebellar penduncles) with the rest of the nervous system. Initiation of a voluntary movement (e.g., reaching for an object) will at first reveal a neural activity in the cerebellum (and in the basal ganglia), followed by the activity in the motor cortex of the cerebrum, that has turned out to be responsible for precise performance of fine voluntary movements but not for their initiation
Central Nervous System
13
(Thompson, 1993). The cerebellum receives a complex set of sensory information from most modalities: vestibular and auditory, muscles, joints, skin, and the eyes. It is the central place for proprioceptive information. Information related to proprioception, motor planning, and vestibular sense is integrated in the cerebellar cortex. The number of afferent fibers by far exceeds the number of efferent ones; in humans the ratio is about 40 to 1 (Judaš & Kostovi´c, 1997). This means that the cerebellum is an important place for integration and processing of input information before transmitting the output signal to other parts of the brain. On the basis of continuous inflow of sensory information from the periphery, the cerebellum coordinates and smooths out activity of the muscles, and in cooperation with the vestibular system coordinates head movements and body position with all other activities of the body. In other words, it regulates the speed, range, force, and the orientation of movements (Webster, 1995). It seems that the cerebellum plays an important role in learning, particularly in learning and remembering skilled movements, but also in nonmotor learning and numerous cognitive processes (Drubach, 2000). Injuries to the cerebellum may cause clumsiness (particularly purposeful hand movements), balance difficulties, decreased muscle tone, incomprehensible speech, imprecise eye movements, and impaired planning and timing of activities (Drubach, 2000; Kalat, 1995). It might be said that the main role of the cerebellum is to perform temporal calculations that may be used in various perceptive and motor functions, including speech and language. For example, estimating the duration of a sound is impaired in individuals with cerebellar injuries (Gazzaniga et al., 2002; Kent & Tjaden, 1997). As opposed to cerebral injuries, cerebellar injuries cause impairments on the ipsilateral side of the body (due to double crossing of pathways). Body parts are mapped onto the surface of the cerebellar cortex, similarly to the representations found in the cerebral cortex, but due to the lack of association or commissural fibers, each area receives a separate afferent projection and acts as a separate functional unit (Judaš & Kostovi´c, 1997). It is interesting to note that the cytoarchitectonic organization of the cerebellum is identical in all mammals—from mouse to human. The cerebellum and the motor cortex of the cerebrum constitute a constantly active movement control system (Thompson, 1993). The diencephalon is located between the brainstem and the cerebral hemispheres. Its principal parts are the thalamus, hypothalamus, epithalamus, and the subthalamus. Thalamus (originating from the Greek word thalamos, meaning bed or bedroom) is a structure almost in the very center of the brain, where numerous fibers synapse and cross. The visual and the auditory signals pass through it and it is also involved in movement control. It has connections with the cerebellum and the basal ganglia. Messages from the limbic system (of which the hypothalamus and epithalamus are important parts) are relayed to the thalamus. The thalamus integrates and interprets signals prior to forwarding them to other parts of the brain. It also plays an important role in memory. With the exception of olfactory information, all stimuli are processed in thalamic nuclei (seven groups in all) before reaching the cerebral cortex. Primary sensory and motor cortical areas receive direct projections from thalamic nuclei. Below the
14
The Cognitive Neuroscience of Human Communication
thalamus are the subthalamus and hypothalamus, responsible for emotions, basic body functions, body temperature, pituitary gland activity, hormone release, food and liquid intake, sexual behavior, and circadian rhythms. Below the posterior part of the thalamus (pulvinar) there is the lateral geniculate complex (corpus geniculatum laterale) that is a part of the visual system, and the medial geniculate complex (corpus geniculatum mediale) that is a part of the auditory system. Recent studies stress the importance of subcortical structures, particularly thalamus and hypothalamus for higher cognitive functions, including language and speech. The hypothalamus is believed to have a role in memory formation (Kent & Tjaden, 1997). Thalamus attracts attention and directs it to verbal information, recall, and so forth. Its role is to enhance and emphasize the information to which the attention is directed at the moment. However, the consequence of thalamic injuries on naming, word finding, arithmetic, verbal short-term memory, and fluency are most frequently transient and short-lived, as opposed to cortical injuries (Bradshaw & Nettleton, 1983). Thalamic injuries typically cause loss of sensation, motor disorders, and consciousness problems. The cerebrum consists of white (about 39% of volume) and gray (about 61% of volume) matter. There are about 1011 to 1012 neurons in the cerebrum, of which about 50 billion are directly involved in information processing (Kolb & Whishaw, 1996; Strange, 1995). Each neuron makes about 1,000 to 3,000 connections with other neurons, which means that the number of connections is huge—about 1014 (Churchland, 1988). In the cerebral cortex there are about 10 billion neurons. The mass of the adult brain ranges from 1,100 to 1,700 g. On average, adult male brains weigh approximately 1,450 g, and adult female brains about 1,300 g, which makes about 2.5% of total body mass. The smaller mass of the female brain does not imply poorer abilities. First of all, smaller mass is compensated for by richer connections, and second, it seems that the ratio of brain mass to total body mass is a better indicator of brain development than its absolute mass (Drubach, 2000). For example, whales and elephants have larger brains than humans, but the density of their cortical cells is most likely smaller. The differences in cortical size are also associated with the differences in the cerebral cortex circuits (Hill & Walsh, 2005). In any case, of all the species, humans have the largest brain with respect to their body size. Note also that the newborn’s ratio of brain mass to body mass is greater than in the adult (Sternberg, 2003). For as yet unknown reasons, in the past 3 million years there has been an explosion in the size of the human brain, unrecorded in any other species. This sudden growth is mostly attributable to the considerable enlargement of the cortex (Thompson, 1993). The brain is immersed in the cerebrospinal fluid and protected by the firm bony shell of the skull and three tissue layers (meninges). It is connected to the rest of the body (i.e., to the peripheral nervous system) by means of 12 pairs of cranial nerves. The connection with the spinal cord is realized through rich nerve connections (descending efferent and ascending afferent pathways; Judaš & Kostovi´c, 1997). It is generally believed that brains of exceptional people do not differ from those of average individuals with respect to mass, structure, and functioning. However, there is some evidence obtained by magnetic resonance imaging that the
Central Nervous System
15
correlation between intelligence and brain size is about 0.35 (Kalat, 1995). Moreover, by comparing Einstein’s brain with several dozen brains of average people Wittleson found considerable differences in the position of Sylvian fissure as well as in the area and thickness of the inferior temporal lobe (Gazzaniga et al., 2002). Human brains differ from the brains of other primates mainly in the richness of their associative (corticocortical) and projection (subcortical) connections and in quantitatively different organization. For example, the prefrontal area is twice as large as that in other primates, whereas the motor, olfactory, and visual areas are smaller (Kalat, 1995; Kolb & Whishaw, 1996). Philogenetic studies have shown that the proportions of various functional areas change during the development of the species. White matter consists of bundles of myelinated neuron axons (myelin is responsible for its white color) that connect the two hemispheres (commissural fibers), the cerebral cortex with lower parts of the nervous system (afferent and efferent projection fibers), or that connect various parts of the same hemisphere (short and long association fibers). Deep within the white matter are the ventricles (ventriculi; Judaš & Kostovi´c, 1997). Gray matter is made of the cell bodies and their dendrites. It is found on the surface (cortex cerebri, cortex) and deep within the brain separated from the cortex by white matter (basal ganglia; Webster, 1995). The area of the convoluted surface layer is about 2,200 square cm and its thickness varies between 1.5 and 4.5 mm. The fissures (fissurae) and sulci divide the surface into lobes (lobi), lobules (lobuli), and gyri (Judaš & Kostovi´c, 1997). Most of the cortex is organized into six layers of cells that have ontogenetically developed from the inside out and make up the neocortex. This term is synonymous with the term isocortex, reflecting the fact that each part of the adult cerebral cortex developed from the same developmental base, and makes up almost 90% of the total cerebral cortex (Judaš & Kostovi´c, 1997). During development, in a smaller part of the isocortex the number of layers either decreases or increases. In evolutionarily older parts of the cortex (allocortex) there are commonly fewer than six layers: two in the paleocortex, three in the archicortex, and five in the mesocortex. These parts make up the limbic system. The five basic functional groups of cortical areas to the largest extent correspond to the basic types of cerebral cortex. They are: (a) primary sensory and motor areas; (b) unimodal association areas; (c) heteromodal association areas; (d) limbic areas; and (e) paralimbic areas. The term cortex is commonly used to refer to neocortex. The cells vary in form, size, and distribution across areas, so cytoarchitectonically we talk about nuclei, layers (laminae), areas, and regions that have different cell structure (Judaš & Kostovi´c, 1997). This diversity is partly responsible for the complexity of brain structure. Kolb and Whishaw (1996) have summarized the principles of cortical organization: 1. The cortex is made of many different types of neurons that are organized into six layers (laminae). 2. The cortex is organized into functional columns (columnae), which means that the neurons that share similar functions are grouped into
16
The Cognitive Neuroscience of Human Communication
3. 4. 5. 6.
columns that stretch throughout the cortex, making the cortical column the principal organizational and functional unit. There are multiple representations of sensory and motor functions in the cortex. These functions are plastic. Cortical activity is influenced by feedback from several regions of the forebrain (e.g., from the limbic system or the basal ganglia). The cortex operates on the principles of hierarchical and parallel information processing.
The cerebrum is divided by the longitudinal fissure (fissura longitudinalis cerebri) into two hemispheres that are connected by three large systems of commissural fibers: corpus callosum, anterior commissure (commissura anterior), and the hippocampal commissure (commissura hippocampi). Corpus callosum is the largest and the most important interhemispheric connection. Each hemisphere is divided morphologically into four lobes clearly delimited by anatomical landmarks (fissures and sulci): (a) central sulcus, which is sometimes referred to as Rolandic fissure (fissura centralis Rolandi); (b) lateral fissure, also called the Sylvian fissure (fissura lateralis cerebri Sylvii); and (c) the parieto–occipital fissure (sulcus) or incision (fissura s. incisura parietooccipitalis). The sulci and fissures alternate with gyri. At the bottom of the Sylvian fissure there is the insula that it sometimes referred to as the fifth lobe (Drubach, 2000; Judaš & Kostovi´c, 1997; Figure 1.4). At the beginning of the 20th century Corbinian Brodmann used different staining techniques on samples of brain tissue to establish a system of cell types with respect to their structure (cytoarchitecture). He drew brain sketches using different symbols to represent cell groups differing in shape, density, and laminar organization, assuming that cells that have the same or similar structure perform Central sulcus Precentral gyrus Postcentral gyrus Superior frontal gyrus Superior parietal lobule Middle frontal Supramarginal gyrus gyrus Occipital gyri
Pars opercularis + Pars triangularis
Parieto-occipital sulcus Angular gyrus
Inferior frontal gyrus Orbital gyri Lateral fissure Temporal pole Superior temporal gyrus
Pre-occipital notch
Middle temporal gyrus Inferior temporal gyrus
FIGURE 1.4. Lateral view of the brain.
Central Nervous System
17
the same or similar functions. This yielded about 50 cytoarchitectonically relatively homogenous areas that are called Brodmann’s areas and are marked by Arabic numerals (Figure 1.5). The frontal lobe (lobus frontalis) is anterior to the central sulcus and above the lateral fissure. It is the seat of primary and secondary motor areas. The primary motor area corresponds to Brodmann’s area 4. The secondary motor area includes premotor cortex and the supplementary motor area (dorsolateral and medial parts of Brodmann’s area 6, respectively), Brodmann’s areas 8, 44, and 45, and the posterior cingulate area. Each location in the primary motor areas controls a particular group of muscles on the opposite side of the body. Secondary motor areas are anterior to the primary ones. Each of them controls several primary centers and they are responsible for complex movements and voluntary muscle control. In the language-dominant hemisphere (usually left, but see chap. 8, this volume, for discussion), in Brodmann’s area 44 and probably 45, there are centers that control speech (Broca’s area) and writing. The prefrontal region is responsible for memory storage and retrieval, ethical attitudes, decision making, and psychological makeup of a person (Judaš & Kostovi´c, 1997). In humans, this region constitutes about half of the entire frontal lobe. It is connected with almost all other parts of Parietal lobe
Frontal lobe 4 3
6 8 46
10
7
2 19
40
9 45 44
43
47 11
5
1
52 38
22
39
41
18 17
42
Occipital lobe
37
21
Insula Temporal lobe
20
Frontal lobe
4
6
31 2
8 9
7 31
24
23
32 33
10
Parietal lobe
5
30
26 27 29
11
18
25 38
19
17 34 28
35 36
37
19
Occipital lobe
18
20
Temporal lobe
FIGURE 1.5. Brodmann’s areas in lateral and midsagittal view, including the four lobes.
18
The Cognitive Neuroscience of Human Communication
the brain and receives information from all sensory areas, memory and emotional stores (Kalat, 1995; Webster, 1995). The prefrontal cortex sends projections into all areas from which it receives them, into premotor and motor regions in both hemispheres. The motor cortex is considered to be a place where a multitude of signals involved in initiating and shaping motor control from other parts of the cortex and deeper levels, such as basal ganglia or the cerebellum, are integrated (McKhann, 1994). It seems that, in addition to the neurons in the primary motor cortex, there are a number of circuits, including the cerebellum, basal ganglia, and the thalamus, that generate motor activity. Consequently, it is logical that the motor aspect of language relies on similar circuits. Almost all types of behavior involve both frontal lobes, so in most tasks, with the exception of higher cognitive functions, unilateral brain injury will have negligible consequences. Frontal lobes are crucial in learning a new task that requires active control. Once the activity has become routine other parts of the brain may assume control (Lieberman, 1991). Raichle (1994) proposed that the parts of the nervous system that are involved in learning certain motor patterns are not the same ones that are used for performing the once learned patterns. Functional magnetic resonance imaging (fMRI) data revealed increased activity in the supplementary motor area and lateral premotor cortex after initial task presentation—when the planning of movement starts. As planning turns into execution the activity shifts toward the more posterior regions, and as the movement becomes more complex, the structures outside the primary motor cortex are activated (Gazzaniga et al., 2002). Some of the manifestations of frontal lobe injuries are impaired motor functions, ignoring of social conventions, inflexible and unorganized behavior, inability to correct errors, perseveration, poor temporal memory, poor egocentric orientation, changed social and sexual behavior, and disorders related to damage of face representations, including language and speech or some of their segments (Kolb & Whishaw, 1996). Positron emission tomography (PET) data have revealed that frontal lobes are activated during internal generation of stimuli (by the subject), as opposed to external stimuli, which has led to the conclusion that they probably play a part in conscious distinction between actual and imagined stimuli (Drubach, 2000). During evolution, frontal lobes of humans have undergone enormous enlargement, particularly in the anterior portions. This enlargement is related to the higher cognitive abilities characteristic of humans (Gazzaniga et al., 2002). In fact, there seems to be a correlation between evolutionary patterns and gene function in humans. Characterization of genes for neurological disorders (such as mental retardation, autism, and dyslexia) that affect intelligence, social organization, and higher order language will hopefully shed more light on human evolutionary history (Hill & Walsh, 2005). The parietal lobe (lobus parietalis) is bordered anteriorly by the central sulcus, posteriorly by the parieto–occipital fissure (sulcus), and laterally by the Sylvian (lateral) fissure. Immediately posterior to the central sulcus there are primary somesthetic centers that receive information from sensory organs (proprioception, touch, pressure, temperature, pain; Brodmann’s areas 1, 2, and 3). In the postcentral gyrus there are four parallel representations of the body (Kalat,
Central Nervous System
19
1995). A part of that lobe is the angular gyrus (gyrus angularis), which plays a very important role in word reading and arithmetic (in the left hemisphere). The inferior parietal lobe is involved in writing, which makes that lobe (together with the temporal lobe) essential for language processing and comprehension. That lobe is also important for orientation on one’s own body, particularly with respect to the left–right orientation. The disorders that occur as a consequence of injury to this lobe (apraxia, tactile agnosia, alexia, agraphia, acalculia, autotopagnosia) suggest that it is important for secondary processing of input information (i.e., for coordination of input information from sense organs and output commands to the muscles). It is particularly important for the association and coordination of visual and spatial information (Kalat, 1995). This lobe also seems to be the seat of the short-term/working memory (Kolb & Whishaw, 1996; for more discussion on types of memory see chapter 7, this volume). The temporal lobe (lobus temporalis) is inferior to the lateral (Sylvian) fissure, and extends posteriorly to the parieto–occipital fissure (sulcus). When the lateral fissure is pulled open one to three Heschl’s gyri (gyri temporales transversi Heschl) are revealed. This area is the seat of cortical representation of the sense of hearing—the auditory cortex (Brodmann’s areas 41 and 42). Immediately posterior to Heschl’s gyri is a relatively flat area, the so-called planum temporale. The posterior part of the superior temporal gyrus (Brodmann’s area 22), commonly in the left hemisphere, is the seat of secondary processing of auditory speech stimuli—the center for speech perception (Wernicke’s area). Although processing of music stimuli has been associated with right-hemisphere function, there is evidence that hemispheric differences are dependent on proficiency in music (Ivry & Robertson, 1999; Pinel, 2005). Amusia patients reveal that the anterior parts of the superior temporal gyri are responsible for music production and processing (Gazzaniga et al., 2002; Grbavac, 1992). The cortical representation of vestibular function is also in the superior temporal gyrus. The temporal lobe is involved in complex aspects of visual information processing, for example, in face recognition (Kalat, 1995). The temporal lobe in the left hemisphere is the seat of verbal long-term memory (recollection of stories, word lists, etc., regardless of modality of their presentation). The temporal lobe in the right hemisphere is the seat of nonverbal long-term memory (geometrical drawings, faces, tunes etc.; Kolb & Whishaw, 1996). The temporal lobe is also the seat of learning and memory functions that require conscious effort—declarative or explicit learning (Kandel & Hawkins, 1992). Injuries to the temporal lobe cause disorders in stimulus categorization, and Kolb and Whishaw (1996) have grouped the consequences of such disorders into eight groups: 1. 2. 3. 4. 5.
Disorders of auditory sense and perception. Disorders of selective attention in the auditory and visual modality. Disorders of visual perception. Disorders in the organization and categorization of verbal material. Disorders in language comprehension, including the inability to use context.
20
The Cognitive Neuroscience of Human Communication
6. Disorders in visual and auditory long-term memory. 7. Personality and affective behavior changes (motivation, fear). 8. Changes in sexual behavior. Obviously, contribution of the frontal, parietal, and temporal lobes to all of the above processes is additive and probably hierarchically organized. The fact that similar types of behavior may occur after injuries to different areas supports the claims that the same cognitive processes may be disordered in different ways. These cognitive processes are based on joint activities of large areas of neocortical and subcortical tissues, and may therefore be impaired as a consequence of functional disorders or injuries in any of the involved regions (Kolb & Whishaw, 1996). The occipital lobe (lobus occipitalis) is the area posterior to the parieto– occipital fissure (sulcus). It is the seat of the primary visual area (Brodmann’s area 17). Next to that area are Brodmann’s areas 18 and 19, which are responsible for secondary visual processing. Due to its striped appearance the primary visual area is also known as the striate cortex or the striate area (area striata). The neighboring areas are called the extrastriate visual area (area extrastriata) (Judaš & Kostovi´c, 1997). The occipital lobe is exclusively responsible for visual information processing (Webster, 1995). Information from the left visual field reaches the right hemisphere, and information from the right visual field travels to the left hemisphere, but almost all regions of the visual cortex are reached by the information from the thalamus and from the opposite hemisphere (Lomber & Payne, 2002). The location of an injury will determine which visual field will be blind. Information about different visual attributes is not stored in one place. The cells responsible for visual perception are highly specialized: some are activated only by vertical lines, others by horizontal lines, whereas others are sensitive to color or specific forms, movements of particular speed, and so forth. Each of the cells contributes to the overall mental image. Apart from being highly specialized, the cells are hierarchically determined, so that the first line processes only the simplest visual data, such as contrast, the second line processes shapes, and the following layer interprets them. In this way, each subsequent layer provides additional data to the mental image until the object is recognized (Drubach, 2000). The processing is not exclusively one way, from periphery to the visual cortex (bottom up); there are also feedback projections from the higher order visual areas that contribute to the analysis of the basic properties of the response and structural properties in the primary visual cortex (top down). In other words, processing of any visual stimulus is actually based on interaction among several cortical areas (networks) at different hierarchical levels (Galuske, Schmidt, Goebel, Lomber, & Payne, 2002). One path goes from the striate area through the extrastriate area into the temporal lobe (ventral pathway) and carries information about the properties and the appearance of the stimulus. This is the so-called what pathway, responsible for identification and discrimination of the stimulus. The other path goes into the parietal lobe (dorsal pathway) and carries information about the movement and spatial position of the object. This is the so-called where pathway. Knowledge about the object is located in a distributed cortical
Central Nervous System
21
system, so that the information about particular features is stored in the vicinity of the cortical areas that participate in the perception of these features (Ungerleider, 1995). A person suffering from cortical blindness has normal peripheral vision, but cannot perceive patterns and has no awareness of visual information due to an organic lesion in the visual cortex. Other disorders of visual perception (agnosia) may take the form of impaired ability to recognize colors, faces, objects, depth, movements, and so forth (Drubach, 2000; Kalat, 1995). The insula got its name from the fact that it is separated from the surrounding areas by a circular sulcus. It is located at the bottom of the Sylvian fissure and covered by parts of the parietal, frontal, and temporal lobes. These parts that hide the insula are called opercula (plural of operculum—Latin for lid), and depending on the lobe they belong to, are called the frontal operculum (operculum frontale), the fronto–parietal operculum (operculum frontoparietale), and the temporal operculum (operculum temporale; Judaš & Kostovi´c, 1997). The insula is considered to be an important structure for speech, and there has been some evidence that the insula in the dominant hemisphere, rather than Broca’s area, might be the seat of speech motor planning (Dronkers, 1996, 2000; Duffau, Capelle, Lopes, Faillot, & Bitar, 2000). The anterior portion of the insula is active during processing and integration of autonomic and body information. Its posterior part is connected to other neocortical areas; the connections with the cortical and subcortical structures, particularly with the thalamus and basal ganglia, reveal its importance for somatosensory, vestibular, and motor integration. It is an integrative multimodal association area for information arriving from different senses. The insula plays an important part in the cardiovascular, gastrointestinal, vestibular, olfactory, gustatory, visual, auditory, somatosensory, and motor processes. It seems that it plays a part in conditioned learning, affective and emotional components of nociception (perception of pain), stress-invoked immunosuppression, mood stability, sleep, and language (Flynn, Benson, & Ardila, 1999). The association cortex comprises parts of the cerebral cortex that receive data from several modalities, and thus its role is integrative rather than exclusively motor or sensory, which is crucial for higher mental processes. For example, the association cortex at the border between the parietal, temporal, and occipital lobes of the left hemisphere is essential for successful processing of language data. Although for a long time the prevalent opinion had been that most parts of the neocortex are associative, in the past 15 years it has become increasingly clear that the cortex is mainly sensory and motor, and that complex brains do not develop by expansion of the association cortex but rather through the increase of sensory and motor areas and connections among them (Kaas, as cited in Gazzaniga et al., 2002). The limbic system lies along the corpus callosum (Latin word limbus means borderline). It is related to biological rhythm, sexual behavior, feelings of fear, anger, and motivation (Figure 1.6). In other words, the limbic system controls and processes emotions, and manages endocrine and autonomic systems. Amygdala are particularly important for the regulation of drives, affective and motivational states, and autonomic and endocrine functions (Judaš & Kostovi´c, 1997; Sternberg,
22
The Cognitive Neuroscience of Human Communication
Corpus callosum Hypothalamic nuclei
Cingulate gyrus
Anterior thalamic nucleus
Fornix Olfactory bulb
Mammilary body
Olfactory tract Amygdaloid body
Parahippocampal gyrus Hippocampus
FIGURE 1.6.
The limbic system.
2003). During evolution some parts of the limbic system (e.g., the hippocampus) have assumed other functions as well. It is believed that, in higher animals, the hippocampus is one of the key structures in learning and memory. The hippocampus may host different types of information at the same time and one of its major roles is integrating different details or elements of episodic memory traces and coding current experience, to be subsequently stored in memory (Hampson & Deadwyler, 2002; Payne, Jacobs, Hardt, Lopez, & Nadel, 2002). More on the hippocampus may be found in chapter 7. Within the limbic system the hypothalamus is the key passage for different neuronal circuits (Judaš & Kostovi´c, 1997). The cingulate gyrus (gyrus cinguli) is located immediately above the corpus callosum. Physical and mental emotional expressions are integrated in the hypothalamus and in the insula, which explains the fact that each of these two types of emotions may have manifestations characteristic of the other (Drubach, 2000). Basal ganglia are located deep within the cerebrum (Figure 1.7). This structure is actually a group of nuclei (putamen, globus pallidus, nucleus caudatus, substantia nigra, subthalamus) made of gray matter. They interact with the cerebral cortex, thalamus, reticular formation and parts of the midbrain and spinal cord, and are important for motor functions (primarily voluntary and many involuntary ones), including speech (Webster, 1995). Due to their connections with the association areas of the cerebrum they have a direct influence on the affective, language, and other cognitive processes (Judaš & Kostovi´c, 1997; Lieberman, 1991). Their injury will result in weak and uncoordinated movements (Kalat, 1995), in several types of involuntary movements, such as jerks, tremor, and so forth (Judaš & Kostovi´c, 1997), and in cognitive disorders as well. Parkinson’s disease is the most frequent and the most extensively studied neurological disorder related to the basal ganglia.
Central Nervous System
23
Longitudinal fissure Ventral lateral thalamic nucleus
White matter
Cortex (gray matter)
Corpus callosum
Internal capsule Putamen Lateral fissure
Basal ganglia
Caudate nucleus
Globus pallidus
Fornix
Amygdala
Substantia nigra
Subthalamic nucleus
Mammillary body
FIGURE 1.7. Coronal (frontal) view of the brain with the basal ganglia.
Although the cerebellum and basal ganglia are important for planning, initiation, and performance of movements, their roles are different. The cerebellum creates the movement by trial-and-error learning and the final movement is optimal for the particular situation and the set goal. Basal ganglia release the movement from general tonic restraint, allow nonrival positions and movements to develop and prevent rival activity (Thach, Mink, Goodkin, & Keating, 2000). Warren, Smith, Denson, and Waddy (2000) have shown the importance of basal ganglia in speech planning, word recall, and short-term verbal memory. Their 53-year-old bilingual (English–German) patient, who had suffered a stroke in the left posterior nucleus lentiformis (a part of the subthalamus, i.e., basal ganglia), presented with apraxia of writing and speech in both languages. Detailed language tests revealed disorders in articulation, fluency, repetition of auditory stimuli, interpretation of complex semantic relations, definition forming, and short-term verbal memory. The connections between the cortex and subcortical areas are very important for the functioning of the brain, because the injuries in those pathways may cause behavioral disorders that are identical to those caused by injuries in particular functional areas (Kolb & Whishaw, 1996).
24
The Cognitive Neuroscience of Human Communication
The brain’s major foodstuffs are glucose and oxygen. Although it is only about 2% of the total body mass, it is responsible for about 20% of total oxygen and about 50% of total glucose consumption (Strange, 1995).
SENSATION AND PERCEPTION The prerequisites for the occurrence of sensation are (a) the existence of a stimulus; (b) the processes that convert the stimulus into bioelectrical signals suitable for neuronal transmission; and (c) the specific response of the organism to the coded message. All sensory systems are organized hierarchically, in a parallel manner, and topographically (Judaš & Kostovi´c, 1997). The first intermediary between the outside world and the body is the sensory receptor. It converts the received input energy, which may be mechanical, thermal, chemical or electromagnetic, into electrochemical energy, which fulfills one of the most important requirements of information transfer to the central nervous system (in the form of action potentials). Besides the general functioning principles common to all senses, each sensory modality has its peculiarities. For example, in the visual modality, different receptors are responsible for the blackand-white data/contrast (rod cells of the retina) from those that are responsible for colored input (cone cells). Within a single modality there may exist different receptors for specific stimulus features (different segments of a visual stimulus have been discussed already). In addition to being modality-specific (visual, auditory, gustatory, olfactory, somatosensory), all receptors may transmit the sense of pain in cases of excessive stimulation. The sensory transduction of the input stimulus into electrochemical energy is followed by neural coding of the stimulus in the primary sensory neurons that are in direct contact with the receptors. Neural coding includes information about stimulus modality, intensity, duration, and location. With the exception of olfaction, information from all sensory organs passes through the thalamus on its way to the cerebrum. The thalamus selects the bits of information to be passed on and directs them to the appropriate location in the cortex. The selected information is transmitted to the primary (modality-specific) areas in both hemispheres. Approximately at the level of the thalamus and the primary cortical areas information is grouped into wholes (gestalts). This process ensures the distinction of form from the background, in other words, drawing relevant information from all available stimulus characteristics. Information is then transferred from the primary to the secondary areas, where the wholes (i.e., their meaning) are recognized. Secondary cortical areas are also modality-specific. The term perception is used when there is sensory awareness, and that awareness is typically formed at the secondary cortical level. In other words, perception is conscious reception, adoption, and interpretation of external stimuli (exteroception) or those from one’s own body (interoception, proprioception). One of the prerequisites of perception is the existence of intracortical phenomena. Conscious sensation may be elicited only if stimulation lasts at least several hundreds of milliseconds (as indicated by electrical stimulation of the sensory cortex in
Central Nervous System
25
awake patients). Perception is the keenest in the areas of sharpest contrast (Judaš & Kostovi´c, 1997). Complex information is processed in the tertiary areas, so that the information arriving from the secondary areas is integrated and combined with affective data. In other words, this is where new information is interpreted in the light of previous experience and existing knowledge. That is part of the reason why these areas are also called association areas: a familiar odor may bring to life the image and the sound associated with it, because at some point in our life the stimuli from all three modalities were present at the same time. Pleasant and unpleasant emotions may also be associated with the event. Cerebral (mental) representation of an object, person, or anything else, for that matter, includes the corresponding physical properties, but also the unique affective and experiential components. Although it is impossible to draw a clear line between sensation and perception, it is generally accepted that sensation is subcortical whereas perception is cortical (Pinel, 2005). To put it more precisely, sensation is the outcome of the activity of receptors and their afferent pathways to appropriate sensory areas in the cortex, and perception is the result of the activity of the cerebral cells after the first synapse in the sensory cortex (Kolb & Whishaw, 1996). Attention plays a very important role in stimulus perception. In speech, it may be crucial for message comprehension. Some studies have shown that the subjects were able to extract very little information from the message presented to one ear while they were focusing on the other. Many of them had not even noticed that the stimuli presented to the unattended ear were in a foreign language or that it was speech reproduced backwards. Accordingly, very little of the information presented to the unattended ear was stored in memory. Contrary to that, if the other message is physically different or has been changed it will be easier to detect: for example, a male voice presented to one ear and a female voice to the other (Eysenck & Keane, 2000). Escera, Alho, Schröger, and Winkler (2000) found, using event-related potentials (ERP), that very small changes in the properties of an auditory stimulus activate the auditory cortex (temporal lobe) in both hemispheres, even in situations where the stimuli were not attended to. There is an ongoing controversy about the minimal perceptual unit of speech. There are equally abundant arguments for it to be the sound, the phoneme, the syllable, or the speech measure. The authors, who claim that the smallest unit is the sound or the phoneme, nonetheless believe that the decisions pertaining to sounds and phonemes are made in a wider context—at least to the syllable level. The criterion of processing efficiency does not help resolve the problem either. Namely, smaller units (e.g. phonemes) are less taxing on the memory, but processing is considerably slower than where larger units are involved. On the other hand, larger units, due to their great number of possible combinations, are more demanding on the memory. In any case, it is clear that prelexical processing implies some sort of transformation of acoustic–phonetic information into abstract representations. Since the main topic of this book is speech, it seems appropriate to shed some additional light on hearing as the primary path of its perception and on movement as the primary means of its production.
26
The Cognitive Neuroscience of Human Communication
NEURAL BASES OF SPEECH PERCEPTION AND PRODUCTION HEARING, LISTENING AND THE AUDITORY CORTEX In humans, the first reaction to sound, manifested as startle reflex, occurs approximately 16 weeks before birth. Such a reaction, albeit very primitive, already points to the active relationship between the auditory stimulus and motor reaction to it. More sophisticated reactions, such as orienting reflexes, take longer to develop. The auditory system is ready to receive sound and initiate action potentials at birth. It has been shown that electrical stimulation of the auditory pathway may cause impulses in the auditory cortex even before the final maturation of the synapse between receptor cells and the auditory nerve (Aitkin, 1990). Studies of congenitally deaf cats have revealed that electrical stimulation of the cochlea may result in normal spatial and temporal patterns in their auditory cortex (Klinke, Hartmann, Heid, & Kral, 2002). Keeping in mind the importance of the spectral pattern of auditory stimuli in determining the firing patterns of mature auditory cortical neurons, it is possible that temporal characteristics of surrounding sounds are important factors in synaptogenesis that occurs in the auditory cortex. The auditory pathway is the path from the peripheral organ (i.e., the ear), to the auditory cortex, structured as a very complex network of connections. Stimuli from each ear are transmitted ipsilaterally and contralaterally along several lines. Approximately one third of the information reaches the same side, and two thirds the opposite. The first neuron of the auditory pathway is located in the inner ear, specifically in the spiral ganglion of the cochlea, and it is in direct contact with the receptors in the organ of Corti. The cochlear nerve (nervus cochlearis) is a branch of the vestibulocochlear nerve (nervus vestibulocochlearis—the VIII cranial nerve, which is commonly referred to as the auditory nerve), and consists of some 30,000 to 40,000 axons. The vestibular branch (nervus vestibularis), which is responsible for balance, consists of some 8,000 to 10,000 axons (Judaš & Kostovi´c, 1997). Axons of the auditory nerve synapse in the ipsilateral cochlear nucleus (nucleus cochlearis). In addition to transmitting the tonotopic representation of the cochlea, considerable processing of the auditory signal takes place there. At this level there are at least six different neural responses, resulting from complex interactions and neural processing (Seikel, King, & Drumright, 1997). A number of projections from cochlear nuclei reach the superior olivary nuclei (nucleus olivaris superior) at the same level. Some go to the ipsilateral superior olivary nucleus, others to the contralateral side. Neurons in olivary nuclei have two large dendrites each: the right one receives information from the right ear and the right cochlear nucleus, the left one receives information from the left ear and the left cochlear nucleus. These dendrites can detect time differences in the activation from the left and right ear that are as short as several microseconds. This ability is partly responsible for sound localization (Gazzaniga et al., 2002; Thompson, 1993). Superior olivary nuclei are the first location where interaural differences in auditory signal intensity (particularly for high frequencies) and phase (particularly for low frequencies) differences are detected. The differences reflect different stimulation of the left and the right ear by the auditory stimulus
Central Nervous System
27
due to its position in space relative to the listener. In other words, there are neurons that react to specific stimuli features (the so-called feature detectors) and transmit the recognized and analyzed signal features to the cerebral cortex. In this way the complex sound signals are analyzed into their components. Inferior colliculi (colliculus inferior) receive bilateral signals from superior olives and indirectly from cochlear nuclei via the lateral lemniscus (lemniscus lateralis). Several responses occur here as well; it seems that at this level the intensity-difference data are combined with the phase-difference data again. Each inferior colliculus is an integration nucleus of the central auditory pathway, where about a dozen projections from the lower auditory nuclei in the brainstem, from the contralateral inferior colliculus, and from the auditory cortex come together (Syka, Popelar, Nwabueze-Ogbo, Kvasnak, & Suta, 2002). The medial geniculate body (corpus geniculatum mediale) is the last synapse in the diencephalon (in the thalamus). Here, the tonotopic organization is manifested as the projection of: (a) the ventral part of the medial geniculate ipsilaterally into the primary auditory cortex in the temporal lobe (Brodmann’s area 41); (b) the medial part into other regions of the temporal lobe (Brodmann’s areas 42 and 43); and (c) the dorsal part into the so-called cortical association areas. The diversity of neural responses has been preserved at this level as well. Some neurons are sensitive to very small intensity differences between signals reaching the two ears (Seikel, King, & Drumright, 1997). In summary, the cochlea performs the first analysis of complex sound stimuli into their components. Cochlear nuclei, superior olives, and lateral lemniscus nuclei code different aspects of the stimulus (frequency, intensity, timing) and transmit the processed information via six parallel paths into the medial nuclei of inferior colliculi. At the lateral lemniscus level, the left and the right auditory pathways are connected by Probst commissure. This commissure also has crossed projection fibers by means of which the connection with the contralateral inferior colliculus is established. All the information is integrated and synthesized here before being forwarded to the medial geniculate body in the thalamus and subsequently to the primary auditory cortex where the different aspects of the stimulus are to be recognized. The left and the right auditory pathways are connected (in addition to the level of lateral lemniscus) by the inferior colliculi commissure and by corpus callosum at the cortical level. Tonotopic organization is preserved in all auditory nuclei and in auditory cortices of both hemispheres. Cochlear representation is preserved through the auditory pathway, but it multiplies at the central levels. This redundancy is the key to our ability to analyze and understand very complex sounds. Obviously, the auditory pathway is a network of separate but intertwined neural mechanisms that are adapted for the transmission of highly specific, detailed information from periphery to the brain. Cells in the subcortical nuclei react to the wider frequency bands than the cells that are higher in the system. Other sensory systems have parallel ascending pathways, but in the auditory pathway there are more parallel paths than in the other modalities, there are more synapses in the brainstem nuclei, and, finally, after converging, they again go their separate ways into different parts of the auditory cortex (Webster, 1995). Giraud et al. (2000) have found, using the fMRI, that the human auditory
28
The Cognitive Neuroscience of Human Communication
system is organized as a hierarchical filter bank with parallel processing, so that each processing level has its preferred stimulus frequency but is able to respond to all others. Such structure makes possible separate transmission of groups of frequencies containing different bits of information, but at the same time preserves the possibility of integration of complementary features in the auditory cortex. Besides the dorsal surface of the superior temporal gyrus and the small transverse temporal gyri of Heschl, the majority of the human primary auditory cortex (Brodmann’s areas 41 and 42) and the neighboring areas of the secondary auditory cortex (Brodmann’s area 22) are located deep within the Sylvian (lateral) fissure. There are variations in the location and the size of cortical areas. This is especially true of the primary auditory area, mostly due to the great variability in the Heschl’s gyrus (Schönwiesner, Von Cramon, & Rübsamen, 2002). The primary auditory cortex receives input primarily from the opposite ear via the ipsilateral medial geniculate body. At that level the tonotopic representation is realized as columnar organization such that each column is made up of neurons that react to similar frequencies—much like the organization of hair cells in the cochlea. However, Schönwiesner et al. (2002) question the strict tonotopic organization, based on their fMRI studies of humans and other primates in which they found several locations characteristic for each frequency. Takahashi et al. (2002) reported that tonotopic organization is affected by the stimulus intensity: stimuli of higher intensity are less clearly tonotopically organized. Different neurons in the columns react to different elements of the stimulus (e.g., upward or downward frequency or intensity modulation). This is analogous to the already mentioned columnar organization of the primary visual cortex, with the neurons in different columns sensitive to different components of the visual stimulus such as movement, color, slant, and so forth (Scheich, Ohl, & Brechmann, 2002; Seikel et al., 1997). Neuroimaging techniques have revealed that different structures are involved in the processing of sound properties (the ventral “what” path) and location (the dorsal “where” path). In that respect, the auditory and the visual systems are comparable as well (Anourova et al., 2002; Kusmierek, Laszcz, Sadowska, & Kowalska, 2002). Although both pathways are present in both hemispheres, even a unilateral brain injury may cause difficulties in sound recognition and localization (Clarke et al., 2002). With respect to the perception of tactile information, however, Van Boven, Ingeholm, Beauchamp, Bikle, and Ungerleider (2005) found no dorsal/ventral dissociation. In their fMRI study of tactile form and localization processing, they found a hemispheric lateralization effect manifested as selective left intraparietal sulcus activation in the form detection task, and selective activation of the right temporoparietal junction in the location discrimination task, independent of the stimulated hand. By means of fMRI Binder et al. (2000) have found that parts of the auditory cortex in which spectral information is represented are separated from those where the intensity sensitivity is represented. Both are located on the dorso-lateral surface of the temporal lobe in both hemispheres, and although they involve neighboring regions, there is no overlap. Sound intensity is represented in the caudomedial part of Heschl’s gyri and the neighboring parts of the transverse
Central Nervous System
29
temporal sulcus, whereas spectral information is represented in the rostrolateral part of Heschl’s gyri and the neighboring parts of the transverse temporal sulcus. Due to its location deep within the Sylvian fissure, the auditory cortex is well protected from injury and even the most severe bilateral injuries to the temporal cortex do not result in deafness. Patients with damaged auditory cortex perceive simple auditory stimuli without problems, but they have difficulty in identifying very short stimuli, in discriminating sounds presented in short intervals and in judging the temporal sequence of sounds presented in short intervals. Consequently, patients with extensive auditory cortex damage often complain of difficulties in perceiving fast speech (Pinel, 2005). Damage to the auditory cortex in the left hemisphere, dominant for speech in most people, usually has more serious consequences. Although the auditory cortex is mostly located in the temporal lobe, several areas of the secondary auditory cortex (each of them a tonotopically organized map) extend into the parietal lobe. It has been shown that patients with extensive auditory cortex damage that extends into the parietal lobe have trouble locating the sound source, especially if the injury is in the right hemisphere. The right hemisphere was found to be crucial in the discrimination of the direction of frequency modulation—upward or downward (Scheich et al., 2002). Despite the fact that speech information has been processed to a great extent before reaching the auditory cortex (a large portion of information necessary to decode speech is processed in subcortical auditory nuclei), many aspects of speech (e.g., speaker variability, coarticulation) are attributed to perception that takes place in the cortex. Durif, Jouffrais, and Rouiller (2002) found that some cortical neurons process both physical properties of sounds and their relevance for behavior. The auditory cortex is not just a final point of bottom-up analysis, but also an important part of the top-down system, that is actively directed toward particular aspects of the sound stimulus, depending on the type of information that needs to be extracted from the input pattern (Scheich et al., 2002). Although there is no clear distinction between auditory projections and auditory processing on the one hand, and areas where speech and language are decoded, on the other, it is believed that conscious processing of speech takes place at the cortical level. Even very young (1-month-old) infants are able to distinguish sound categories, for example, /p/ versus /b/. Electrophysiological studies have shown that 3month-old infants have neuronal networks for phonological processing similar to adults (Dehaene-Lambertz & Baillet, 1998). These results used to be taken as evidence of the inborn speech ability to categorize phonemes perceptually. However, it has been found that children are just as successful in distinguishing among nonspeech sounds, and that the ability is present in other mammals as well. Apart from being able to categorize the sounds of their mother tongue, in their first six months of life children learn to normalize across speakers, sound contexts, and prosodic features. Cross-linguistic studies of children have revealed selective sensitivity to their mother’s voice and the sounds of the mother tongue, suggesting the influence of prenatal auditory experience. The recognition of the mother tongue and its discrimination from other spoken languages within the first
30
The Cognitive Neuroscience of Human Communication
six months relies primarily on prosodic features, only to be followed by the development of sensitivity to segmental features. This is attributed to the fact that, in infant-directed speech, the mothers (or other caregivers) talk more slowly, using higher pitch and more pronounced intonation variations. These suprasegmental features enable babies to distinguish comforting (long, smooth, falling intonation), from warning and disapproval (short, sharp intonation patterns), or calling and drawing attention (rising intonation; MacNeilage, 1997).
MOVEMENT AND SPEECH PRODUCTION Movement is a readily visible product of cerebral activity directed at acting on the world around us. Movement initiation and coordination are controlled by the brain. The entire course of the movement, from its planning to execution, is hierarchically organized. In this respect it resembles the reception and interpretation of stimuli (in reverse order). The highest wrung on the hierarchical ladder is occupied by groups of neurons whose main purpose is to plan movements. These neurons are located in the tertiary (premotor, association) cortical areas. Actions are planned on the basis of available perceptual data, experience and goals, with the help of the cerebellum and basal ganglia. Tertiary areas are responsible for very complex movements (e.g., speech) and they control primary and secondary motor areas. Secondary areas control and coordinate several muscle groups. The commands are relayed to lower levels until they reach the neurons in the primary motor cortex (Brodmann’s area 4) that will activate individual muscles or small muscle groups in order to fulfill the set goal—executing the desired movement. The execution is controlled by motor neurons in the brainstem and the spinal cord by means of commands that reach effector organs, that is, muscles. Simple movements require minimal processing and are limited mostly to the primary motor and sensory areas. Increasing complexity results in increased activated areas, in such a way that the activation spreads to the regions anterior to the primary motor area in both hemispheres. This bilateral activation can be explained by the activation of some abstract motor plan that is yet to be perfected for a specific activity, or it may reflect activation of several alternative motor plans with the same goal. The one finally selected will result in increased lateralized activity of the motor cortex, related to the activity of the contralateral effector organ, for instance, the hand. This selection process involves the supplementary motor area (Brodmann’s area 6) as well. It is particularly important for hand coordination in complex motor tasks. Obviously, the entire process has to be parallel and interactive at the same time. Feedback is present at all levels. Based on the sensory data the execution programs are modified with the purpose of fulfilling the task optimally and efficiently—yielding maximum results with minimal effort. In the process, the thalamus plays a very important role: it relays information from the sensory areas to the corresponding motor areas. However, the movement does not depend exclusively on feedback involving peripheral data. There are internal representations of movement patterns as well. Even profoundly hearing-impaired patients, who have
Central Nervous System
31
lost their hearing postlingually, can speak well despite the loss of auditory control. Obviously, kinesthetic feedback is at play here, but it would not be sufficient were it not for fixed movements the patient can rely on. The longer the period without the feedback, the less reliable the patterns will be. As it has already been mentioned above, the cerebellum and basal ganglia also play very important parts in planning, coordination, and execution of movements. It is believed that the cerebellum is particularly important for the functioning of the speech rhythm generator. This means that it is the seat of the neural center for syllable timing that is independent of afferent (sensory and proprioceptive) information or that it actually controls the center (Horga, 1996). Traditionally, it was believed that the motor cortex was the only starting point of the pyramidal tract (that was thought to be the highest level of motor control). It seems, however, that only about 60% of pyramidal tract motor neurons originate in the motor cortex, whereas the rest of them originate in Brodmann’s area 6 and in the parietal lobe. It also seems that the motor cortex is not the place where the movement originates, but rather it is a conduit for information about movement arriving from other cortical and subcortical areas. Studies have shown that the pyramidal tract comprises numerous fibers that originate in the cortex and that influence the sensory transmission at the subcortical level and at the spinal cord level (for a review see Kent & Tjaden, 1997). In short, the studies of humans and other primates indicate that motor planning involves parallel circuits. One of them—including the parietal lobe, lateral premotor pathways, and the cerebellum—is important for movements in space. The other circuit, that includes the supplementary motor area, basal ganglia and, possibly, the temporal lobe, is activated once the skill has been mastered (Gazzaniga et al., 2002). The type of functional disorder depends, naturally, on the location and extent of lesion. Lower-level injuries will impair the execution of specific movements of particular muscles or muscle groups. Higher-level injuries will impair planning and coordination, entirely or to a great extent, which will result in impaired execution of all but the simplest movements. In the case of speech production, which requires a very high degree of planning and coordination of numerous muscles, this means that an injury to the tertiary area will impair speech production, despite the preserved ability to move individual muscles involved in speech. Contrary to that, after damage to the area responsible for the movement of one of the articulators, speech may be difficult but not necessarily impossible. For example, damage to Brodmann’s areas 6 and 44 will result in speech apraxia that is manifested as nonfluent speech and uncoordinated complex speech movements; damage to Brodmann’s area 7 will cause conduction aphasia, manifested as phonological paraphasias, omissions, and transpositions, but without affecting fluency. Stuttering is explained by some authors as a consequence of inappropriate lateralization and/or uncoordinated functioning of both hemispheres. This may be realized as simultaneous and uncoordinated bilateral control or as uncoordinated callosal communication between the two hemispheres. Both explanations are based on the right hemisphere involvement in speech production to the
32
The Cognitive Neuroscience of Human Communication
extent that hinders efficient processing and control by the left hemisphere (generally dominant for speech). In other words, stuttering is a consequence of intraand interhemispheric interference between functionally related areas of cerebral cortex (Webster, as cited in Horga, 1996). Let’s say a few more words about movement in speech. All speech production models assume three phases, of which the second and the third are basically simultaneous: (a) organization of speech motor programs; (b) realization of these programs by converting the commands to appropriate muscles into a series of articulatory movements; and (c) conversion of articulatory movements into actual speech sound. The articulation program is a part of the speech motor program, not its synonym (Horga, 1996). The distinction between the articulation and speech program is clearer if we agree that motor program does not include predefined commands to the articulators, but that it is conditioned by context and the set goal. In harmony with the goal and the continuous feedback (auditory, tactile, proprioceptive) it adjusts actual efferent stimuli (i.e., execution of the articulation program) to the current context (Horga, 1998, 2002a). It is believed that some common mechanisms are at work in movement in general, and specifically in speech movement. There is evidence that regions around Broca’s area are not only involved in series of speech movements, but also in series of movements in general. There are numerous examples of brain injuries that result in parallel impairment of speech and general motor functioning. Saygin, Wilson, Dronkers, and Bates (2004) have found that not just speech production but speech comprehension as well may be impaired together with comprehension of action (pantomime interpretation). They concluded that brain areas important for the production of language and action are also active in their comprehension. Moreover, they suggest that brain is organized for action processing. This is not to be confused, however, with the composition of the muscles themselves—Kent (2004) reviewed a large body of literature and concluded that speech muscles (mandibular, lingual, palatal and laryngeal) are unique in their fiber composition, distinguishing them from other muscles in the human body. Hauk and Pulvermüller (2004) also studied the correlation between action words and specific cortical areas, and similarly to Saygin et al., suggest a possible role of mirror neurons in the premotor cortex in language processing. Another argument in favor of this view is the parallel development of hierarchical organization of speech (language) and movement. As opposed to automatic motor behavior patterns controlled by subcortical structures and reflexes, speech movements are controlled by higher levels. This may be described as a central neural feedback mechanism that is realized in the cerebral cortex due to the proximity of motor and sensory association areas; the effects of motor actions are recorded as patterns of activity in the association cortex, from whence they in turn control the muscles. Although traditionally, swallowing has been considered one of the automatic motor behavior patterns under subcortical control, there is increasing evidence that the cortex (particularly the cingulate cortex, the insula, and the inferior frontal gyrus) and the cerebellum are involved as well (Daniels, Corey, Fraychinaud, Depolo, & Foundas, 2006;
Central Nervous System
33
Stickler, Gilmore, Resenbek, & Donovan, 2003; Suzuki et al., 2003; Tasko, Kent, & Westbury, 2002; Watanabe, Abe, Ishikawa, Yamada, & Yamane, 2004). According to some authors, the center for production of speech patterns is located in the brainstem. This opinion is based on studies that have revealed mechanisms, the so-called pacemakers that keep the rhythm necessary for cyclic life-sustaining functions—for breathing, movement, and chewing—with the assumption that speech, as a higher complex function, has developed from these lower functions. There is, however, evidence to the contrary as well: patterns of muscle activity recorded during speech have been found to be different from those recorded during chewing or some other nonspeech tasks. Gracco (cited in Kent & Tjaden, 1997) suggested a compromise: lower rhythmic functions and speech differ in motor patterns, but share mechanisms for their implementation and principles of organization. For a critical review of related literature see Kent and Tjaden (1997). Just as in perception, there has been no consensus on the minimal units of the speech motor program and its execution, in other words, the articulation motor program. Distinctive features, individual sounds or phonemes, syllables, words and word groups or short sentences have been discussed as the likely candidates. Studies of both types of programs (speech and articulation alike) converge on the value of approximately seven syllables. This is the average capacity of working memory (the famous 7±2 number) and may be related to the motor speech program, but it is also the average syntagmatic length in speech, which makes it a plausible articulation unit (for review see Horga, 1996). In view of the efficiency and rate of speech, in spite of the enormous number of muscle activations (several hundreds in one second), some preset motor subroutines probably exist. On the basis of the degree of variance among speech segments Horga (1996) opts for phonetic word as this subroutine size. Besides dealing with the size of the minimal units of speech and articulation motor programs, most speech production models address the question of the relationship between execution—for example, the movement itself—and goal attainment—for example, speech. Here is a very simplified short list of examples: 1. Speech is a series of relatively constant context-independent articulatory gestures (Haskins Laboratories’ authors). 2. Movements of the articulators are determined by their relations to all other elements of the system (Chistovich and associates). 3. The physiological model oriented toward the mechanics of speech movements and its acoustic supplements (Peterson; Shoup). 4. Binary functional models that assume phonetic and phonological levels, based on distinctive features founded on the acoustic and articulation characteristics of speech (Jacobson; Fant; Halle; Chomsky). 5. Sound systems of natural languages are harmonized with human perceptual and articulation abilities, so that the sounds that are articulated in natural areas have particular quantal characteristics, in other words,
34
The Cognitive Neuroscience of Human Communication
there are areas where small articulatory changes result in considerable acoustic changes, and areas where large articulatory changes result in negligible acoustic changes (Stevens); distribution of vowels in acoustic space is such that it provides maximum perceptual contrast, regardless of the number of vowels in the system (Liljencrants; Lindblom). 6. Model of speech production based on a closed feedback involving the respiratory, phonatory and articulatory levels on the one hand, and the afferent sensory and proprioceptive feedback, on the other, with the appropriate comparator that continuously coordinates afferent stimuli with the final goal (Fairbanks). 7. The parametrical approach, according to which speech is a unique and complex system with continuous interaction between different levels, oriented toward neural mechanisms of speech production (Liberman). 8. According to several dynamic models that are based on the target theory, and that rely on neurophysiologic foundations of speech, all movements are oriented toward the set goal, without assuming fixed, preset articulatory movements (MacNeilage; Noteboom; Kelso; Saltzman; Tuller). Horga (1996), Kent and Tjaden (1997), Obler and Gjerlow (1999), Eysenck and Keane (2000), and Gazzaniga et al. (2002), among others, have reviewed and discussed such models. The language network, albeit physically close to the general sensorimotor one, is a separate system. Beatse, Sunaert, Wilm, Van Hecke, and Marchal (2000) have revealed by fMRI that the language network involves the superior temporal gyrus in both hemispheres, the left angular gyrus, the left middle and inferior frontal gyrus, and the cingulate gyrus in both hemispheres. Within the network, the receptive language regions are separate from the expressive ones. The hand, however, is represented in the sensory and motor cortex located in the precentral and postcentral gyrus, respectively.
RELATIONSHIP BETWEEN SPEECH PRODUCTION AND PERCEPTION The fundamental issue in discussing the neurolinguistic aspects of the relation between speech perception and production is the question of their cerebral representation. In other words, do speech production and perception share a common cerebral representation or are they two neurofunctionally different areas with stronger or weaker connections? Neurolinguistic literature most frequently deals with descriptions of the production (expressive) or the perception side of speech, and their relationship is commonly described by claims that production is more clearly lateralized to the language-dominant hemisphere (usually left) whereas perception is more widely distributed with greater involvement of the nondominant (usually right) hemisphere. Cortical stimulation studies have revealed that the mouth and face muscles are represented at the points that are responsible for phoneme identification. This supports the motor theory of speech perception—one of the best known and most
Central Nervous System
35
extensively discussed theories of speech perception (Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967) according to which speech is perceived by means of movements necessary to produce the heard sounds, not by means of the sounds themselves. (The theory is discussed in more detail in chapter 5, this volume.) Penfield and Roberts (as cited in Bradshaw & Nettleton, 1983) have named that region of the brain the voice control area. In other words, speech and language comprehension are parts of a single system. In their studies of chimpanzees and speech acquisition in children Premack and Premack (1994) have found that comprehension and production start off as separate systems, but become united on the basis of even very limited experience. Albeit limited, that experience is a necessary prerequisite to initiate the merging of the two systems. Hurford (1994a, 1994b), on the other hand, believes that there is no period in early childhood during which a child might know a new word or structure without being able to use it. Ojemann (1994) also concluded that language processing includes motor mechanisms even without actual production. However, the relationship is not as straightforward as it might seem. Recordings of neural activity have shown that different cells are responsible for speech perception and production, even when the word to be perceived and produced is the same (Creutzfeldt, Ojemann, & Lettich, 1989a, 1989b). The auditory speech perception system shuts down briefly during speech production. Ojemann (1994) believes that the proximity of the neurons responsible for perception and those involved in production may explain the stimulation results. They become parts of the network that has properties not inherent to individual cells. The major brain areas involved in production and comprehension of speech and language are shown in Figure 1.8. Cortical representation of speech and language is discussed in more detail in chapter 8. Jäncke and Shah (2000) and Specht, Shah, and Jäncke (2000b) also found in their fMRI studies that language production activates parts of the cortex involved in perception and vice versa; language recognition tasks activate classical speech production areas as well. Kent and Tjaden (1997) discussed possible points of contact and common representations. They may be summarized as three types of connections underlying the integration of speech production and perception: (a) neighboring location of cerebral and cerebellar motor and sensory neurons; (b) existence of multimodal neurons, or nerve cells that react to more than one sensory modality; and (c) neural structure that includes parallel and recurrent pathways. Fowler and Galantucci (2005) also provide some evidence of close correlation between mechanisms supporting action and those supporting its perception. Neighboring Location of Motor and Sensory Neurons A case in point is the location of motor and somatosensory cortex, in other words, cortical regions responsible for efferent (motor—motor neurons) and afferent (somatosensory—sensory neurons) activities of the body. Although the central sulcus separates the motor (precentral, anterior to the sulcus) from the somatosensory (postcentral, posterior to the sulcus) cortex (see Figure 1.8), some motor
36
The Cognitive Neuroscience of Human Communication
4 6
8
5
3
7
1 2
19
40 9
46
10
41
43
18
44
45
52
47 11
39
22
17
42 37
21
38
20
4
6
31 2
5
7
8 24
9
31 23
32 33
10
30
26
18
27 29
11
17
25 38
19
34 35 28
36
37
19
18
20
FIGURE 1.8. Major brain areas involved in the production and perception of speech and language.
neurons can be found immediately next to the sensory ones. In addition to the already mentioned cortical representations of mouth and face muscles involved in speech that share their location with the points responsible for phoneme identification (Ojemann, 1983), motor and sensory neurons are found in the vicinity of each other in the cerebellum as well. Such results, in combination with clinical observations, suggest that among the important roles of the cerebellum are temporal calculations that are used in many perceptual and motor functions, including speech. Multimodal Neurons These neurons respond to two or more types of stimuli. They can be found in different animal species: in macaques about 55% are bi- or multimodal (visual and
Central Nervous System
37
auditory, visual and somesthetic or visual, auditory and somesthetic); in songbirds the pathways that control song also respond to auditory stimuli. Separate sensory pathways allow stimulus distinction with respect to modality, and multisensory pathways make possible integration of stimuli from different modalities (Rosenblum, 2005). This is particularly important for speech because of its multimodal properties, and it has proven to be invaluable in speech and hearing rehabilitation (Guberina, 1992–1993; Pansini, 1995, 2002). Parallel and Recurrent Pathways Parallel pathways enable comparisons among various neural representations. In songbirds this explains their ability to learn their species-specific song—by continuously comparing their own song with that of adult birds. These comparisons are possible due to multiple neural mechanisms, modalities, multiple loops and representations of song patterns. They are based on two neural branches that mature at different times. One develops gradually during song acquisition and it is essential for song production in adult birds. The other reaches full maturity very early, quickly reacts to stimuli, and its purpose is quick and recurrent comparison of the bird’s own (motor) activity with the model (control, sensory activity). According to Williams (1989), when the species-specific song is acquired, this branch activity decreases (as cited in Kent & Tjaden, 1997). Despite the lack of comparable direct physiological data for humans, Kent and Tjaden (1997) propose the following analogy to speech: (a) different parts of the neural structure mature functionally at different ages, which explains the existence of the sensitive (critical) periods in speech acquisition; (b) different parts of the neural structure have different representations of speech sounds, such as auditory, tactile, motor, or kinesthetic; (c) there are branches that are used for recurrent comparisons of the different elements of sound representations, and as speech acquisition progresses, humans rely on their activity less and less.
2 Sex Differences
There are several aspects of differences between men and women with respect to brain structure, lateralization, and behavior. Before we go on discussing them, it should be stressed that the differences among individuals of the same sex are greater than the differences between sexes.
STRUCTURAL DIFFERENCES There is no evidence that neural connections or brain architecture are any different in men than in women, although at all ages the average brain mass of men is approximately 10% greater than in women. That difference can probably be attributed to the difference in neuron size rather than number, and it does not imply differences in performance (Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Differences in mass between the two hemispheres (right hemisphere has greater mass, whereas the left has greater density) are smaller in women than in men. Thicker cortex of the right hemisphere as compared to the left has been found in male, but not in female, fetuses. The shape of the left lateral (Sylvian) fissure corresponds to the right in more women than men. In women, the left planum temporale is not as frequently larger than the right as it has been found in men. Relative brain size is different at 2 years of age, and in the hypothalamus, sex differences occur around the fourth year of life (Swaab & Hofman, 1995). Verchinski et al. (2000) conducted an MRI study on 111 healthy subjects ranging in age between 17 and 54. They reported significantly greater gray matter density in most neocortical regions in women. The greatest sex differences were found in the left middle temporal gyrus (Brodmann’s area 22), in the right parietal lobulus, and in the left precuneus. None of the areas in male subjects had greater gray matter density. Although they reported no statistically significant differences, Amunts, Schleicher, Ditterich, and Zilles (2000) found a trend toward greater asymmetry in cytoarchitecture of Brodmann’s areas 44 and 45 (that correspond to Broca’s area) in men than in women. Callosal size has been the object of study by several groups of authors. Most have agreed that some parts of corpus callosum are larger in women than in men, but there is no general consensus. Cowell, Allen, Zalatimo, and Denenberg (1992) found that in men, the corpus callosum reaches its maximal width by age 20, and in women much later—between 41 and 50 years of age. These age-dependent differences may be one of the causes of discrepancies in the literature. Besides, the callosal surface size is not a reliable indicator of the actual number of commissural axons (Judaš & Kostovi´c, 1997). In any case, even if the size difference is accepted, it does not necessarily result in different cognitive functioning (Kimura, 1992). 39
40
The Cognitive Neuroscience of Human Communication
DIFFERENCES IN FUNCTIONAL ORGANIZATION OF THE BRAIN According to Kimura (1992), sex hormones begin exerting their influence on brain organization so early in ontogenetic development (prenatally) that from the very moment of birth the environment acts on the differently structured brains of girls and boys. She found that 3-year-old boys were better at aiming than girls of the same age and that sex differences in rotating objects in space were evident before puberty. Although the influence is less pronounced in humans than in other animals, early exposure to sex hormones has permanent effects on brain organization. Cohen and Forget (1995) described possible hormonally determined regulation of activities in brain hemispheres during cognitive tasks. According to Bradshaw and Nettleton (1983), left-hemisphere injuries affect language abilities to a lesser degree in women than in men, due to their less prominent left-lateralization of language functions. Kimura (1983, 1992) does not believe that these different consequences of left-hemisphere damage are due to more bilateral organization of language functions in women. According to her, unilateral focal damage occurs more frequently in posterior areas than in anterior ones, regardless of sex. The difference lies in the fact that in more women, speech and speech-related motor functions are represented in more anterior regions of the left hemisphere as opposed to men, who on average, more frequently have the same functions represented in posterior regions of the left hemisphere. Still, Kimura (1992) reports no evidence of sex differences in functional brain asymmetry with respect to the basic speech, motor, or spatial abilities. Kanasaku, Yamaura, and Kitazawa (2000) also found sex differences in lateralization to vary along the anterior–posterior axis. She did report, however, differences in more abstract verbal tasks. For instance, damage of any hemisphere caused poorer results on verbal tests in women, whereas the results were poorer only in men with left-hemisphere damage. Shaywitz et al.’s (1995) results of fMRI studies of rhyming tasks show that women exhibit more bilateral activation than men. Other authors also report task-specific sex differences (for example, Kitazawa & Kansaku [2005], and Rossell, Bullmore, Williams, & David [2002]). Developmental dyslexia is more frequent in boys than in girls. Moreover, it has been shown that dyslexic girls still function better than dyslexic boys, which suggests greater plasticity during development (Spreen et al., 1984). As it has been mentioned at the beginning of this chapter, male and female brains are organized differently from the first days of life. During prenatal development sex hormones control that differentiation. Similar mechanisms probably cause within-sex variation, as suggested by the relationship between levels of some hormones and adult cognitive styles (Kimura, 1992; Sherwin, 2003).
BEHAVIORAL AND COGNITIVE DIFFERENCES Sex differences in cognitive abilities are most frequently and most consistently manifested in the language and spatial domains (Spreen et al., 1984). In general, it is believed that the two hemispheres are more asymmetrically organized for
Sex Differences
41
speech and spatial functions in men than in women. The considerable difference in the size of Brodmann’s area 22 (by approximately 11%) is thought to underlie the sex differences in language abilities (Verchinski et al., 2000). On IQ tests, girls score consistently better on verbal tasks and poorer on visuospatial tasks than boys, especially after puberty. It seems that the most pronounced differences in cognitive functions are to be found in the types of abilities and not in the overall intelligence level as measured by IQ (Bradshaw & Nettleton, 1983). On average, men outperform women on some spatial tasks. They are particularly more successful on tests that require mental rotation of an object in space or some other type of manipulation. They are more successful on mathematical reasoning tests and in orienting on a given route. They are also more precise on tests of target-directed motor skills—in other words in aiming and intercepting. Women have greater perceptual speed (they are quicker to find objects that match some target object). Women display greater verbal fluency, including the ability to find words that begin with a given letter or that satisfy some other criterion. Women outperform men in mental arithmetic and remembering landmarks. On precise manual tasks (e.g., O’Connor’s dexterimeter) women are faster than men. These differences are manifested in gender distribution in certain occupations (composers, chess players), disorders (developmental dysphasia and dyslexia, stuttering, autism), in normal developmental stages (speech acquisition), and in overall behavior during childhood (Swaab & Hofman, 1995). Boys tend to investigate more, girls tend to be more social (Bradshaw & Nettleton, 1983). These traits may be encouraged by parents and society. Hereditary factors probably have a stronger impact on spatial skills whereas environmental factors seem to influence verbal abilities more. Sex differences seem to be more pronounced in more authoritarian cultures with rigidly defined gender roles (Bradshaw & Nettleton, 1983). Kimura (1992) believes that human brain organization has been determined by natural selection through generations. During the thousands of years of human brain development people have lived in relatively small groups of hunters and gatherers, with probably clear division of labor between men and women. Men engaged in hunting, which frequently involved covering great distances and developing strategies for finding and overpowering animals. They were responsible for the defense of the group against predators and enemies as well as for design, manufacture, and use of weapons. Women were most probably in charge of tending to home, which involved child care, gathering food nearby, preparing meals, and making clothes. Such specialization assumed different pressures on men and women. Men had to develop skills of aiming and finding their way on the basis of terrain configuration, and women had to develop fine motor skills in a limited space and sensitivity to small differences in the environment as well as in the appearance and behavior of children. These speculations aside, there is no firm evidence that would explain sex differences in cerebral representations of verbal and spatial functions as a consequence of environmental and/or social factors. Sex differences are not exhausted by the verbal–nonverbal aspects of processing. There are also differences in the degree of lateralization. At first it was
42
The Cognitive Neuroscience of Human Communication
believed, probably due to different rates of maturation, that women were more lateralized than men. At present the opposite is more commonly found and believed to be true. Smaller differences between the left and the right hemisphere in women compared with men have been found in studies of morphological asymmetries, clinical studies (women are less affected than men by left-hemisphere trauma), and behavioral studies on healthy subjects. Although, paradoxically, men are more frequently and strongly left-handed than women, they exhibit more pronounced right-visual-field and right-ear advantage for verbal material. Sex differences are as a rule easier to show on verbal material, just as verbal tasks generate stronger and more consistent asymmetries than nonverbal, visuospatial materials (Bradshaw & Nettleton, 1983). Women frequently exhibit left-visual-field advantage on lexical decision tasks (but not when vocal latencies are measured) early on in the task. On phonological tasks (as revealed by MRI) men exhibit more left-lateralized brain activity compared with women, whose activation is more diffuse (Shaywitz et al., 1995). Women resemble left-handed men by exhibiting the verbal contribution of the right hemisphere, suggesting that in women the areas otherwise reserved for visuospatial processing in the right hemisphere are taken over by the secondary speech mechanisms that facilitate comprehension of difficult and/or unfamiliar material. Moreover, women with right-hemisphere injuries score more poorly on tests of verbal intelligence, but show no signs of dysphasia. It seems that the right hemisphere in women is more active in speech comprehension than in production, which may explain the absence of obvious effects when it is suppressed by the unilateral sodium amytal injection (Bradshaw & Nettleton, 1983). However, it should be said that cerebral blood flow measurements obtained by PET studies have revealed that, both in men and in women, blood flow increases in the left hemisphere during verbal tasks and in the right hemisphere during spatial tasks (Kolb & Whishaw, 1996). McRoberts and Sanders (1992) reported that in the recognition of fundamental frequencies, men were significantly better than women when the stimuli were presented binaurally (i.e., to both hemispheres simultaneously). Dichotic listening tests have revealed no significant sex differences. They speculate that the reason for that may be the differences in relative callosal width. In a study of the relationship between active hand and ear advantage carried out on 72 male and 72 female subjects, Mildner, Stankovi´c, and Petkovi´c (2005) reported consistently lower right-ear advantage for male than for female subjects. Sex differences in verbal and spatial tasks may be a consequence of unequal maturation rates. Early maturing adolescents (typically women) achieve better results on verbal than on spatial tests and are less clearly lateralized. Those who mature more slowly have a chance to stabilize their hemispheric dominance later, which helps spatial abilities if language develops before them. Thus, late maturers (boys) can develop language abilities in the left hemisphere without interference with the right-hemisphere functions. Accordingly, the right hemisphere is free to develop its visuospatial abilities (Bradshaw & Nettleton, 1983). Besides resulting in more pronounced lateralization and separation of functions, an extended period of maturation typical of men might mean a longer critical period during which
Sex Differences
43
malfunctions in verbal ability development may occur. Developmental dyslexia in boys may be a consequence of too much reliance on bilaterally represented spatial functions that develop late, which does not happen in girls because they are protected, having had their language skills developed earlier. The greater degree of hemispheric specialization probably puts men at risk with respect to language disorders, because they have no alternative support system in the nondominant hemisphere. So, if the differences are summarized into the different maturation rates it may be concluded that, regardless of gender, early maturers have better developed verbal abilities, whereas late maturers develop better spatial skills. Contrary to this, authors such as Shucard (as cited in Spreen et al., 1984) have reported that sex differences in cognitive functioning are apparent in babies as young as three months. Sex differences have been attributed to hormonal influences as well. It is possible that sex hormones change male and female brains in such ways that they respond differently to social influences. Some genetic models have also addressed variations in spatial abilities (Kolb & Whishaw, 1996). Perceptual asymmetries may vary in men and women because of different approaches to the task. For instance, women might prefer verbal strategies during spatial tasks, so that the obtained lateralization effect would not be a measure of sex differences in cerebral lateralization of that particular function, but rather a manifestation of the preferred cognitive style in task solving (Kolb & Whishaw, 1996). Cahill (2003) found sex differences in memory for emotionally arousing events. Right hemispheric processing of global/central aspects of a situation is more pronounced in males, whereas left-hemispheric processing of more local/ fine detail aspects is more pronounced in females. Finally, it is possible that genetic, developmental, and environmental factors predispose men and women for a particular cognitive style, which influences their successfulness in solving verbal and spatial problems. Most probably, gender differences are at least partly under the influence of neurological factors that may be changed by the environment (Kolb & Whishaw, 1996). Sommer, Aleman, Bouma, and Kahn (2004) reviewed 14 studies involving 377 men and 442 women. Their meta-analysis indicated no significant difference in language lateralization between men and women (see also Sommer, Aleman, & Kahn, 2005). They proposed that general sex differences in cognitive performance and the neuronal basis for these cognitive sex differences are not to be found in different language lateralization.
History of 3 Brief Neurolinguistics from the Beginnings to the 20th Century First descriptions of language disorders date back to pharaonic medical texts recorded on papyrus, some 3000 years B.C. The outlines of the first theory of brain and nervous system functions appeared in the first centuries A.D. and were based on ideas originating in the last few centuries B.C. The so-called ventricular theory was based on fluids and their flow, because the scientists of the day believed that something had to move in the brain in order for it to work. That “something” flowed between the sensory and the effector organs and was called animal spirits. The theory held on until the 17th century and Descartes (1596– 1650) incorporated it into his hydraulic theory. Some principles of ventricular theory (later also referred to as the medieval cell doctrine) have been kept in the more recent models of brain function developed within cognitive neuroscience (e.g., modular structure according to which the functions are localized in various regions of the brain). Between the 11th century and Renaissance, written descriptions of medical cases included drawings from which it was obvious that speech production was located, together with memory, in the third ventricle. Moreover, language and memory disorders were commonly described as a single problem. In the 16th century, the first associations between language disorders and certain brain injuries were determined. Different aspects of impaired language function were observed, which may be seen from descriptions of alexia and agraphia, aphasia and even selective aphasia, with preserved internal speech and thought but with seriously impaired speech production. There were also records of aphasia without alexia or agraphia. This was accompanied by descriptions of differences between articulatory control and language production, because clearly, there were patients who had preserved motor functions but could not speak. In the 17th century, the increasing number of experimental studies of brain brought about abandonment of ventricular theory and acceptance of models that proposed localization of functions in particular brain structures. The Renaissance belief that the nerves relayed information to and from the brain was kept but the idea that nerves were hollow tubes through which energy flowed back and forth was abandoned. Instead, it was posited that information was coded so that different ideas were transmitted by means of different vibrations. The number of 45
46
The Cognitive Neuroscience of Human Communication
descriptions of simultaneous and related occurrences of language disorders and right-sided paralysis increased; descriptions of cognitive function impairments became more thorough, including discussions about acalculia, agrammatism and other disorders, and with discussions about differences between disorders accompanied by agraphia and/or alexia and those without them. Language was still closely associated with memory. The scientists could distinguish language disorders from general cognitive impairments, production from comprehension, lexical errors from sound or letter errors, as well as syntactic and morphological disorders. The brain was believed to function as a whole. The idea of localization of functions in the brain was adopted and further advanced in the 18th century. It was believed that each ability or trait was connected with a special nerve bundle in the brain. At the time, sensory, moral and intellectual traits and abilities were considered to be equal candidates for localization. Moreover, some authors believed that each word had its own neuronal fiber. Franz Joseph Gall (1758–1828) thought that the shape of the human brain with all the individual traits left a trace on the inside wall of the skull, which could be observed from the outside as bumps. This was called craniology. He also proposed two separate brain areas devoted to two language levels and marked them as Wortsinn (for the lexical component) on the left orbit, and immediately above that was the place for expressive speech—Sprachsinn. The relationship between abilities and skull shape established by Gall served as the foundation for phrenology, a very influential pseudoscience that had a significant impact on the 19th century, most closely connected with Johann Gaspar Spurzheim (1776–1832). In line with Gall’s descriptions, the phrenologists proposed specific locations in the left and the right hemisphere for different functions that were reflected on the external shape of the skull. There were areas responsible for hope, love, caution, hate, shyness, language and many other functions, abilities and traits; within the brain there were the organs of hope, caution, and so forth. For example, it was believed that people with bulging eyes had exceptionally developed language abilities, because the seat of language functions was thought to lie behind the eyes. One of the most heated opponents of phrenology was Pierre Jean Marie Flourens (1794–1867). On the basis of his tests on animals he concluded that all senses, perception, and voluntary activities had common brain substrates. In spite of numerous critics, many authors were looking for connections between impaired functions and brain damage as support for phrenology until well into the third decade of the 19th century. Although not a fervent supporter of phrenology, Jean Baptiste Bouillaud (1796–1881) supported the theory of brain localization, particularly localization of language function in frontal lobes. He offered ample clinical evidence of the connection between frontal lobe injury and loss of speech. Moreover, he described several cases of isolated speech disorders without other motor impairments and with intact nonspeech tongue movements, on the basis of which he concluded that the center for motor control of speech is separate from the center(s) for general motor control. He also distinguished between the organ of spoken language
Brief History of Neurolinguistics From the Beginnings to the 20th Century
47
(i.e., speech), located in the white matter, from the organ of memory for words (i.e., lexical representation), located in the gray matter, in both frontal lobes. Pierre Paul Broca (1824–1880) focused on disorders of speech production because until the sixth decade of the 19th century speech referred only to production, whereas disorders of speech comprehension were not discussed within language proper. At the time comprehension processes of any kind were thought to belong to the mind that was studied by philosophers, theologists, and psychiatrists. By the time his well-known papers were published in the 1860s, a 40year-long debate about the role of frontal lobes in speech production and about the associations between speech disorders and injuries, and stroke in that region had been raging in medical circles. It was clear from many published case studies that most physicians were aware of the hypothesis about the existence of a language center in frontal lobes. Cases of aphasia without frontal lobe damage were taken as counterarguments, and not as evidence of different types of aphasia. Before Broca’s papers, an unknown village doctor, Marc Dax (1770–1837), reported at a medical convention in Montpellier, France, his observations that of his 40 brain-damaged patients who presented with speech problems, none had a right-hemisphere injury. That report went unnoticed, and Dax died a year later, unaware of having anticipated research into lateralization of function. Whitaker (1995) believes that even when Broca’s results were published 25 years later, they might have gone unnoticed had not scientists and doctors already begun accepting clinical pathological associations between specific brain regions and particular functions. Broca presented postmortem results of two of his patients, followed by seven more aphasic patients, who had suffered left-hemisphere injuries, specifically in the frontal lobes, anterior to the motor area. Because of these results and his detailed case descriptions, this area—for speech production—is called Broca’s area. Soon after that, authors began characterizing neurologically acquired speech comprehension disorders. Several names were the most prominent: John Hughlings Jackson (1835–1911) and Carl Wernicke (1848–1905), as well as, according to Whitaker (1998a) the unjustly overlooked Theodor Meynert (1833–1892). It became widely accepted that the left hemisphere was responsible for language in general, not just for speech production. Studies showed that left-hemisphere damage caused difficulties in reading, writing, and speech comprehension, which was not the case with right-hemisphere damaged patients. Because of Wernicke’s descriptions of sensory aphasia occurring as a consequence of injuries to the posterior superior temporal gyrus (approximately at the point of contact of the temporal and the parietal lobe), the center for speech comprehension was named Wernicke’s area. Case studies revealed that apraxia was another consequence of left hemisphere damage. On the basis of these results the left hemisphere became known as the dominant hemisphere, due to its crucial role in the control of all complex behavioral and cognitive processes. Some authors, among whom Whitaker (1998a) particularly points out Henry Charlton Bastian (1837–1915), were aware of the fine differences between various aspects of language functioning and types of memory. Bastian found that function disorders (e.g., aphasia) may
48
The Cognitive Neuroscience of Human Communication
be caused by injury to the center itself, to its connections with periphery, or to some other connections. Obviously, his model had some properties of the neural networks models. According to him, words establish connections with percepts and concepts, and during the thought process brain activity is not limited only to strictly localized centers, but there is a widespread activity in various cortical areas in both hemispheres. He proposed that concepts were formed in cortical areas immediately next to sensory centers, which would correspond loosely to the idea of association areas. Whereas studies by Broca, Wernicke, Bastian, and many others dealt mostly with the taxonomy analysis of the observed phenomena at the word level (or lower), John Hughlings Jackson was the first to give any serious thought to agrammatism. He talked about the propositional nature of language and concluded that propositional meaning did not depend only on constituent words, but also on relationships among words, making the sentence the unit of meaning. Hughlings Jackson also claimed that the right hemisphere was involved in automatic (involuntary) expression and perception. His particular contribution is a description of the motor cortex (precentral gyrus), especially the observation that different parts of the contralateral side of the body are represented in specific locations of the cortex. He also noticed that complete loss of function was rare, and based on that, concluded that a particular type of behavior was managed by many regions of the brain. Another one of his conclusions was that localization of symptoms does not necessarily imply localization of function (Gazzaniga, Ivry, & Mangun, 2002). Arnold Pick (1851–1924) believed that sentence plan (i.e., syntactic structure) preceded the selection of lexical constituents and that content (lexical, open class) words were selected before functional (grammatical, closed class) ones, with surface grammatical form being determined last. He talked about two types of expressive agrammatism: motor (quantitative) and sensory (qualitative), and attributed both to the loss or decrease of control over grammatical devices. He related motor agrammatism to motor aphasia, frontal lobe injuries and telegraphic speech, and sensory agrammatism to temporal lobe injuries, but his emphasis was on expressive aspects of speech. After the relatively long and very active period of localizationism, a more holistic approach to brain study was adopted at the beginning of the 20th century. One of the most influential proponents of that view was Karl Lashley (1890–1958). In his studies of rat behavior he concluded that there was no single location in the brain whose injury might prevent maze learning. His most important finding was that the extent of the injury, more than its location, will affect a particular function. He defined two important principles: the principle of mass action and the principle of equipotentiality. According to the first, certain types of learning are controlled by the cerebral cortex as a whole (contrary to the view that each cognitive function has a special location in the cortex). Italian physiologist Angelo Mosso (1846–1910) noted a relationship between mental activity and increased number of pulsations in the cortex, in other words, between cerebral blood flow and neuronal activity (Gazzaniga et al., 2002). According to the second principle,
Brief History of Neurolinguistics From the Beginnings to the 20th Century 49
which mainly refers to sensory systems, some parts of the system can take over functions from the other parts. By the second decade of the 20th century it has become clear that the nervous system was built of dynamically polarized nerve cells, whose axons and dendrites were organized into groups, and made specific neural connections by means of synapses. In the 1930s the first motor and sensory maps were produced, and it became obvious that each sensory modality has more than one of those maps. After World War II different methods of imaging and quantifying these connections were designed (more on that in chapter 4, this volume). The obtained results and techniques ensured better understanding of brain physiology and changed the approach to speech and language. However, the key question—does the brain function as a whole or as a set of independent parts—continues to be debated. According to Kosslyn and Andersen (1992), it seems plausible that simple processes are localized, whereas complex functions such as perception, language and others are more widely distributed (as cited in Gazzaniga et al., 2002). This simple distinction does not solve the problem, of course. Rather, it moves it to a different plane: to the question of the level of complexity at which a process is localized, or at what point it becomes complex enough to warrant wider distribution.
4 Research Methods Methods of research into brain functioning have developed alongside advancements in knowledge and technology. The application of a particular method depends, naturally, on the function we wish to study, subjects, available resources and equipment, and frequently on what is in vogue, or for some other reason the focus of interest, at the particular point in time in the scientific community. Existing methods may be characterized in several ways: according to the subjects studied (e.g., clinical population vs. healthy subjects); according to the functions studied (e.g., language, motor, etc.); according to type of tasks used (e.g., detection, identification, or discrimination); chronologically (from the oldest to the most recent); or according to the techniques used. Obviously, some overlap is unavoidable and, regardless of the criterion, one method may fit into several categories. This is not an exhaustive list, but it includes brief descriptions of the methods most frequently used in neurolinguistic research on clinical and healthy subjects, as well as those that are, due to their invasiveness, costliness, or relative obsoleteness, used less frequently. Many methods used with healthy subjects are also applicable to clinical patients, such as neuroimaging methods. Since there are advantages and disadvantages to every single one of them, most authors agree that a combination of several different methods on the same subjects will yield the most reliable results. This chapter is limited mainly to the methods acceptable in work with humans, so different lesion methods, genetic and/or pharmacological manipulations, deprivation, and similar methods are not included. Their descriptions may be found in neuroscientific literature (Gazzaniga, Ivry, & Mangun, 2002; Judaš & Kostovi´c, 1997; Purves et al., 2001; Sternberg, 2003).
CLINICAL STUDIES This large group of subjects includes patients with brain damage, regardless of etiology—trauma (i.e., open or closed head injury), cerebrovascular problems (e.g., stroke), tumor or degenerative diseases and conditions (e.g., Alzheimer’s or Parkinson’s disease, epilepsy, etc.). The first documented studies of brain language functions were carried out on clinical patients, particularly in the 19th century. The authors, mainly physicians and surgeons, described observed speech disorders and tried to make connections between their observations and diagnosed brain injuries. Whenever possible they looked for confirmation of their findings in postmortem analyses. They are referred to as diagram makers, because they tried to establish unambiguous relations between injuries and function disorders. In other words, their goal was to 51
52
The Cognitive Neuroscience of Human Communication
draw inferences about the ways different parts of the brain contribute to behavior. The disadvantages of those studies were the strictly localizationist approach, insufficiently defined psychological terms, and unsatisfactory empirical descriptions. However, brain injuries and their consequences cannot be reduced to the loss of a single component. Damage to one function does not mean that other aspects of cognitive functioning are intact. For instance, formation of scar tissue or disorders in blood supply may affect functioning of neighboring regions. Finally, it is debatable how much we can infer about the functioning of individual components from information about the consequences of their removal. Even the most detailed description of difficulties will not answer the question of how the language system actually works. At best we will find out what aspects of the system have been damaged. Obviously, we cannot reliably make conclusions about functions of the removed area on the basis of facts about how behavior changes when the area has been removed. Furthermore, the key technical issues here are whether the structure in question has been removed entirely and what other structures may have been damaged (Gordon et al., 1994). It is difficult to infer about functioning of the intact brain on the basis of the injured one, because the exact extent of damage or recovery rate is usually not known. It has been shown that more reliable associations between injury and functional disorder may be seen approximately one year after injury. Recovery is most remarkable in the first few weeks and months, but to a certain extent it continues after that initial period (Dronkers & Ludy, 1998). It is also possible that methods to determine functioning are just not sensitive enough to register change. Obviously, one of the more serious drawbacks of forming any conclusions about intact functioning on the basis of clinical cases is that normal functions of the system are not likely to equal those lost by damage. Consequences of injury may be masked by positive symptoms, for instance, events occurring in the injured individual that would not have occurred in a healthy one (Legg, 1989). Moreover, it is usually impossible to control some other characteristics of the patient—the natural aging process, psychological status, other health problems, and so forth. Another important issue is which results are more reliable: those obtained from large groups of subjects or those based on single case studies? The average behavior of many patients who supposedly suffer from identical disorders or injuries to a particular brain region does not necessarily offer a better insight into cerebral representation of a cognitive function. In averaging, considerable anatomical and functional individual variation may be lost. Moreover, individuals who differ widely not only in the extent of their injuries but also in other characteristics that may significantly affect recovery, such as education, age, socioeconomic status, attitude toward therapy, motivation, and so forth, are frequently studied together as a homogenous group. This approach ignores individual compensatory strategies that also play an important role in recovery and tends to place too much emphasis on some areas of the brain, neglecting others that are seemingly inactive or less active during certain functions. Although group studies are carried out on patients who share some common characteristics of linguistic behavior, or that have been diagnosed with damage to the identical region of the
Research Methods
53
brain, the groups are still quite heterogeneous. Obviously, it is an advantage that individual case studies avoid unjustified overgeneralizations and do not involve large groups of patients with similar cognitive deficits. In this type of study individuals are subjected to large numbers of different tests, which can offer insights into their functional cerebral organization and help to test theories and models of cognitive functioning. One of the most obvious disadvantages of this kind of research, naturally, is the fact that one case cannot serve as a basis for general conclusions, because individual behavior is affected by many different factors (some of which have been mentioned previously) that may not be noticeable or directly measurable. Willmes (1998) and Eysenck and Keane (2000) propose a sensible compromise: start with individual cases and look for what they have in common with other individuals. Despite all the drawbacks, clinical case studies are a valuable source of data about cognitive and other functioning.
STUDIES OF SPLIT-BRAIN PATIENTS One type of brain injury is severing of connections between the two hemispheres (commissurotomy). In humans, the most frequent version is cutting the corpus callosum (callosotomy) with the purpose of preventing the spread of epileptic seizures from the affected hemisphere to the healthy one. Another small group of patients without callosal connections are the so-called acallosal patients, in whom the connection was never established. Commissurotomies became the method of choice between the 1940s and 1950s, but were subsequently replaced by increasingly more effective antiepileptic medications. At present the procedure is used only exceptionally, if all other attempts to stop epileptic seizures fail. In cases of nonexistent callosal connections between the two hemispheres, the centers in each of them are intact and each hemisphere can operate on its own. Studies have shown that, under experimental conditions, such patients function as if they had two separate brains, which is not noticeable in everyday life (Milner, Taylor, & Sperry, 1968; Sperry, 1981; Zaidel, Zaidel, & Bogen, 2005). As it was explained in chapter 1, the corpus callosum is the largest connection between the two hemispheres. It is more than just a vital communication line, because it coordinates hemispheric functioning and behavior. Although we have two hemispheres capable of neural activity and—as hemispherectomy patients show us, we can function with only one—we behave as one mind, not two. Two hemispheres between which the corpus callosum has been severed can learn identical tasks, whereas intact corpus callosum prevents that. In callosotomy patients, one hemisphere has no awareness of the existence or activities of the other. This was supported by observations of perception, learning, cognition, volition, and memory (Levy, Koeppen, & Stanton, 2005). However, the two hemispheres can exchange information, for example, by means of facial expressions that are bilaterally controlled, and via some subcortical connections (Kalat, 1995; Sternberg, 2003). A typical experiment carried out on split-brain patients looks like this: the patient is seated at a table with test objects hidden from view by a partition. The
54
The Cognitive Neuroscience of Human Communication
subject is required to pick up or feel the object within reach with one hand, and to (a) name it, or (b) feel for an identical object with the other hand. The tests are run for each hand separately. Due to crossed connections between the periphery and the brain the right hand has direct access to the left hemisphere, and the left hand to the right hemisphere; because the two hemispheres have been disconnected, the information that reaches the brain cannot be transferred from one hemisphere to the other. Hemispheric differences in language functioning are manifested as the inability of the patient to name the objects felt with the left hand (right hemisphere), with preserved ability to name those felt with the right hand (left hemisphere). The results are highly dependent on the type of commissurotomy—whether it was complete or partial, and if it was partial, how much of the connection was preserved. For example, an intact splenium will preserve the connection between occipital lobes, and the anterior commissure will enable transfer of semantic (but not sensory) information (Gazzaniga et al., 2002).
CORTICAL STIMULATION Cortical stimulations involve administering stimuli to a part of a neural circuit and measuring its consequences at some other point. Stimuli may be electrical or chemical. Although the first results of cortical stimulation in an awake person were reported as early as 1909, the method was used more systematically by Penfield and Roberts (1959) with the purpose of determining cortical locations crucial for language functioning during brain surgery in order to spare them. Since the brain has no pain receptors the procedure is performed while the patient is awake. Electrical stimulation of particular points in the cortex elicits various sensations, images, and experiences that are entered into the so-called stimulation map as the patient describes them. Hence the term brain mapping. Stimulation of speech areas causes interference in object naming. Patients may lose the ability to speak entirely or just the ability to name objects presented in pictures. Hesitations, slurred speech, repeated attempts to name the objects, and other effects have also been recorded. Such stimulation maps have been compiled for functions other than speech production and perception as well. There are several methodological problems to be dealt with here. One is the problem of whether the stimuli are physiologically real in terms of their intensity and/or duration as well as the issue of interpretation of localized stimulation of a system that under natural circumstances most probably receives complex stimulus structures. The main technical problem is how to apply a stimulus of intensity that faithfully reflects the level of activity occurring spontaneously in the brain, and how to determine which structures were affected by stimulation: the brain is structured so that information about the same event may be coded in the activity of a large number of not necessarily neighboring cells. Furthermore, the information on when a cell is inactive is just as important as knowing when it is active, because the brain’s electrical activity is temporally organized. If we do not know in advance how the information is represented in the particular brain region, stimulation will not be realistic, and the results will be worthless (Legg,
Research Methods
55
1989). Besides, it has been shown that human brains do not have identical functional organization of language functions (Whitaker, 1998b).
TRANSCRANIAL MAGNETIC STIMULATION (TMS) This method is used for experimental purposes on healthy subjects. It simulates injury of a desired brain region. An electrical coil is positioned on the skull over the location of interest. The resultant magnetic field stimulates neural activity on the area of about 1 to 1.5 cm2 around the coil position. For example, stimulation of the motor cortex activates wrist and finger muscles in about 20 ms; stimulation of the visual cortex causes inability to recognize letters. An obvious disadvantage of this method is that it may be used only to study cortical surface. Furthermore, stimulation effects are very brief, and the timing of the signal beginning and stimulation onset is crucial: in the previously mentioned example of letter recognition, the errors will be elicited only if stimulation is applied 70 to 170 ms after the letter has been presented (Gazzaniga et al., 2002). Another drawback of this method is a well-established risk of inducing seizures, and recently it’s been suggested that memory can be affected as well (Boatman, personal communication).
WADA TEST This method, first introduced by Huhn Wada in 1947, is used clinically to determine the language-dominant hemisphere in patients who are candidates for neurosurgery. The procedure involves injecting the anesthetic sodium amytal into the carotid artery on the side of the neck, which supplies blood to the brain. This temporarily blocks the function of the ipsilateral hemisphere, with practically no effect on the contralateral one. Within seconds of the injection, the hand opposite the injection side starts to drop, because the anesthetized hemisphere cannot control it any longer. During the 5 to 10 minutes the anesthetic is effective the patient is presented with various (mainly linguistic) tasks. For example, reciting the alphabet, days of the week, or months, and naming well-known objects. The procedure is carried out for one side at a time. The patient’s inability to perform the tasks successfully is taken as an indication that the anesthetized hemisphere is dominant for that particular activity. With the anesthetized left hemisphere about 95% of right-handers and approximately 70% of left-handers cannot talk at all for about 2 minutes. For several more minutes after that they behave as aphasics—making errors in order and/or naming. With the anesthetized right hemisphere there are typically some minor speech problems, but speech is never inhibited entirely (Kolb & Whishaw, 1996). About one half of the remaining 30% of left-handers behave just the opposite—total failure of speech functions occurs with anesthetized right hemisphere, with some inhibition with the anesthetized left hemisphere. The other half has apparently bilateral speech control because the consequences are the same regardless of the anesthetized hemisphere. The results of Wada tests are usually compared with behavioral tests performed on the same subjects. The main disadvantage of this method is its invasiveness and
56
The Cognitive Neuroscience of Human Communication
inappropriateness for healthy subjects. Apart from that, if used on its own, it can only help determine the dominant hemisphere, but not regions of increased activity during a particular function.
NEURORADIOLOGICAL METHODS A conventional X-ray can reveal, for example, skull fracture, but it is not appropriate for visualizing different brain structures. Somewhat better are contrast techniques. These techniques involve temporary substitution of some of the cerebrospinal fluid with air (nowadays considered obsolete) or injecting the cerebral artery with a dye impermeable to X-rays (angiography); such techniques enable visualization of ventricles and fissures, as well as of the vascular system of the brain. However, they are invasive and their use is only clinically justified. Besides that, they are not appropriate for functional neuroscientific research because they do not provide clear identification or differentiation of gray and white matter structures.
COMPUTERIZED (AXIAL) TOMOGRAPHY—C(A)T This technique involves computer-supported use of X-rays. It is based on the principle that different biological materials vary in their density, which affects absorption of X-rays. High-density tissues (e.g., bones) absorb a great deal of radiation, neural tissue somewhat less, and low-density materials such as blood or air the least. Due to this it is possible to obtain a three-dimensional image of the living brain by putting together a series of X-ray scans of its horizontal sections. By means of contrast, this procedure offers good view of injury location, with somewhat poorer images of more recent injuries and those closer to the skull. Due to their similar density, it is difficult to distinguish white matter from the gray. Spatial resolution is between 0.5 and 1 cm of tissue (Gazzaniga et al., 2002).
MAGNETIC RESONANCE IMAGING (MRI) The images are obtained by measuring the energy of waves emitted by hydrogen protons after being activated in the magnetic field. Depending on the model, the magnetic field of an MR machine is more powerful than the earth’s magnetic field by about 500 times or more (0.001 vs. 0.5 tesla, or more, even up to 7 tesla). The advantage of this technique is its noninvasiveness—the brain is not exposed to any radiation—but it is relatively slow (it takes about 15 minutes to complete) and noisy. Spatial resolution is less than 1 mm (Gazzaniga et al., 2002). This method provides good images of the brain structure and structural changes (gyri and sulci, white and gray matter, injuries, tumors, or morphological changes such as one finds in Alzheimer’s disease patients), but poor functional images (Eysenck & Keane, 2000; Kalat, 1995). A better method for studying brain activity during cognitive functions is functional magnetic resonance imaging, which is based on this technique.
Research Methods
57
FUNCTIONAL MAGNETIC RESONANCE IMAGING (FMRI) This method provides a series of functional images of the brain while engaged in a particular activity. It seems that our brains resort to anaerobic metabolism to supply energy necessary for temporary increases in neuronal activity. As a result, the amount of blood oxygen in the activated area increases and the image of brain activity is based on the magnetic properties of hemoglobin, the principal oxygen carrier in the bloodstream. When the oxygen is absorbed, what remains is deoxygenated hemoglobin that is more sensitive and has more powerful magnetic properties. The ratio of oxygenated and deoxygenated hemoglobin can be measured, and this is called the blood oxygenation level dependent (BOLD) effect. During increased activity, the amount of oxygen is greater than the possibility of its absorption, so the ratio is usually in favor of oxygenated hemoglobin. This method poses minimal biological risk to the subject, but is counter indicated in patients who wear pacemakers, cochlear implants and/or other metal implants. The signal is obtained by functionally provoking changes in brain tissue (there is no injection or inhalation of radioactive or other substances); it provides anatomic and functional information about each subject, and its spatial resolution is good (1 to 2 mm, i.e., 3 mm3 of volume). The disadvantages of the technique are that it is an indirect measure of neural activity, and it cannot keep up with the pace of most cognitive processes. Temporal resolution is several seconds: the first changes are visible several seconds after the start of activity, with the greatest increase occurring about 6 to 10 seconds later (Eysenck & Keane, 2000; Gazzaniga et al., 2002; Kalat, 1995; Posner & Raichle, 1997). fMRI is the method of choice for most cognitive neuroscientists. Similarly to PET (see the following for a more detailed description), it is based on subtraction of activity patterns at rest from those recorded during a task. Its great advantage is the practically unlimited number of measurements (Demonet, 1998). Due to this, the BOLD effect may be temporally aligned with a particular event, and combined with multiple repetitions of the same event and averaging (similarly to event-related potentials, see following discussion) it may provide a clearer image of activity in particular brain regions (Gazzaniga et al., 2002). The method is extremely useful in longitudinal studies that monitor recovery from aphasia and the plasticity of language functions (Papke, Hillebrand, Knecht, Deppe, & Heindel, 2000).
RECORDING OF ACTIVITY These methods are based on the following assumption: if a particular brain region is crucial for a process (e.g., cognitive, motor, or sensory) there will be increased activity in that region when the process is initiated. The technology and techniques of recording brain electrical activity have been developing for approximately 100 years. During this time it has become possible to measure more than just electrical activity, for example, oxygen and glucose consumption, or blood flow in a particular brain region while the subject is performing a task.
58
The Cognitive Neuroscience of Human Communication
Measuring regional cerebral blood flow (rCBF) underlies many techniques, and it is based on the fact that blood flow increases in the activated parts of the nervous system. Several blood flow measurement techniques are in use. One of them involves injecting radioactive, chemically inert gas, xenon (133Xe) into the carotid artery (or inhaling it), and measuring radioactivity accumulated in different parts of ipsilateral cortex. Blood flow may be measured by (functional) infrared spectroscopy (f)NIRS that is based on the brain’s permeability to infrared rays. This method actually measures changes in the concentration of deoxygenated and oxygenated hemoglobin. It is noninvasive, fast, and relatively cheap, but its spatial resolution is poor (Cooper & Delpy, 1995). The problem common to all these techniques is that increased blood flow is present when excitatory and inhibitory neurons are active, which makes the results unreliable with respect to the nature of activity (Colier, Jaquet, van der Hoeven, Hagoort, & Oeseburg, 2000; Posner, 1995). It is also interesting, and should not be neglected when interpreting findings, that people with high blood pressure present a different picture from those whose blood pressure is normal. van der Veen et al. (2000) found their hypertonic patients showed increased bilateral blood flow during cognitive tasks that commonly cause increased activity in only one hemisphere in control subjects. In their patients, even the classical dichotomy between verbal (left-hemisphere controlled) and spatial (right-hemisphere controlled) working memory tasks was substituted by increased blood flow in the left hemisphere regardless of task.
ELECTROPHYSIOLOGICAL METHODS Some methods of measuring electrical activity register activity of large groups of cells, whereas others record action potentials of single cells. Single-Unit or Single-Cell Recording This method was developed some 40 years ago with the aim of studying the activity of single neurons. Because of its invasiveness it is typically used in laboratories on experimental animals, which is its major drawback. A tiny electrode, approximately 1/10000 mm in diameter, is inserted into the animal’s brain. This enables measurements of extracellular and intracellular potentials. It is a very sensitive method that can record electrical charges of as little as one millionth of a volt. It has helped provide invaluable information about the functioning of the primary visual cortex (Eysenck & Keane, 2000) as well as data on the auditory cortex in experimental animals. On the basis of obtained results it was possible to make retinotopic and tonotopic maps of primate brain (Gazzaniga et al., 2002; Ivry & Robertson, 1999). Its main advantage is that it can make very precise recordings of neural activity during different time periods—from very short (one second) to as long as several days. Besides its invasiveness, the fact that it offers only information about the single-cell level makes it unsuitable for studying larger brain
Research Methods
59
areas. More recent techniques make possible the recording of activity of several neurons at a time (multiple-cell recording), which takes care of at least one of the disadvantages. Electroencephalography (EEG) This method is used to continuously measure brain activity through the skull. It reflects the totality of electrical events in the head, including action potentials and postsynaptic potentials generated by neurons, as well as electrical signals from the skin, muscles, blood, and the eyes. It is suitable for determining the functional state of the brain, but the results can hardly be associated with the activity of individual areas. Event-Related Potentials (ERP) In this context we talk of late, so-called cognitive, event-related potentials. The procedure is based on EEG, but the potentials are evoked by a specific stimulus (hence the alternative term evoked potentials) and extracted from the EEG record by signal averaging. Electrodes are attached to the scalp before the procedure. The stimuli are presented and ensuing electrical activity is recorded. Underlying the procedure is the fact that EEG contains all the changes in potential that are time-locked with the specific event, such as stimulus presentation, or movement, that become visible by signal averaging over a large number of these events. In the averaging process, the noise (the uninformative part of the signal) is cancelled out and the relevant signal emerges. Ten to 100 repetitions of the same stimulus are usually sufficient to produce a reliable potential that reflects characteristics of the studied brain and stimulus. Each wave is determined by latency and polarity (positivity or negativity). Evoked potentials recorded in the hemisphere contralateral to the auditory stimulus are generally stronger than those recorded in the ipsilateral hemisphere. It has also provided insights into the time course of processing, as well as detailed analysis of the influence of experimental variables. It is possible to compare the effects of two experimental conditions on early or late components of the response: the early component is taken to represent sensory events and the late component is attributed to cognitive ones. For example, P300 is a positive peak that occurs about 300 ms after the stimulus only if the stimulus has some relevance for the subject. Studies with language stimuli have revealed that N400 (a negative peak occurring approximately 400 ms after the stimulus onset) reflects semantic processes (Hagoort & Brown, 1995). It is possible, however, that a large portion of cellular activity relevant for behavior is not manifested as action potentials. A serious disadvantage of this method is its poor spatial resolution (i.e., impossibility of precise spatial localization of the studied activity), and it is also unsuitable for basic cognitive processes. Its principal advantage is good temporal resolution.
60
The Cognitive Neuroscience of Human Communication
Compared with the early evoked potentials (visual, auditory, somatosensory), cognitive potentials are to a certain extent stimulus-independent. This means: (a) they do not necessarily occur every time a stimulus reaches the threshold, even with intact sensory pathways; and (b) they may occur without stimulation if the stimulus is expected. Different modalities can elicit almost identical cognitive evoked potentials. The response is affected by several factors: probability of stimulus occurrence, psychiatric status, selective attention and arousal, habituation, maturity of the central nervous system, aging, and task relevance, to name just a few (Dabi´c-Jefti´c & Mikula, 1994; Praamstra & Stegeman, 1993; Stenberg, Lindgren, Johansson, Olsson, & Rosen, 2000). The response amplitude may be affected by type of stimulus, and even by such fine differences as noun categories. Müller and Kutas (1996) have found that personal names have the same latency (N1 and P2) but greater response amplitude than common nouns, and that the subject’s own name, compared with other names elicits an additional response with a peak latency of about 400 ms over parieto-central electrodes and another even later response between 500 and 800 ms over left lateral frontal points. A word of caution concerning some inconsistencies in presenting the evoked response test results is necessary. Response latency can be expressed in hundreds or single digits, so P1 designates the first positive peak, and P100 a positive peak of 100 ms latency. They may correspond to one another, but not necessarily. Sometimes, both alternatives are present in the same figure. Single digits are used to mark the peaks of same polarity from 1 on, which means that we can have N1 followed by P1, N2, P2, N3, and so on. Additional confusion may be caused by the fact that some authors turn the negative peaks upward and positive downward, whereas the others do not. Hagoort and Brown (1995) point out, as well as many other authors, that a complete understanding of the neurobiological foundations of language requires the combination of several different methods. Cortical Cartography Recently, new combinations of methods have been introduced. One of these techniques is cortical cartography that combines mapping, event-related potentials and fMRI. It provides three-dimensional images of the skull and brain. Temporal aspects may be recorded by 3D movies. Magnetoencephalography (MEG) Neurons maintain electrical potentials along the external membrane, and when electrical potentials of many neurons are summed up, it is possible to record them encephalographically. These currents generate magnetic fields that may be recorded on the skull surface providing us with a magnetoencephalogram. This method allows precise (to the order of 1 ms) monitoring of the temporal structure of cognitive processes. It is considered a good direct measure of neural activity, as opposed to fMRI or PET, as the latter two methods reflect blood flow, which is an
Research Methods
61
indirect measure of neural activity (Wang, Sakuma, & Kakigi, 2000). The advantage of MEG over encephalography is the insensitivity of magnetic potentials to the cranial bones and other tissues. Although its spatial resolution is superior to that of encephalography, it still provides relatively rough, hence unsatisfactory, anatomic and structural data. Another disadvantage of MEG is possible interference of other, task-independent sources of magnetism. Although MEG is very sensitive to tangential projections from sulcal structures, it is not as sensitive to radial projections from gyral structures (Boatman, personal communication).
RADIOISOTOPIC METHODS Positron Emission Tomography (PET) This technique provides information about metabolic activity of the brain. Metabolism-relevant substances (glucose or oxygen) are marked by a low dose of radiation. The patient is injected with radioactive isotope 15O (tracer), or the isotope is inhaled. Approximately one minute later the radioactive water accumulates in the brain proportional to the local blood flow. During radioactive decay, the tracer emits positrons. As a consequence, an activity performed by the patient (e.g., reading) will reveal parts of the brain that are the most active at that moment. A special apparatus records different levels of radioactivity (i.e., metabolic activity), which results in brain images. The final record is not the absolute level of activity at the moment of task performance, but the result of subtraction of activity patterns at rest from those recorded during a task (similarly to fMRI). Regions with the greatest difference after subtraction are interpreted as areas of highest activity during a function. After 10 minutes no radioactivity can be detected any longer. One advantage of this method is its applicability to large numbers of different tasks, and also its relatively good spatial resolution (3 to 4 mm, or 5 to 10 mm3 of volume). Its disadvantages are technical difficulties in application, high price, and invasiveness in exposing the subjects to some (albeit small) doses of radioactivity, in addition to limited temporal resolution, that can be up to 60 s (Eysenck & Keane, 2000; Posner, 1993, 1995). Cognitive processes take one second or less, so it is obvious that this technique will not reveal everything that goes on during testing. Demonet (1998) suggests that it be combined with the results of responsetime measurements. Interpretation of results is additionally complicated by the fact that even the subjects’ anticipation of what they are about to say changes the final picture of which parts of the nervous system are involved or particularly active. It is also debatable to what extent increased isotope concentration reflects faithfully changes in the neural activity: there are indications that it might be true only for early stages of processing (Eysenck & Keane, 2000). Great intersubject variability precludes attributing specific language processes to limited brain regions (Fromkin, 1995). Again, it needs to be pointed out that, just as with fMRI, for example, increased activity in a certain area during a task does not necessarily mean that the area plays a crucial role in the task at hand.
62
The Cognitive Neuroscience of Human Communication
Single-Photon Emission Computed Tomography (SPECT) This scanning technique uses computerized tomography technology, but instead of gamma rays, individual photons emitted by an externally applied substance are recorded. Its resolution is worse than PETs, but due to its lower price it is more suitable for wider use in clinical practice.
ULTRASOUND METHODS Functional Transcranial Doppler Ultrasonography (fTCD) This method is used to measure increased blood flow related to an event. It has been proven reliable and valid in lateralization studies. The results are comparable to fMRI and Wada test results, and they are replicable (Anneken et al., 2000; Buchinger et al., 2000; Lohmann et al., 2000; Zavoreo & Demarin, 2004).
SUMMARY When evaluating methods of brain activity imaging, one must consider whether the information is represented in the brain at the level of the individual cell or at the level of cell groups. If the former is true, changes in the electrical activity of each individual cell may be associated with the function of the system that is based on it. In the latter case, the situation is much more complex. Another issue is the relevance of the recorded change to the brain. In other words, are the other parts of the brain “aware” of the activity? If they are, how do they interpret this increased activity: as relevant for further action or as a sign of some random and/ or functionally irrelevant activity. Combining several methods may reveal whether the recorded connections are causal or just correlational, possible differences in the timing of activations in different locations in the brain as well as their temporal organization. In tests where the subjects were asked to recognize the objects by feeling them, fMRI revealed that even with their eyes closed their visual areas were activated. Combination with TMS showed that activation of visual representations is necessary for successful recognition by touch (Gazzaniga et al., 2002). Most neuroimaging studies report average results of group studies, and they have to be interpreted in the light of the discussion at the beginning of this chapter (on the advantages and drawbacks of single case studies and group studies). Although general principles of organization and functioning of the nervous system are common to humans, individual variations are more likely than uniformity. Methods involving recording of brain activity may lead to results that are different from the ones obtained by cortical stimulation methods, or by observation of clinical cases. Eysenck and Keane (2000) describe amnesiacs whose hippocampi were key structures for declarative or explicit memory. On the other hand, most PET and fMRI studies failed to replicate the findings of high activation in the same structure in healthy subjects during tasks that rely on declarative memory. One of the reasons may be that PET and fMRI show all areas that are active during a task,
Research Methods
63
not just the crucial ones. Furthermore, damage to a part of the brain will result in functional disorder when the affected region is relevant for a particular function, and it may be that functional reorganization of neighboring areas that might take over has not yet taken place. Furthermore, the subjects’ ages vary widely across studies: PET and fMRI methods involve mostly young healthy subjects, whereas in brain-injured groups there are older subjects. Increased sophistication and sensitivity of activity recording methods is a mixed blessing: on the one hand, it is their great advantage, and on the other, it makes interpretation much more complicated because registering even the smallest levels of activity frequently makes it difficult to decide which of the recorded active locations is really crucial for the function studied. Another problem is the fact that various techniques may be differently sensitive to the many features of the same function and its representation (Eysenck & Keane, 2000; Gordon et al., 1994). In addition to the recommendation to combine different methods, it seems increasingly necessary to include different tasks that examine the same function (e.g., different language tasks studied by several methods). This is known as combined tasks analysis (CTA). Obviously, if a certain structure proves to be relevant for the same function on several different tasks, it will enable us to make more valid inferences about the relationship between structure and function (Ramsey, Sommer, & Kahn, 2000; Rutten, van Rijen, van Veelen, & Ramsey, 2000; Sternberg, 2003).
BEHAVIORAL METHODS Since the brain is the organ of behavior, its functions can be studied by studying behavior, as it reflects psychological processes (Legg, 1989). The subjects are presented with different cognitive, sensory, or motor tasks, their solution is associated with physiological manipulations, and on the basis of the obtained results, functions of particular brain areas are defined. Complex psychological processes may be broken down into stages, and researchers can look for associations between individual stages and corresponding brain areas. Numerous psychological tests include general IQ tests, memory tests, tests of perceptual and motor functions, and various language tests, such as Wechsler’s intelligence scale (WAIS), Raven’s matrices, and so forth. They can be implemented with a healthy population as well as clinical patients. It is important to have neuropsychological assessments for those who have suffered brain injury— regardless of etiology—for the purpose of choosing a therapy and monitoring recovery. However, clinical assessments are limited by the type of damage, in other words, the patients’ ability to understand directions and perform the task. Obviously, someone with a paralyzed right arm as a consequence of stroke will not be able to perform motor tasks that require the use of the right hand. Response time and the percentage of correct answers (or errors) are the most frequently used indirect measures of task difficulty. However, we must keep in mind the limitations of these measures. For example, the time it takes to give a motor response (e.g., raising one’s hand or pressing a button) to a certain cognitive task (e.g., finding the target letter in visual tests, or determining rhyming status in auditory tests),
64
The Cognitive Neuroscience of Human Communication
includes time needed to make the decision and the time needed for the actual motor action. Technical limitations of equipment are, of course, additional factors. The following section includes brief descriptions of the most frequent behavioral experimental methods used in studying language functions. Paper-and-Pencil Tests These are all the tests (including language) in which the tasks are presented graphically (as text or images), and the subjects are expected to respond by writing, drawing, or marking. Word Association Tests These tests are usually designed in a way that the subjects are presented with one word at a time, and are expected to provide the first word that comes to mind. Variations of the test include providing superordinate terms, synonyms, antonyms, additional members of the same category, and so forth. Another subtype of this test is the so-called word fluency test in which the subjects are asked to provide as many words that begin with the target letter or sound as they can in a limited period of time. Stroop Test The original Stroop test involved naming the colors in two ways: as blots of different colors and as words designating colors that were printed in letters of (in)congruous colors (e.g., the word blue printed in blue or red ink). Response times were longer to incongruous letter–color combinations due to the disagreement between the two bits of information. In a wider sense, the Stroop effect may be elicited by other stimulus combinations (Preston & Lambert, 1969). The Wisconsin Card Sorting Test (WCST) Different numbers of symbols varying in color and form are presented on cards. The subject picks up one card at a time and places it on one of the several piles in front of him/her. On the basis of the examiner’s reactions, the subject concludes, by trial-and-error, whether the cards should be sorted according to color, shape, or number of symbols. After 10 successful consecutive attempts at correctly sorting cards, the examiner changes the sorting criterion without warning. The subject is supposed to discover the new criterion. Failure on this test—most frequently manifested as perseveration, or insistence on sorting according to the first adopted criterion—reveals disorders of frontal lobe functions. Priming and Interference In these tests one stimulus is used to facilitate or interfere with another that is related to the first phonologically, syntactically, lexically, or semantically. The
Research Methods
65
degree of facilitation or interference (measured by response times to the second stimulus) offers information about the organization and relationship between the first stimulus (prime) and the second stimulus (target). For example, in a visual task in which a subject is asked whether the presented letter was A or V, the response will be faster if before the target A, an a (prime) rather than v is presented briefly. These experiments may reveal temporal sequence and degree of interference between the two competing cognitive processes or representations of different stimuli, but do not tell us much about the functioning of these processes and representations, or about their interference or facilitation (Westbury, 1998). Shadowing In auditory tests, the subjects are presented with different stimuli to the left and the right earphone. They are asked to listen carefully and repeat simultaneously with the speaker the message heard in one of the ears. When a sudden change is made in the message presented to the unattended ear (an unexpected sound or switch of the speaker from male to female), the subjects typically start to hesitate or lose continuity. This procedure may reveal language deficits even when they are not noticeable in everyday communication. The test may also be visual, in which case two stimuli are presented in different locations in space, or in a bimodal manner (e.g., one stimulus is auditory and the other visual). It has been shown that the success of shadowing may be affected by several factors. For example, meaningful stimuli are more easily shadowed than meaningless ones. Shadowing is more successful if the stimuli are further apart in space, if they are presented at different pitch (e.g., male and female voices), and if they are presented at different rates, and so forth (Coren & Ward, 1989). Gating This method was introduced by Grosjean (1980). A recorded speech segment (typically one sentence) is spliced into shorter segments (usually about 20 ms) by sound-editing computer software. The subjects are presented with stimuli of progressively longer duration (each time one 20-ms segment is added, yielding stimuli that are, for example, 60, 80, 100, and 120 ms long). The subjects’ task is identical at each presentation, or gate: “Say what you heard and/or what you think the entire sentence was.” The responses of all subjects constitute a cohort. The subjects are frequently required to grade the certainty of their response (Erdeljac & Mildner, 1999). The procedure is repeated until the subjects guess the entire sentence, or the recorded material comes to an end. Comparison of the subjects’ responses and the stimuli, in addition to an analysis of their progression, allows inferences about how much and what kind of information is needed to reach the correct response. The number of activated members of the cohort, and the frequency of their occurrence among subjects, as well as the reduction of their number and the final single response are considered indicators of organization, activation, and functioning of the mental lexicon.
66
The Cognitive Neuroscience of Human Communication
Dichotic Listening This is one of the most frequently used behavioral lateralization tests with auditory stimuli. In the original version of the test (Kimura, 1961a, 1961b; 1967) the subject hears through the headphones three numbers presented one after another to the left ear, and simultaneously, three different numbers presented to the right ear. During a very brief pause the subject is asked to report as many of the six numbers as possible. Then another triad is presented and responded to, and so on. Some versions of the test include manipulating attention to one ear or the other (Hugdahl & Anderson, 1986; Hugdahl, Anderson, Asbjornsen, & Dalen, 1990; Mildner, 1999; Mildner & Golubi´c, 2003; Mondor & Bryden, 1992; Obrzut, Boliek, & Obrzut, 1986). Laterality indices, calculated from the subjects’ responses, are taken as indicators of ear advantage, i.e., contralateral hemisphere dominance. Dichotic tests in a wider sense include all auditory tests in which two different stimuli are presented to the two ears at the same time—these can be two signals, or a signal can be presented to one ear and noise to the other. If the untested ear is continuously presented with white noise or murmur, the attention is directed toward the tested ear. The more the distracting stimulus differs from the test stimulus, the greater attention is directed toward the tested ear (Sparks & Geschwind, 1968). For example, if the test stimulus is speech, white noise will direct attention to the tested ear more than if it were speech-based noise (murmur or backward speech). As one channel is loaded more, the response to stimuli in the other one is improved, so the overall response is actually constant (Berlin, 1977). This is equally applicable to healthy subjects and commissurotomized patients. Such tests have shown that the ear contralateral to the speech-dominant hemisphere is superior to the other ear. In a great majority of people the superior ear is right, and the speech-dominant hemisphere is left—phenomena supported by independent sodium amytal (Wada) tests. Different varieties of the dichotic listening tests have revealed right-ear advantage (REA), with a corresponding lefthemisphere dominance for digits, words, nonsense syllables, formant transitions, backward speech, Morse code, difficult rhythms, phonological tone, and many other speech and language-related stimuli. However, test–retest procedures reveal that only about 50% of the subjects exhibit consistent statistically significant REA for dichotically presented speech sounds. Left-ear advantage (LEA), with a corresponding right-hemisphere dominance, was reported for tunes, music chords, environmental sounds, emotional sounds, tones that are processed independently of the language content, pitch perception, and similar stimuli. No ear advantage, or inconsistent ear advantage, was reported for vowels, isolated fricatives, rhythms, nonmelodic humming (Kolb & Whishaw, 1996), and also for linguistic prosody (Mildner, 2004). Ivry and Robertson (1999) report results of studies that claim auditory information is indeed lateralized; however, it is not with respect to ear, but with respect to the side of space from whence the sound comes. Accordingly, each ear may have projections in both hemispheres, but sound input from one side of the space will project to the contralateral hemisphere: the sound coming from the right will
Research Methods
67
be projected to the left hemisphere, and the sound coming from the left will be projected to the right hemisphere, along both ipsi- and contralateral pathways. Divided Visual Field This is one of the most frequently used behavioral lateralization tests with visual stimuli. It is based on the anatomic and physiological specificity of the visual pathways: all stimuli from the left visual field reach the right cerebral hemisphere, and all stimuli from the right visual field reach the left hemisphere. Since the hemispheres communicate via commissural connections, primarily the corpus callosum, visual information from the two hemispheres is combined in a matter of tens of milliseconds. Nevertheless, it is possible to design the experiment so that very brief stimulation of one or the other visual field by instantaneous presentation of stimuli to the left or to the right of the fixation point results in response-time differences depending on whether the stimuli were verbal or nonverbal and whether they were presented to the left or the right visual field. Some 60 to 70% of right-handers exhibit right-visual-field advantage (RVFA), in other words, left-hemisphere dominance for language material but not for nonlanguage (Pinel, 2005; Tadinac-Babi´c, 1994, 1999). Since it takes the eye about 150 to 200 ms to shift from the fixation point, in order to get the lateralization effect the stimuli must be kept shorter than that (Gazzaniga et al., 2002). Dual Tasks These tasks may tell us something about possible common neural substrates of speech and language functioning and motor activity. The classical experiments of Kinsbourne and others (Chang & Hammong, 1987; Cherry & Kee, 1991; Hicks, 1975; Kinsbourne & Cook, 1971; Singh, 1990; Tsunoda, 1975) have shown that the results of simultaneous tasks that rely on the same neural structures differ from those where processing is controlled from different brain regions. The test typically includes two tasks that have to be carried out at the same time, and it attempts to test the relationship between the two tasks (i.e., between their cerebral representations). It is expected that success on one or both tasks will be poorer if performance requires the activation of identical areas. For example, it has been shown in right-handers that language processing has a significantly greater negative effect on the accuracy and speed of simultaneous motor activity of the right hand than of the left, and that the effect becomes larger as the language processing complexity increases (Mildner, 2000). The same study revealed that even designating one of the two tasks as primary, and purposefully directing attention to it, does not necessarily result in its unhindered performance compared with the secondary one. The two tasks may facilitate or inhibit each other—simultaneity does not necessarily have to cause interference. It is not quite clear to what extent the results of dual tasks are affected by attention mechanisms and the fact that they share a common neural substrate. Dubost, Beauchet, Najafi, Aminian, and Mourey (2002) studied the relationship
68
The Cognitive Neuroscience of Human Communication
between walking and two different cognitive tasks—mental arithmetic and speech production. When performed simultaneously, both cognitive tasks slow down the person’s pace and shorten the step, but the walking efficacy is preserved (the subjects’ pace is still even). On the other hand, walking affects each of the cognitive tasks differently. Whereas mental arithmetic (working memory) was not affected by walking, word generation (semantic memory) was facilitated by it (expressed as the greater number of words produced while walking than in the speech production task alone).
SUMMARY Some other lateralization tests look at gesticulations as motor accompaniment and reflection of language functions. Kimura (1973b, 1973c) found that people gesticulate more with the hand contralateral to the language-dominant hemisphere, and that hand gestures accompany linguistic expression. Video recordings of people talking revealed that right-handers make bigger movements with the right side of their mouth than with the left. This was true not only for speech production but also of all complex nonverbal mouth movements. Careful observations have also revealed that right-handers turn their heads and/or eyes to the right while solving verbal problems, but upward and/or to the left when solving numerical or spatial problems (Pinel, 2005). According to some authors, writing posture is indicative of the language-dominant hemisphere, regardless of whether the person is left-handed or right-handed (Levy & Reid, 1976). Namely, if the writing hand wrist is above the finger level, with the hand pointing downward, the language-dominant hemisphere is ipsilateral to the writing hand. If the person writes with the fingers pointing upward (as most people do) the contralateral hemisphere is language-dominant. The problem of studying lateralization by means of behavioral methods is that in healthy subjects, as opposed to the split-brain patients, we cannot be certain whether the response to the lateralized stimulus is the result of processing in the contralateral hemisphere or whether it reflects joint efforts of both hemispheres after callosal transfer. The most reliable results are obtained if different methods are combined, for example, behavioral methods, with noninvasive activity recording methods, especially if they are compared with disordered functions in clinical patients and the functioning of split-brain patients.
APHASIA TEST BATTERIES One large subset of neuropsychological methods for studying speech and language functions are tests for aphasics. These are used to test different levels of language functioning—from the most general division into production and perception to tests that tap into specific levels of processing: phonological, syntactic, semantic, and so forth. One of the most widely known and used is the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983). This battery comprises 34 subtests for 12 language areas, two additional tests of music abilities and seven tests for the assessment of parietal lobes functions. Grammatical-syntactic tests,
Research Methods
69
tests of nonverbal functions, and profile assessment scales are usually added as well. Many other aphasia tests are in use too, but they all assess four basic language skills, all or some of which may be disordered in aphasia: spontaneous speech, repetition, comprehension, and naming (Juncos-Rabadan, 1994; Turdiu, 1990; Vuleti´c, 1987).
Central 5 The Nervous System Principles, Theories and Models of Structure, Development and Functioning On the basis of systematic characteristics, the principles of structure, development, and functioning of the nervous system are established. These principles serve as starting point for the design of theories and models. Empirical and experimental tests and evaluations of theories and models complement existing knowledge. This, in turn, enables us to develop new principles or modify them in such a way that we come to have new models or new versions of the old ones. Due to this obvious interactivity, it seems reasonable to discuss the principles of structure and functioning of the central nervous system in the same chapter with the theories and models that refer to them. The four principles that are discussed in this chapter are hierarchical organization, parallel processing, plasticity, and lateralization of functions.
PRINCIPLES HIERARCHICAL ORGANIZATION One of the most important principles of the nervous system structure and functioning is its hierarchical organization (Fischbach, 1992; Kalat, 1995; Kolb & Whishaw, 1996; Webster, 1995). Higher levels provide greater precision. Neurons respond to increasingly abstract aspects of complex stimuli as the distance, measured by the number of synapses from the source, increases. Examples of this principle are reflexes. They are quick precisely because the reflex arc, also measured by the number of synapses, is short. For instance, the cochleo-stapedial reflex (i.e., the response of the stapedius muscle in the middle ear to high sound intensity) is initiated at the level of the superior olive rather than completing the entire auditory pathway that ends in the auditory cortex. On the other hand, phoneme discrimination takes place in the medial geniculate and cortex after the stimulus has passed through a much longer part of the auditory pathway (cochlear nuclei, olivary complex, inferior colliculi). 71
72
The Cognitive Neuroscience of Human Communication
PARALLEL PROCESSING Another important principle of the functioning of the nervous system is that bits of information do not travel along a single pathway. Various aspects of the same sensation are processed in parallel ways (Fischbach, 1992; Kalat, 1995; Kolb & Whishaw, 1996; Webster, 1995). Zeki (1992) described examples of visual stimuli. He found that in the brain there is a division of labor manifested in anatomically discrete cortical regions and subregions specialized for different visual functions. There are four parallel systems in charge of different visual properties: one for movement, one for color, and two for shape, of which one is closely related to color and the other is independent. These specialized areas are interrelated directly or through other areas. Signal integration in the parietal or temporal lobe proceeds through local circuits. Some of these areas are fully developed at birth (the so-called V1 region—where the entire retinal map is located—is synonymous with the primary visual area in traditional terminology). The development of other areas is completed after birth, which indicates that their maturation depends on experience. Auditory stimuli behave in a similar manner. Sound localization depends on interaural differences in phase and amplitude, and these differences are processed along several paths in three synaptic relays in the brain. Phase differences provide information about the location on the horizontal plane, whereas amplitude differences provide information about the location on the vertical plane. In humans, as little as 3º shift on the horizontal plane (i.e., a temporal difference of only several μs in the activation of the dendrites of the left and right cochlear nuclei) is sufficient to identify the difference in sound source location (Kalat, 1995; Webster, 1995). This type of parallel processing is probably characteristic of other sensory systems, association areas, and motor pathways as well. Primates have about a dozen different cortical maps for visual stimuli, about six for auditory (Lieberman, 1991), and four for the somatosensory ones (Kalat, 1995).
PLASTICITY The third principle of the structure, development, and functioning of the central nervous system—primarily the brain—is its plasticity. This is the ability of the central nervous system to adapt or change under the influence of exogenous or endogenous factors. There are three main types of plasticity: (a) changes in connections that occur after birth, as a consequence of interactions with the environment; (b) changes in connections that occur after lesions; and (c) changes induced by learning and experience. The term developmental plasticity that is often found in literature refers to the first, and partly the third type. Plasticity is closely related to the notion of functional reorganization—a phenomenon referring to the change in (cerebral) representation of a function or sensory modality. The change may be caused by lesions or natural aging processes, but also by practice and learning. Functional reorganization may be manifested as a change in the size of the cortical area where the particular function or sensory organ is represented, or as a different location of representations. For example, if one finger is amputated, the neighboring fingers will be represented on a greater area than before, at the
The Central Nervous System
73
expense of the amputated finger. Shifting of speech representation to the opposite hemisphere is sometimes found in patients with brain injuries in the speech-dominant hemisphere. Without plasticity, functional reorganization would be impossible. This topic is discussed in more detail in chapter 7, this volume.
LATERALIZATION OF FUNCTIONS The fourth important principle of brain functioning is lateralization of functions. Since the brain appears to be anatomically symmetrical, early works on the biological foundations of behavior assumed that functions were symmetrically organized as well. However, more detailed studies have revealed that the brain is not exactly symmetrical and that the two hemispheres function differently; in other words, they are both anatomically and functionally asymmetrical. This was supported by findings that lesions to homotopic regions in the left and the right hemisphere have different consequences for behavior and functioning in general. Hence the term lateralization of functions. After many repeated unsuccessful attempts to associate the site of lesion with a function disorder, the localizationist approach of the late 18th century was replaced by a holistic one—understanding and describing cerebral functions as a process in which each behavior requires interaction of many different structures as a whole. Therefore, the first attempts to relate language deficits to left-hemisphere damage went largely unnoticed. Broca emphasized the difference between anterior and posterior injuries, but only as Wernicke’s findings on the importance of the left hemisphere for language functioning emerged. As a consequence, the left hemisphere was called the dominant or principal hemisphere, whereas the right hemisphere was treated as less important or minor. In this volume, as indeed in most contemporary literature, the term dominant may refer equally to both hemispheres. It is used in the sense of being primarily involved in the processing of a function, not in the sense of being more important in general. The few authors who studied right-hemisphere functions at the turn of the 19th century (e.g., Hughlings-Jackson and Balint) found that damage to that hemisphere most frequently resulted in deficits in spatial functioning (perception of objects in space and orientation in space). This was the foundation of the propositions that language and spatial functioning are two basic human cognitive abilities, each represented in a hemisphere of its own. Primary sensory processing and motor functioning are bilaterally represented. Lateralization occurs in higher, more complex functions, such as language, spatial processing, music, and so forth, so that at the highest levels of processing the two hemispheres have different roles. Lateralization of function is discussed in more detail in a separate chapter.
THEORIES AND MODELS The brain consists of numerous cortical areas and subcortical nuclei that differ in afferent and efferent connections, appearance under the microscope, and
74
The Cognitive Neuroscience of Human Communication
characteristics of their physiological reactions. It is, therefore, logical that their contributions to psychological processes are different as well. Although phrenology turned out to be unacceptable, anatomical studies have shown that some areas have direct connections with the motor system, others with the sensory, and still others are to be found between motor and sensory areas. Accordingly, some areas are closely connected with a particular sensory modality, others with movements, whereas others exhibit complex response properties that are neither exclusively motor, nor solely sensory. The parts that are most closely connected with specific functions are those having the closest connections with the sensory organs. Experimental and clinical data have shown that damage to different parts of the brain may have considerable consequences for behavior and cognitive functions. Theories and models of central nervous system functioning, especially of the brain, attempt to address several key issues: (a) where does a particular function take place? (b) What is its course? and (c) how autonomous is it? The processing may proceed in parallel (more aspects or levels are processed at the same time) or serially (one at a time); processing may be one-way (bottom-up or top-down) or there may be more or less interaction (between levels or within one level), with total or partial autonomy of processing at each level. Processing may go on in separate and specific brain structures or it may be more diffuse. Active theories posit the existence of very complex transformations of neural information and programs that may be compared, and on the basis of that comparison perceptive decisions may be made. Each level is connected to other levels and processing proceeds in a parallel manner in all of them—there is no necessary order of levels of processing (e.g., acoustic material serves as a basis for certain hypotheses, but at the same time as a test of the existing ones). Passive theories are as a rule one-way—processing is only bottom-up or top-down, with no interaction among levels. Obviously, each model may belong to several categories (e.g., localistic and serial). Furthermore, not all models deal with all or the same components of language functioning—some are limited to language production, others to perception or word recognition. In this chapter, I discuss some of the most influential and most widely quoted general theories and models of structure and functioning of the central nervous system. This includes the application to language functioning, as well as some specifically linguistic theories.
PARALLEL OR SERIAL PROCESSING? The first models of sensory system functioning were serial. According to them, the information flowed between components, but only through a single channel. Later, it became acceptable to talk about parallel models that were believed to represent sensory functions better than the serial ones. In parallel models, the information may be relayed among components through different channels. Although there is ample evidence for parallel processing in sensory systems, the nature of language processing is not so easily agreed upon (Gazzaniga, Ivry, & Mangun, 2002). Examples of serial processing are Levelt’s (1989) model of
The Central Nervous System
75
speech production and the Wernicke–Geschwind model of language functioning. These models are discussed in more detail later. Serial or cascade models were supported and discussed, among others, by Bentin, Mouchetant-Rostaing, Giard, Echallier, and Pernier (1999) and Schmitt, Muente, and Kutas (2000). On the other hand, Starreveld (2000) does not agree with the serial model of speech production.
LOCALISTIC MODELS A principal characteristic of localistic models is establishing direct connections between structure, condition, and functioning of a particular brain region with behavior. According to these models, focal injury to some brain region will result in specific patterns of cognitive deficits, because brain functions are assumed to be localized. An obvious drawback of these models is the inability to deal with cases of disordered functions without damage to “responsible” brain regions, and with cases of preserved function in spite of damage to corresponding regions. The models are acceptable to the extent that it has been shown that particular functions are in fact localized within specific brain regions, with the higher functions (e.g., attention and thought) spread over greater areas of the cortex and including subcortical parts. Some advocates of these models were Broca, who localized the speech production center in the left frontal lobe; Wernicke, who localized the speech perception center in the left temporal lobe; and Hughlings–Jackson, who associated motor activity with the posterior parts of the frontal lobe, on the basis of postmortem examinations of his epileptic patients. One of the first models was Lichtheim’s in 1885. It consisted of three main parts and connections among them. They were: (a) Wernicke’s area with the phonological store, (b) Broca’s area with programs for planning and execution of speech, and (c) an area where concepts were stored. Lichtheim’s was a classic localizationist approach, elaborated later by Geschwind and Galaburda (Galaburda, 1994; Gazzaniga et al., 2002). On the basis of the properties of the Hebbian synapse, Pulvermüller (1992) calls functional neuronal groups cell assemblies. Turner and Frackowiak (1995) call these restricted areas cortical fields (each field comprises about 106 to 107 neurons) and consider them to be the substrates of specific processes. However, these cannot be counted as true representatives of localistic theories because, in essence, their approach relies on neural networks. For some 30 years now, all discussions about localization of language functions have included the Wernicke–Geschwind model. In spite of its inconsistencies and imperfections, over the years the model has instigated research and discussions about cortical representation of language. Before the model was proposed, there had been attempts to describe localization and lateralization of language functions on the basis of deficits in language functions (chap. 3 on the history of neurolinguistics). These localistic theories lost ground at the beginning of the 20th century to holistic theories that see the brain functions as a whole. Geschwind incorporated these ideas into the so-called Wernicke–Geschwind model based on a theory stemming from the belief that damage to language areas in the left hemisphere and/or connections between them caused aphasia and other language disorders.
76
The Cognitive Neuroscience of Human Communication Primary motor area
4 8
3 1 2
6
5
7
Angular gyrus 19
Broca’s area
10
41
9
46 45
18
44
47
22
11
21 38 Fasciculus arcuatus
FIGURE 5.1.
39
17
42
Primary visual area
37 Wernicke’s area
20 Primary auditory area
Wernicke–Geschwind model of speech and language.
Wernicke–Geschwind Model The Wernicke–Geschwind model consists of seven components: primary visual cortex, primary auditory cortex, primary motor cortex, angular gyrus, arcuate fasciculus, Wernicke’s area, and Broca’s area. All the components are located in the left hemisphere (primarily in the precentral and the postcentral areas, immediately anterior and posterior to the central sulcus, where face representations are located). The model is shown in Figure 5.1. When we read aloud, the signal arriving in the primary visual cortex is relayed to the angular gyrus, which transforms the visual form of the word into its auditory code and transfers it to Wernicke’s area where comprehension takes place. Wernicke’s area initiates appropriate responses in the arcuate fasciculus. In Broca’s area the signal activates the appropriate articulation program that activates neurons of face and mouth in the primary motor cortex, and finally, muscles that participate in word articulation. This model belongs to the group of localistic and serial models, because it involves a series of responses that occur in linear succession, and this is one of its most serious drawbacks. Any model that involves any number of highly localized cortical centers connected linearly, is not a true description of actual brain functioning and cannot be applied to complex cognitive processes. Broca’s and Wernicke’s aphasias are, in fact, primarily but not exclusively expressive and sensory, respectively. Moreover, it has been found that these types of aphasia do not necessarily ensue from damage to Broca’s and Wernicke’s areas. The model was not supported by the examples of patients who had, for medical reasons, undergone surgical removal of parts of the cortex that are supposedly
The Central Nervous System
77
components of the model. For example, if the entire Broca’s area was removed, but with little or no surrounding tissue, the patients had no speech difficulties. Similar examples from clinical practice may be found for other components of the model as well. The advocates of the model explain this phenomenon by the fact that the patients had longstanding neurological disorders before the surgical procedure, which might have resulted in some functional reorganization prior to surgery. However, the model is not supported by cases of sudden brain injury caused by trauma or illness. Only extensive damage involving three lobes caused difficulties proposed by the model (Pinel, 2005). It has also been consistently found that extensive damage to the anterior regions is associated with articulation problems as opposed to equally extensive damage to posterior regions. CAT scans have shown that damage limited to Broca’s and Wernicke’s areas does not cause language problems, and that in all patients presenting with language difficulties there had been some subcortical damage. All studies confirmed a greater probability that extensive damage to left anterior parts would cause difficulties in language expression, as opposed to damage to posterior parts, and that comprehension difficulties would be caused by extensive posterior damage, rather than by anterior damage. Additional explanations were supplied by Kimura (1983; see also chapter 2, this volume) and Hier, Yoon, Mohr, Price, and Wolf (1994). Cortical stimulation of large (widely distributed) numbers of sites in the left hemisphere cause speech arrest or selective speech deficits (inability to name objects, incorrect naming, counting, etc.). Aphasic symptoms occurred very rarely in response to right-hemisphere stimulation. Contrary to the model’s predictions, cortical areas that were affected by stimulation in terms of language functions were found well outside the language areas defined by the model. Another important result emerged from these studies: individual differences were substantial among subjects in the organization of language abilities. Ojemann (as cited in Pinel, 2005) proposed that the language cortex is organized like a mosaic, in which columns of tissue responsible for a certain function are widely distributed throughout the entire cortical language area. He also concluded, on the basis of results from a series of recording and stimulation studies (Ojemann, 1994), that there were separate systems of neurons and basic areas for additional language dimensions. Different components of each system obviously work in parallel. The neurobiological substrate of language probably involves (mainly) separate processing of each language dimension, but with the systems related to each dimension active at the same time. Human language, like so many other functions in the primate cortex, is probably processed in multiple parallel distributed systems. Scanning techniques have revealed increased activity during speech in the left frontal, temporal and parietal lobes, as well as in the left thalamus and basal ganglia (Kalat, 1995). Gordon et al. (1994) present evidence that confirm the existence of fine subdivisions in representation of language and related functions and conclude that a considerable number of these subdivisions may be controlled by higher mechanisms. Evidence may be found in the speech perception system, as well as in verbal and visual semantic systems and their connections to other language subsystems,
78
The Cognitive Neuroscience of Human Communication
including phonology and its connections. Global regulation and coordination is supported by the same data about semantic processing difficulties. With respect to neuroanatomic connections, cortical stimulation maps differ from the classical language function maps. First, Broca’s and Wernicke’s areas are probably spatially more circumscribed for any individual than is usually shown. Second, other areas crucial for language may be identified more reliably, for example, those related to the posterior connections between the temporal and occipital lobe. As expected, resection of the speech cortex can cause acute deficits, but recovery is quick—within several days or weeks. Such speedy recovery of even those skills that had taken longer to acquire (e.g., reading) suggests that, in normal situations, there are multiple but latent copies of language functions and representations (Gordon et al., 1994). The serial Wernicke–Geschwind model is challenged by the cases of deficits in reading aloud. According to the so-called dual-route models proposing that the reading process is in fact mediated by two different pathways of neural activity, there are two different systems of reading aloud. One is the lexical procedure, which is based on the information about pronunciation of written words stored in our vocabulary. The other one is the nonlexical procedure, which is based on general rules of pronunciation that enable us to read unfamiliar and nonsense words. Patients who have trouble with one procedure but not with the other are taken as strong support for these models. For example, in surface dyslexia, the nonlexical procedure is intact, but the patients have considerable difficulty reading words that require unusual pronunciation (impaired lexical procedure) and most frequently pronounce words in accordance with the rules (e.g., reading /ei/ instead of /æ/ in have as if the word rhymed with cave). With impaired nonlexical procedure but intact lexical procedure, the patients are typically unable to pronounce nonwords, but their ability to read real words is spared. PET scans have shown that looking at words activates only the occipital lobe with no additional activation in Wernicke’s area or in the angular gyrus; the frontal lobe processes spoken and written language in the same way (Pinel, 2005). Furthermore, the activity recorded in the right hemisphere during these studies is not predicted by the model. Obviously, these results cast serious doubt over any serial model of language processing. The problems with the Wernicke–Gerschwind model, as indeed with any localistic model, may be summarized in the following points: (a) localization of language system parts is identical in all healthy subjects; (b) specific functions of subregions of the language cortex are derived from the connections of these subregions with the motor and sensory cortical regions. These features are problematic, because they do not take into consideration the nature of many components of language processing, and are in disagreement with the evidence of variability in language function localization (Caplan, 1988). Despite the problems we have discussed with the Wernicke–Geschwind model, in combination with the observed language difficulties in brain-damaged people, it can provide sufficient information about the general location of damage.
The Central Nervous System
79
HIERARCHICAL MODELS These models are based on the principle of hierarchical organization primarily of sensory systems, such as visual, tactile, auditory, olfactory, and gustatory. The main path goes from each sensory organ to the thalamus, a group of nuclei located at the top of the brainstem. Nuclei responsible for individual sensory modalities relay the majority of the arriving information from the sensory organ to the limited part of the neocortex that is the primary sensory cortex for that particular sensory system. The complexity of coding increases with proximity to the neocortex. A large portion of outputs from each area of the primary sensory cortex are relayed to the neighboring cortical area—the secondary sensory cortex for that sensory modality. According to the traditional model, the association cortex is an area of cortical tissue not yet entirely understood that is the final destination of the sensations, loosely defined. It is believed that association cortex connects activities of different sensory systems, that it transforms the sensory input into the programs necessary for motor output, and that it mediates complex cognitive activities, such as reasoning and memory. There are multiple representations of sensory information in visual, auditory and somatosensory systems. At the highest levels, the functions are lateralized. Injuries to primary visual, auditory, and somatosensory systems cause serious deficits in sensory discrimination, which are manifested as significantly reduced sensory acuity. When secondary regions are damaged, acuity is frequently preserved, but not the ability to recognize objects, shapes, or patterns. It is assumed that the hierarchical stages are closely connected. A generally accepted view is that structures at increasingly higher levels of the sensory system have a less and less sensory and more and more perceptive role. This is based on the notion that each level of the system analyzes the input from the previous level, adding to the complexity of analysis. The Triune Brain According to MacLean (as cited in Spreen et al., 1984), the mammalian brain may be anatomically and conceptually divided into three hierarchical systems that make up a whole. The first system comprises upper parts of the spinal cord, the midbrain, the diencephalon, and the basal ganglia. On the evolutionary scale this system corresponds to the lowest developmental level. This part plays a crucial role in instinctive activities necessary for survival of the individual and the species. The second part corresponds to the limbic system and is on a higher evolutionary level. It is involved in the integration of emotional expressions and self-awareness, and it may inhibit and suppress the first part. The third part corresponds to the cortex, and according to MacLean, it is responsible for nonemotional, integrative, detailed analysis of the external world. In humans, these properties are represented by language in the left hemisphere, enabling us to reason and think about future actions. The third part may override the first two and, in terms of evolution, it is on the highest level. Such a division corresponds to the ontogenetic development of the central nervous system. Van der Vlugt (1979;
80
The Cognitive Neuroscience of Human Communication
cited in Spreen, 1984) divides this ontogenetic development into three stages as well, with each developmental stage depending on previous ones, and each subsequent structure having increasingly more refined and more complex adaptive and integrative functions. Luria’s Model of Functional Systems Luria (1973, 1982) worked out three functional units of the brain that are responsible for reception, interpretation, and use of sensory information. These units act serially, producing behavior. Any weakness in any of the structures may affect processing in another. The First Functional Unit The arousal unit (upper and lower parts of the brainstem, reticular formation and hippocampus) is primarily responsible for arousal level, attention, motivation, and reasoning or impulsiveness. Inappropriate arousal or attention may have a negative impact on information processing that takes place in the second unit, which in turn may influence planning performed by the third unit. The Second Functional Unit The sensory (input) unit (posterior cortical areas, including the occipital, temporal, and parietal lobes with the accompanying projection zones) is responsible for reception, processing, and storing of information. In this unit the information is processed (in parallel or serially) and coded; it is also responsible for integration of information from different sensory modalities. This unit is, to some extent, involved in every task. The Third Functional Unit The motor unit (for output and planning, located in the frontal lobe and associated areas) is responsible for planning and programming of behavior, as well as for actual performance. This unit depends on the efficient functioning of other units and is probably in highest correlation with intelligence. It is the highest functional level of the brain. Sensory and motor units may be further divided into primary, secondary, and tertiary areas, which represent levels of increasing complexity and integration in information processing. The first functional unit may be identified with selective attention; the second with sensory data, information processing, motor responses, and cerebral dominance; and the third unit may be identified with planning, organization, and higher-order language functioning. It should be kept in mind that several functional units in the brain participate in production of behavior; there is no direct one-to-one correspondence between a cerebral area and particular behavior. Luria related his theory to ontogenetic development of brain and behavior. Secondary zones cannot develop before the primary zones have fully developed. Only after the secondary (gnostic) zones have been developed can higher cortical zones become operative. The tertiary zones become dominant in adults.
The Central Nervous System
81
This hierarchical development of the three zones is mirrored in the development from the maximally modal specificity of the primary zones to the supramodal organizational and interpretative functions of the tertiary zones. This means that the primary areas are maximally modality-specific, whereas the tertiary areas are associative and integrative across sensory modalities. The development entails progressive lateralization of functions as well: left and the right primary areas have identical roles in an individual’s functioning, but hand dominance and language development require functional organization and specificities that develop concurrently with the secondary and tertiary areas and are radically different in the two hemispheres. Performance at any level does not predict the quality of performance at subsequent levels. Since psychological functions dependent on tertiary zones develop in late childhood or adolescence, clear differences between children and adults in this respect can be expected. Luria reserved a special place for the role of language in a child’s development. Newborns have only the basic ability to express themselves by movement, crying or facial expressions (the first signal system), but older children begin to complement their actions with verbal expressions and eventually develop the second signal system. With the maturation of the tertiary cerebral zones (according to Luria, occurring around age 6), language becomes more sophisticated, and accompanying motor actions are no longer necessary (this does not pertain to normal hand gesturing accompanying speech). Language does not have to be overt any longer—it becomes internalized, and it mediates and directs behavior. Kolb and Whishaw (1996) address several drawbacks of Luria’s model. First, it is practically impossible to set a clear distinction between sensory and motor functions at any level of the nervous system, particularly when it comes to the cerebral cortex. Besides, the model does not predict the possibility of different perceptions of the same object under changed circumstances. Another difficulty is that the model does not allow the possibility of information bypassing the frontal lobe or primary motor cortex. Finally, it does not predict parallel sensorymotor systems, for the existence of which there is anatomical evidence. One of the hierarchical models of perceptual speech analysis was proposed by Chistovich (1984). Her model posits the existence of hierarchically organized stages of perceptual analysis—auditory, phonetic, morphological, syntactic, and semantic. The result of analysis at one level is not final—at the higher level the ambiguous decision may be resolved. Each level operates on its units: at the morphological level decisions cannot be made about phonemes, for example, but only about morphemes, yet the higher levels may influence the outcome of the lower levels. Processing is parallel (at several levels simultaneously: morphological, phonological) and interactive (mutual influence of different levels). Some sort of normalization mechanism is assumed that takes care of stimuli arriving from different speakers. The model also posits that perceptual decisions are formed on the basis of (auditory) feature detectors. Perception starts at the subphonemic level because a decision is reached before the entire phonemic segment has been produced.
82
The Cognitive Neuroscience of Human Communication
Jurgens’ Model of Neural Vocalization Control Jurgens (as cited in Kent & Tjaden, 1997) proposed a three-subsystem model of neural vocalization control, based on the studies of human and animal vocalizations. Subsystem I This is primarily responsible for coordination of breathing, movements of the larynx and upper respiratory pathways. It includes reticular formation, and motor and sensory nuclei of the brainstem and spinal cord responsible for phonation. Apart from coordinating muscle activity, this subsystem may have subroutines necessary for genetically programmed vocalization patterns. Subsystem II This initiates the vocal expression and selects among the existing vocalization patterns. It is hierarchically organized into three levels: • Lowest level—periaqueductal gray and the surrounding tegmentum in the midbrain. This structure plays an important role in vocal expression of emotions. • Middle level—subcortical limbic structures including the hypothalamus, medial part of the thalamus, amygdala, etc. These same structures are involved in regulating emotions. Their role is integration of external stimuli with the internal motivational ones. • Highest level—anterior cingulate—the role of this structure is to maintain control of emotional states. Subsystem III This comprises the motor cortex and its primary input–output structures, primary somatosensory cortex, cerebellum, ventrolateral part of the thalamus, putamen, nucleus ruber, and the pyramidal tract. This subsystem executes fine voluntary control of vocalization patterns. Obviously, this is also a hierarchical model. Kent and Tjaden (1997) proposed a somewhat more elaborate variant of the model. Their model also has three main subsystems, but they have modified Jurgens’ subsystem III so that it reflects a greater number of connections and specific functions of some structures. It includes more inputs to the motor cortex, particularly from premotor areas. Basically, their model predicts parallel connections and processing.
MODULAR MODELS Legg (1989) claims that the brain owes its resistance to damage to its modular structure. As opposed to the interactive principle of functioning, in which damage to any part causes collapse of the entire system (because the subregions are connected to each other), modular structure is such that groups of components are
The Central Nervous System
83
functionally isolated from each other. Accordingly, if one group is damaged, the remaining ones will continue to function normally. Modularity can be achieved in two ways: (a) functionally independent modules are created so that each of them performs a different function (complementary system); or (b) several identical modules exist, each of which is capable of performing equal functions (redundant system). Processing is exclusively bottom-up. A higher level cannot be reached before completing the lower one. Modularity has found support in the fact that brain damage results in very specific consequences limited to the damaged area. Specificity does not only imply that a function is lost or weakened by damage, but also that the part survives lesions to other parts that result in different consequences. Legg (1989) concludes that the brain functions on the principle of the redundant modular system (because if it were interactive, it would be much more susceptible to damage), and that the only operative difference between the interactive and the redundant modular system is the degree of dysfunction resulting from damage to individual components. According to Fodor’s (1983) modular model of the mind language does not belong to the central system. It is an input system, and input systems have modular structure. This modular structure is determined by several features. First, the input system receives information from several sensory systems, but it processes them in particular ways. For example, during reading the language input system translates information from the visual modality to the phonological or speech representation modality. Second, processing is one-way, bottom-up, which means that higher cognitive modules do not affect the lower ones (e.g., sentence context does not affect phonemic processing). In cases of ambiguities, both (or all) word meanings are activated and forwarded to higher levels, and ambiguity is resolved by context only after both (or all) meanings have reached the level of context. Third, each module has its place in a particular region of the brain. Hence, modular models are at the same time predominantly localistic. During speech production, actual speech output is preceded by several preparatory stages that are superordinate in terms of cognitive level. One of the most frequently quoted models in the past 15 years is Levelt’s (1989; Levelt, Roelofs, & Meyer, 1999). According to his theory, word preparation is accomplished through the stages of conceptual preparation, lexical selection, morphological and phonological encoding, as well as phonetic encoding, which is followed by articulation. In the process, the result (i.e., the language product—speech) is monitored by means of the speaker’s speech comprehension mechanism. The model includes macroplanning (a decision about what we want to convey to the listener) and microplanning (choosing words and defining their roles). The output of the planning stage is the conceptual message that is forwarded to the formulator, which supplies the message with appropriate grammatical and phonological form (grammatical encoding). During the process, the surface structure of the message is computed. In this surface structure, lemmas are the lowest-level elements. Lemmas are stored in the mental lexicon, and they contain all syntactic and semantic characteristics of the word. Since the concepts are organized with
84
The Cognitive Neuroscience of Human Communication
respect to meaning, in a task where the subjects are required to name an object in the picture, activation of one concept will activate semantically related concepts, and the lemma eventually selected will belong to the term that corresponds to the picture (lexical selection). The lemma then activates the lexeme (word form) with its phonological and phonotactic data, followed by appropriate phonetic encoding based on the data, resulting in the activation of the sound form of the word. Articulation is the final process that results in the spoken word by mapping phonetic gestures to articulators. In summary, the model essentially consists of the following six stages: conceptual preparation (nlexical concepts), lexical selection (nlemma), morphological encoding (nbasic word form), phonological encoding (n syllabification), phonetic encoding (n preparation of speech sounds), articulation (nspoken word) (Eysenck & Keane, 2000). Since this model is modular and serial, phonological processing cannot take place before the lemma has been chosen. Therefore, phonological information has no effect on lemma selection. Auditory-perceptive self-monitoring is the only form, albeit very limited, of feedback (Levelt et al., 1999). The model is illustrated in Figure 5.2. Some ERP studies have confirmed that lemma selection precedes phonological activation at the level of the lexeme (Gazzaniga et al., 2002). Different approaches to studying language functions and models of language function organization more or less confirm that language is modular in the sense that it is autonomous with respect to other higher cortical functions, and that it involves submodules (e.g., syntax, lexicon) that are functionally autonomous. This autonomy, however, cannot be interpreted as exclusively oneway processing or the existence of separate neurological structures for individual submodules, but only as relatively independent functioning (Blumstein, 1988).
CASCADE MODELS These models are similar to modular ones. They include different, but mutually connected neural modules, each of which is responsible for a different level of processing of word information or some other data. In terms of word recognition, they propose that lexical access is autonomous, independent of higher levels, but that lexical selection may be equally affected by the sensory (bottom-up) and by contextual (top-down) levels. Examples of serial or cascade models are supplied by Bentin et al. (1999) and Schmitt et al. (2000). They report data from studies in which subjects were required to name pictures in two concurrent tasks. The semantic task was to decide whether the picture depicted an object or an animal; the phonological task was to decide whether the name of the object starts with a vowel or a consonant. They claim that semantic processing starts before phonological.
The Central Nervous System
Conceptualizer
85
Concepts Lexical concept Lexical selection Lemma Morphological encoding
Formulator
Mental lexicon
Morpheme Self-monitoring Phonological encoding syllabification Phonological word Phonetic encoding
Syllabary
Phonetic gesture
Articulator
Articulation Sound wave
FIGURE 5.2.
Levelt’s model of speech production.
INTERACTIVE MODELS Contrary to modular models, interactive models propose some degree of influence of one processing level on another (both toward the higher and toward the lower levels), and a combination of top-down and bottom-up processing. According to interactive models, phonological activation starts immediately after the beginning of semantic and syntactic activation and proceeds simultaneously. Not only can all types of information participate in word recognition, but context may affect sensory data even before they become available, by changing the activation status of the word form. According to these models ambiguity is not possible, because only those words will be activated that are in harmony with the familiar
86
The Cognitive Neuroscience of Human Communication
context. Most of these models do not predict the existence of words as readymade forms, but a particular word is recognized if within the neural network circuit certain nodes are activated simultaneously (Eysenck & Keane, 2000; Gazzaniga et al., 2002; Pinel, 2005). The models posit some feedback among levels, but also among elements within each level. Two types of processing are at work here: (a) data-driven processing (bottom-up) that starts with input data; (b) concept-driven processing (top-down) that starts with thought processes. These are active models, because they predict active cooperation between different levels taking into consideration context, expectations and attention, as opposed to the passive models in which processing is one way—only bottom-up. The interactivity assumes, naturally, simultaneous processing at several levels, but also some degree of hierarchy. A large number of models involve some measure of distinctive feature analysis at lower levels, which may be reflected in the name of the theory or model or just in its description. In auditory perception, the process starts from the bottom and proceeds upward. At the auditory level, speech segments are not separated, but acoustic analysis takes place (for more details see chap. 1). Processing continues through the phonetic level (where decisions are made about the pronounced sound), and on to the phonological level (where the results of phonetic analysis are compared and coordinated with the specificities of the language at hand). Models and their diagrams vary in the amount of detail within the basic levels, but common to them all, even when it is not explicitly specified, is comparison of the input signal with some already existing patterns (template matching).
CONNECTIONIST MODELS Classical connectionist models are those in which one processing continues after another. In that sense the components are related, but they are still predominantly localistic and serial models. An example is the Wernicke–Geschwind model discussed previously, but also the much older model by Lichtheim (discussed in this chapter). More recently, the connectionist approach assumes interactivity and parallel processing, as well as wide spatial distribution, due to which these models are said to be based on parallel distributed processing. Contrary to the localistic models, the connectionist models assume widely represented functions and connections among all segments of processing. The first computeraided connectionist models were designed by McClelland and Rumelhart (1986). Since they include basic units or nodes mutually connected into networks, they are most frequently referred to as network models or neural network models. The first discussions in which neural networks were mentioned date back to the 1940s (Lon´cari´c, 2002).
NEURAL NETWORKS Many studies have shown that hardly any function can be reliably localized in a particular spot, that performance of a cognitive task (e.g., speech) activates
The Central Nervous System
87 Output layer (units)
Hidden layer (units)
Input layer (units)
FIGURE 5.3.
Example of a simple neural network.
simultaneously several separate cerebral areas, and that stimulation of a limited area may elicit any number of different responses. Language and speech functioning can therefore be interpreted as coordinated activity of more different assemblies within a single unitary network (cortical or neural). As it has already been mentioned previously, it is believed that basic functions are localized in specific locations in the cerebral cortex, whereas complex functions involve parallel processing of information in widely distributed networks that spread over cortical and subcortical structures. Neural network models consist of three layers: the input layer (units), the hidden layer (units), and the output layer (units). The units (that actually represent neurons) are connected and these connections facilitate or inhibit (increase or decrease) activation levels of the specific reception unit. Each unit receives input from the preceding layer and relays its signals to the next layer (Figure 5.3). Each input has a value—weight. The activation level of each unit depends on the weighted sum of all inputs. Hidden units provide information necessary for correct mapping of the input layer to the output layer by changing the strength of connections between units. This is accomplished by developing a set of rules for learning, in other words, a quantitative description of necessary changes in processing, depending on the final outcome. During the practice phase, neural networks “learn” to respond to given inputs with a particular output, by modifying
88
The Cognitive Neuroscience of Human Communication
the connections between units, so that each actual output is compared with the target output, and in case of discrepancies the weights are changed. The rule for weight adjustment is called back propagation algorithm (back prop), because the error message is relayed through the network in the opposite direction—from output units (back) to input units. Rumelhart (1986) called this procedure supervised learning (as cited in Ainsworth, 1997; Gazzaniga et al., 2002). A trained network, in fact, resembles natural systems: it generalizes responses to the patterns that had not been explicitly used during practice, and its efficacy gradually decreases as individual units are eliminated. When we say that many areas function as a network, it means that they consist of mutually highly interconnected sets of neurons that function as units during a higher cortical activity. Neural networks are adaptable; they can learn to differentiate among patterns of input stimuli and to respond to each one of them in an appropriate manner. Information in the neural network is distributed, and each cell contributes to the recognition of every pattern. Particular cells do, however, respond selectively to specific patterns, but there are many of them for each type, and they are distributed throughout the network. This means that complex functions are represented within mutually interconnected areas that together make an assembly. Several conclusions may be drawn: (a) each individual cerebral area has the neural basis for several different types of behavior, and it can be part of several assemblies; (b) damage to one area will probably result in multiple deficits of some function; and (c) different aspects of the same complex function may be impaired depending on which of the many cortical areas and their connections have been injured. It can also be said that the brain consists of a number of smaller areas that function as overlapping neural networks. Network models that include programmed cell death (apoptosis) are even closer to the actual functioning of the brain. Such models have shown that artificial neural networks may successfully solve difficult problems if there are more neurons at the beginning than will be eventually needed. The least useful neurons die away gradually after learning has taken place, and the learned solution can be preserved even with a smaller number of units than was necessary for initial learning—this is obviously similar to the living brain (Brown, Hulme, Hyland, & Mitchell, 1994). Although there are recurrent networks as well as feed-forward networks, speech and language functioning requires a model that predicts a combination of bottom-up and top-down processing. Damage to the network causes overall inefficacy without specific deficits. The actual behavior of the network depends on the extent of damage: after mild damage the remaining cells may take over some of the function affected by damage; more serious damage makes the connections within the network sparse. The cells within the network recognize and respond to input patterns by mutual excitation and inhibition. As a consequence, damaged networks react more slowly, and it is possible that the normal excitation threshold is never reached, which results in malfunction (Deane, 1992). It seems that impulses relayed by neural pathways to the higher cortical areas code information digitally: During each functional state some neuronal fibers are
The Central Nervous System
89
active while others are at rest. Neural networks are more resistant to localized damage than single neuronal tracts. According to the model of neural networks connected by nerve tracts, the brain functions as a series of parallel processors that interact by means of information channels that integrate them into a whole (Deane, 1992). In short, networks have great data processing capabilities. They are resistant to damage, quick and adaptable, which makes them a good model of brain functioning (Lieberman, 1991). Neural network models can engage in some highly sophisticated cognitive processes. Many consequences of brain damage resemble closely what one would get by damaging a network system (be it individual neurons, or connections among them). Seidenberg and McClelland (1989) constructed a language acquisition network. By switching off individual units of the network they simulated brain damage, which resulted in their network behaving as a dyslexic. In studies of speech production, neural network models have been used to simulate articulatory movements on the basis of the data obtained by electromyographic recordings (e.g., the relationships among different articulator positions, speed, acceleration, etc.), or on the basis of experimental results about the influence of changes in the rate of speech and phonetic environment on the kinematics of speech movements (Löfqvist, 1997). On the basis of their PET results, Friston et al. (as cited in Kent & Tjaden, 1997) proposed a model of word production based on the neural network models. Their model comprises three neural systems: (a) a distributed word representation store; (b) an afferent system that relays information to the store; and (c) a modulating system that changes the sensitivity of the neurons in the store. The store is located in the left hemisphere, in Wernicke’s area, and the modulation system is located in the left dorsolateral prefrontal cortex. Simpler versions of the neural network models cannot include temporal variations in speech and are therefore not suitable for speech recognition, but more complex models, with more than one hidden layer, achieve better results because they can learn to recognize different acoustic properties, such as second-formant transitions, vowel onsets, and so forth—independent of temporal organization (Ainsworth, 1997). It seems that we have yet to design a model that would solve all the problems of speech recognition or that would integrate all aspects of speech and language functioning into a whole. The theory of neuronal group selection functions similarly to neural or cortical networks. According to this theory, different populations of neurons are structured by the interaction between genetic and experiential factors. The theory predicts exchange of signals by means of parallel and reciprocal connections among groups of neurons (reentrant signaling) and it attempts to explain such a wide selection of phenomena as perception, motor responses, language, and consciousness (Kent & Tjaden, 1997). Spatialization of form hypothesis (parietal hypothesis) predicts that language processing (grammatical competence) is represented in the part(s) of the brain whose primary function is the representation of body image (schema), in other
90
The Cognitive Neuroscience of Human Communication
words, processing of spatial structures, especially with respect to one’s own body. This is a direct isomorphism between spatial and grammatical knowledge. The cerebral area responsible for this function is located in the region where auditory, visual, kinesthetic and inputs from other sensory modalities converge and are integrated (Deane, 1992). The requirements are best met by the inferior parietal lobe. In other words, according to this hypothesis, the seat of grammatical (syntactic) competence should be in the inferior parietal lobe, and not, as traditionally assumed, in the frontal lobe, particularly Broca’s area. According to this hypothesis, aphasia can be explained by the loss of awareness about one’s own body. Pansini’s (1988) notion of spatioception (spatial perception) as a necessary prerequisite for speech acquisition includes five sensory systems: tactile, proprioceptive, vestibular, auditory, and visual. Stimuli from these five modalities converge in the part of the brain predicted by parietal hypothesis—on the border between the parietal, temporal and occipital lobes. Regardless of whether they are taken as five sensory modalities that take part in perception of space or as a new modality—spatioceptive—the underlying idea is that without them speech acquisition would be impossible just as spatial and language relations (grammar, primarily syntax, is based on these rules and relations) would be unthinkable. This idea is in agreement with the parietal hypothesis about the representation of language functions in the brain. A study of the relationship between space and native versus foreign language revealed a very strong effect of language on the preference of the left or the right hemispace (Mildner, 2002): When the processing was in the native language the subjects preferred reaching into the left hemisphere, and when the stimuli were in a foreign language they preferred the right hemispace. Here is some evidence in support of the parietal hypothesis (Deane, 1992): 1. The inferior parietal lobe is the seat of bodily awareness and orientation toward extrapersonal space. It integrates somatosensory information. Moreover, this is the only cortical area in which information from many modalities is available at the same time. There is ample clinical evidence and evidence from primate experiments to support this claim. 2. The abilities to represent and manipulate complex spatial relations are located in the inferior parietal lobe. This includes the ability to recognize complex objects on the basis of the nature and arrangement of their components (right hemisphere in humans). 3. The inferior parietal lobe seems to be necessary for constructing sequential plans of action—for manipulation of a series of actions as mental entities (left hemisphere in right-handers). The left hemisphere is responsible for mental manipulation of abstract structures. Damage to the parietal lobe causes various symptoms: global aphasia, agrammatism, reading difficulties, and disorders of grammar and logic, and so forth. 4. The right inferior parietal lobe has only visuospatial and somatic functions and is considerably more efficient in spatial processing than the corresponding area in the left hemisphere. Homotopic areas in the left
The Central Nervous System
91
hemisphere have a key role in hand–eye coordination and in general in hand movements. The hand is basically a parietal lobe organ (as extremities in general, it is the part of the body whose main function is interaction with the environment). The parietal hypothesis predicts that the greatest damage to structural thought (ngrammar) will occur as a consequence of injury to the margins of the inferior parietal lobe, and that the nature of the deficit will depend on connections that have been severed. This, in turn, is defined by functional brain geometry (Deane, 1992). According to the hypothesis, aphasia will occur after injury to any segment of the network, not only as a consequence of direct damage in the injured area, but also because the injured area can no longer be activated, or the result of its activity is not a desirable input to other parts of the network. This may explain why there are so few cases of aphasia that can be clearly categorized, as well as cases of, for example, Broca’s aphasia without actual damage to Broca’s area. Damasio and Damasio (1992) represent language processing in the brain through three sets of interacting structures. Their description is similar to Luria’s model of three functional units (see previous discussion), but it is interactive and to some extent it relies on neural networks as well. The first set consists of a large group of neural structures in both hemispheres that represent nonlanguage interactions between body and environment, realized by means of various sensory and motor systems; in other words, everything that a person does, perceives, or feels. The brain categorizes these nonlanguage representations (based on shape, color, order, emotional state, etc.) and creates a new level of representation for the results of this categorization. This is how humans organize objects, events, and relations. Subsequent layers of categories and symbolic representations are the basis of abstraction and metaphor. The second set comprises a smaller number of neural structures, mainly located in the left hemisphere, that represent phonemes, their combinations and syntactic rules for combining words. When stimulated from within the brain, they produce words and sentences that will be said or written. When stimulated by the environment, by means of speech or writing, they engage in initial processing of auditory or visual language signals. The third set (also mainly in the left hemisphere), acts as a mediator between the first two sets. This set may start with the concept and initiate word production, or it may receive words and stimulate the brain to retrieve appropriate concepts. Besides selecting appropriate words to denote a particular concept, these intermediary systems control the production of sentence structures that express relations among concepts. Neural structures that serve as mediators between concepts and words are located along the occipitotemporal axis. It seems that mediation for the many general terms occurs in the posterior part of the axis, whereas the most specific ones are mediated by the most anterior part, near the left temporal pole. Damasio and Damasio (1992) conclude this on the basis of clinical cases of focal injuries
92
The Cognitive Neuroscience of Human Communication
in different parts of the left temporal pole (see also Gazzaniga, 1994). Depending on the location of the injury, there may be selective damage to common or proper nouns. Injury to some parts of the temporal lobe may impair access to verbs and function words, but also the grammatical structure of the sentences produced by the patients. In general, Damasio and Damasio (1992) redefine the notion of “images” in the brain—physical representations of different concepts and experiences. They claim that the brain preserves a trace of neural activity that was elicited in the sensory and motor cortex during interaction with an object or a person, rather than a permanent record of actual “pictorial” representations of objects or persons, as was traditionally believed. These traces are patterns of synaptic connections that may repeatedly combine separate sets of activities defining an object or an event. Each trace may initiate related traces as well. In other words, besides preserving the representations of the external world, the brain records how the body reacts to the outside world. This interaction is represented in simultaneous microperceptions and microactivities in separate functional regions. These separate activities are united in neural assemblies in many areas of “convergence” in the brain, which would correspond to the traditional notion of association areas, as well as Turner and Frackowiak’s (1995) notion of cortical fields, or the Hebbian cell assemblies, as suggested by Pulvermüller (1992). The brain also categorizes information so that related events and concepts may be reactivated together. This possibility is based on the physiological principle that the neurons that are active together tend to group together (see the “fire together–wire together” principle in chap. 1, this volume). Activity within such a network may serve both expression and comprehension. It is precisely due to categorizing percepts and actions simultaneously along many different dimensions that symbolic representations (e.g., metaphors) are possible (Damasio & Damasio, 1992). Localistic and connectionist approaches actually agree on the three areas in which focal lesions may impair language processing: (a) in Broca’s area, that affects speech production; (b) in Wernicke’s area, that affects speech comprehension; and (c) in the angular gyrus, that disrupts written language and mathematical operations and may result in severe aphasia (Deane, 1992).
OTHER THEORIES AND MODELS Many other theories and models of language functioning may be found in the literature. General language models that do not deal specifically with neurofunctional organization of speech and language activity have three layers in common: 1. The conceptual layer—the highest, language-independent level, where thought processes take place. 2. The language layer—in which concepts are given names and are being manipulated (where the signified and the signifier get paired up); the language-specific layer contains all language rules, including syntax, morphology, etc.
The Central Nervous System
93
3. The speech layer—where the message is materialized in the process of speech production, or where the first processing of input signals starts in the process of speech perception. Among the best described models and theories of speech perception and/or recognition that do not go into specific neurofunctional substrates are the motor theory of speech (Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967), theory of acoustic invariance (Blumstein & Stevens, 1979), analysis-by-synthesis model (Halle & Stevens, 1959), Logogen theory of word recognition (Morton, 1979), cohort theory (Marslen-Wilson, 1980; Marslen-Wilson & Tyler, 1980; Marslen-Wilson & Warren, 1994), TRACE model (McClelland, 1991; McClelland & Elman, 1986), native language magnet theory (Kuhl, Williams, Lacerda, Stevens, & Lindblom, 1992), Markoff’s hidden models (short review in Ainsworth, 1997), RACE, Shortlist (review in McQueen & Cutler, 1997), and the neighborhood activation model (Luce & Pisoni, 1998). Some of them are briefly presented in the following section, but this is by no means an exhaustive list. Descriptions of more models and theories can be found in Eysenck and Keane (2000), Gazzaniga et al. (2002), Horga (1996), Ladefoged (2004), Lieberman and Blumstein (1988), Luce & McLennan (2005), McQueen and Cutler (1997), Moore (1997), Pinel (2005), and Sternberg (2003). Motor Theory of Speech Perception According to this theory (Liberman et al., 1967), perception is based on speech production, with the following hypotheses: 1. Since humans are both recipients and producers of speech, it is rational to assume the existence of a single mechanism. 2. Since invariant (i.e., constant) commands to articulators must exist in order to produce the same consonant in different vowel environments, it is logical to assume that the perceptual system makes use of these commands. Imitation may be cortical (not necessarily realized at the level of peripheral speech organs). 3. The hypothesis is valid at the suprasegmental level as well. Several facts and observations support this theory: 1. During speech acquisition, the relationship between acoustic speech signals and sets of articulatory gestures is established by imitation. 2. People move their lips silently to facilitate understanding. 3. Speech perception in aphasics is impaired if they are asked to clench their teeth/keep their tongue between the teeth, which immobilizes their speech muscles. 4. Lip and face reading. 5. Electromyographic measurements have shown that the relationship between muscle activity patterns for a particular sound and its perceptual
94
The Cognitive Neuroscience of Human Communication
result is stronger than the relationship between acoustic properties of the sound and its perception. 6. Neuroanatomically, there are points that are shared by speech production and perception—their stimulation may impair phoneme identification as well as face and lip movements characteristic of speech. However, there are also some inconsistencies and problems with the theory: 1. Electromyographic data have shown that gestures are as variable as the acoustic signal. The same target (acoustic outcome) may be achieved by different movements (e.g., in cases of perturbed speech, such as when bite-block is introduced). 2. In the speech acquisition process comprehension precedes production. 3. The theory has not answered the question of how the listener extracts relevant data from the acoustic signal. At what level of the perceptual process does articulation come in? Besides, if auditory analysis is at the level at which its results can be compared with articulatory patterns, then it must be possible to bring auditory processing to the final decision without the help of articulatory patterns. Avoiding the mediators is more efficient. 4. Processes that would relate listening and articulation, regardless of level, would probably be very complex and therefore very slow, which again goes against the efficacy requirement. 5. In different types of aphasia speech production and perception are not equally impaired. 6. Simultaneous interpreting. 7. Neuroanatomically, there are sufficient examples of different and separate representations of speech production and perception processes. Liberman and Mattingly (1985) proposed a revised motor theory of speech, with partly independently functioning modules. The phenomenon of the socalled duplex perception, which suggests the existence of speech and nonspeech modules for the same stimuli, is also taken to support the motor theory (more on duplex perception can be found in chap. 8, this volume). Analysis by Synthesis Halle and Stevens’ theory (1959) is related to the motor theory—perception is to some extent based on production, but analysis-by-synthesis does not posit actual motor commands. In the process of comparing the input signal with an internally generated referent (listeners decode the acoustic signal by internally generating corresponding signals) the listeners perceive the one that best matches one of the existing internal models. This is called template matching. The process of identifying the phonetic aspect of an utterance goes through several stages: (a) peripheral analysis (preliminary analysis), during which the speech signal is broken down into distinctive features, yielding a rough, hypothetical matrix of phonetic
The Central Nervous System
95
segments and features; (b) dynamic analysis of the matrix by means of a series of generative rules, resulting in a phonemic decision; and (c) comparison of the original pattern and possible solutions (iteratively by means of generative rules) gives a final solution that is transmitted to higher perceptual levels. The main components of the theory are the following: (a) some sort of articulatory description is interpolated between auditory analysis and final perceptual decision; (b) the process is dynamic—analysis is carried out by active synthesis of patterns at a particular stage of the entire process. The calculations resemble those that, in speech production, result in motor activity (on the basis of afferent feedback actual movements are modified to reach the desired target). Chomsky and Halle (1968) proposed a variant of the theory. According to them, comparisons between the input signal and the patterns stored in memory are carried out at phonological and syntactic levels, rather than at the level of neuroacoustic signal processing, as in Stevens. The input signal is the initial impulse that enables the listeners to set hypotheses about the content of the speech message, relying on their linguistic and general knowledge, and to test the hypotheses as the signal goes on. Top-down processing plays a very important role. Decisions made at the lower level are hypothetical, and the final decision is made when data from both directions are merged or when the information received by lower-level processing is confirmed by information coming from higher levels. Auditory Theory According to Fant’s theory (1967), acoustic analysis of the signal is sufficient for speech perception. It proceeds as primary auditory analysis, followed by analysis that determines distinctive features (feature analysis) and finally auditory features are combined into phonemes, syllables, morphemes, and words. The relaying of the message to the speech center in the brain includes motor and sensory components. The theory allows interactions between these mechanisms. Neural (Phonetic, Linguistic) Feature Detectors This theory of Jakobson, Fant, and Halle (1976) proposes neural assemblies that react to different segments of the speech signal (features) and that are in harmony with production capabilities and programs. The following experimental and empirical evidence is taken in support of the theory: (a) selective adaptation—after multiple repetition of a voiced sound (more than 100 times) subjects identify more stimuli as voiceless; (b) speech errors are usually off by one feature and the substitutions are systematic; (c) aphasias; (d) perception of sounds under difficult circumstances. Theory of Acoustic Invariance Blumstein and Stevens’ theory (1979) assumes invariance in the acoustic signal. The mechanisms of vowel perception are applied, with two additional assumptions in case of place of articulation in stops: (a) moment of release and the first
96
The Cognitive Neuroscience of Human Communication
20 ms are crucial—they carry information about place of articulation (before formant transitions); (b) combination of spectral features (compact/diffuse and gravis/acute). The authors claim that the theory enables 85% accurate identification of initial stops followed by vowels, somewhat lower in the final position. The theory does not deny the importance of other cues (e.g., transitions). This theory also assumes template matching. The Cohort Theory According to this theory (Marslen-Wilson & Tyler, 1980; Marslen-Wilson & Warren, 1994), word recognition proceeds in several steps: (a) at the beginning of auditory presentation all words familiar to the listener that start with the same sound(s) as the one presented are activated (ninitial cohort); (b) by increasing the duration of the presented stimulus, words that no longer correspond to further information, semantic or some other context, are eliminated; (c) processing continues until contextual information, and information available from the word itself become sufficient to single out one word (npoint of recognition). In the original theory the key role is played by the word onset: the word will not be recognized if the first phoneme is unclear or ambiguous. Context is important from the very beginning. The modified theory assumes different activation levels of candidate words (words that have identical endings are activated but with some delay). Context effects occur later. Trace Model McClelland’s model (McClelland & Elman, 1986; McClelland, 1991) posits individual processing units (nodes) at three levels: features, phonemes, and words. Feature nodes are connected with phoneme nodes, and phoneme nodes with word nodes. Connections between levels are two-way and facilitatory. Connections among nodes at the same level are inhibitory. Nodes affect each other proportionately to their activation levels and strength of interconnections. Spreading of excitation and inhibition among nodes develops an activation pattern, or trace. The recognized word is determined on the basis of the activation level of candidate words. One of the main disadvantages of this model is attributing too much importance to top-down processing: according to the model, nonwords would be inhibited, which is true only in difficult listening conditions. Another problem is that the model cannot be generalized to different speaking rates, because it assumes fixed timing patterns between levels. The model does not account for learning. The Neighborhood Activation Model (NAM) The neighborhood activation model (Luce & Pisoni, 1998) is a spoken word recognition model aimed at dealing with the representation and structural organization of the mental lexicon. According to this model, a spoken word activates a set (neighborhood) of words in memory that sound like the input word (which
The Central Nervous System
97
is phonologically similar), and that subsequently compete for recognition. The model takes into account the number of words in a neighborhood, the degree of their phonetic similarity, and the frequencies of their occurrence in the language. It predicts that a high-density neighborhood (in which many words share segments or sequences of segments) will affect word recognition in such a way that these words will be recognized more slowly and less accurately than words from a low-density neighborhood. The main problem of the model is its inapplicability to nonsense words, i.e., to the sublexical level. PARSYN PARSYN (Luce, Goldinger, Auer, & Vitevich, 2000) is a localist connectionist version of the NAM model. It is an improvement of NAM in light of its ability to account for the sublexical level as well. It consists of three levels (layers) of interconnected units: an input allophone level, a pattern allophone level, and a word level. The fact that it proposes an allophonic rather than a phonemic intermediate level of representation makes it unique among similar models, such as Trace or Shortlist. Between-level connections are facilitative and within-level connections are inhibitory, with some exceptions. The word level inhibits the lower (pattern) level when a single word gains advantage over its competitors and each unit at this level is capable of inhibiting other word units. The Mirror–Neuron System It has been suggested that mirror neurons may be a basis for higher level cognitive skills such as language and theory of mind (Rizzolatti & Arbib, 1998; Rizzolatti & Craighero, 2004). Mirror neurons are neurons that fire not only when the individual performs an action, but also when observing someone else performing the same action, and are therefore believed to underlie imitation learning and mind reading. This enables us to anticipate other people’s actions and to empathize. They were found in primate (specifically monkey) and bird brains by means of electrophysiological studies, and their existence in human brains is confirmed by correlations between actions and observations and by scanning and neuroimaging techniques (Hauk & Pulvermüller, 2004; Saygin, Wilson, Dronkers, & Bates, 2004). Some mental disorders (e.g., autism) are thought to be a consequence of deficits in the mirror–neuron system. With respect to evolution of language, according to this theory, verbal language has evolved from gestural language (as had been suggested by Kimura, 1975).
6 Lateralization and Localization of Functions In chapter 3 on neurolinguistics, I introduced some of the key terms related to descriptions of neural, primarily cerebral, substrates of speech and language. These are lateralization, localization, and specialization, and the related terms representation and mapping. Before we proceed with a systematic analysis of the relationship between speech and language functioning and brain activity, which is the central topic of the book, each of these terms are briefly defined. The term lateralization refers to relatively greater control of a certain function by the left or the right hemisphere. If during some function, the left hemisphere is more active than the right it is said that the function is left lateralized; if the right hemisphere is more active than the left, the function is right lateralized. A synonymous term is functional asymmetry. The term localization implies that some function has a specific neural substrate, a locality in the brain where it is realized. The term specialization means that some cerebral structure is particularly “skilled” in performing some function: this may refer to one of the hemispheres (e.g., it is commonly said that the left hemisphere is specialized for language) or to some area (e.g., Broca’s area is specialized for language production). The terms representation and mapping refer to the representation of organs, functions, and so forth in the cerebral cortex.
LATERALIZATION OF FUNCTIONS Studies of communication disorders have traditionally been oriented toward left hemisphere damage, because it was believed to be the seat of speech and language, and for that matter, communication. Hence the term dominant hemisphere, which implied that the right hemisphere was less important and, in terms of communication, silent. Although by the end of the 19th century it was known that the right hemisphere had an important role in perception, it was not until the 1940s and the first callosotomies that the language capabilities of the right hemisphere became interesting to researchers. The first studies confirmed that the right hemisphere was involved in visual perception, but they also showed that it can decode simple language patterns. Cognitive disorders and communication deficits started to be associated with right-hemisphere damage in the past 40 years or so. In the late 1960s, dichotic listening and tachistoscopic studies made it possible to study functional differences between the two hemispheres in healthy subjects. More 99
100
The Cognitive Neuroscience of Human Communication
research followed analyzing differences between the hemispheres in cognitive styles, or strategies implemented in information processing. Besides its dominant role in language functioning, the left hemisphere turned out to be crucial for analytic, linear, rule-based processing (e.g., language rules). On the other hand, the right hemisphere was found to be important not only for visual and spatial information processing, but also for holistic, nonlinear, or parallel processing. This means that it is crucial for information synthesis, grasping the whole picture (gestalt), and processing of new data before any processing rules have been formed. It was noticed that right-hemisphere damage may affect one’s body image, visual memory, visual perception, spatial orientation, or awareness about one’s illness. The right hemisphere began to be considered the seat of artistic talent and creativity, uninhibited by rules, adaptable and capable of processing new, unfamiliar data. Simultaneously, communication ceased being synonymous with language and speech, and the term was extended to include body language and other nonverbal forms. At present, the general opinion is that the left hemisphere is specialized for language, arithmetic, detail analysis, logical thought, temporal and sequential analysis, and serial processing of sensory data. The right hemisphere is believed to be specialized for emotions, intuition, recognition of faces and emotional facial expressions, artistic achievements, attention, recognition of tunes and other musical abilities, visuospatial analysis, and parallel processing of sensory data. These supposed specializations are frequently summarized into several dichotomies. It is said that the left hemisphere is specialized for verbal functions, as opposed to the right, nonverbal, hemisphere. In the same context, the left hemisphere is specialized for language functions and the right hemisphere for visuospatial ones. With respect to the type of data processing, the left hemisphere is referred to as more analytic, specialized for detailed functions, as opposed to the right hemisphere that provides a comprehensive image of the world, because it is apparently specialized for holistic information processing. It is also said that the left hemisphere is specialized for propositional functions and the right hemisphere for appositional ones. With the advancement of computer techniques and the development of models of cognitive functioning some new dichotomies surfaced. Kosslyn (1987) believes that the left hemisphere is specialized for categorical data (e.g., where something is, in a relational sense— whether an object is below or above some other object, what its relation is to the observation point, etc.). On the other hand, the right hemisphere is specialized for coordinate information (the exact location of the objects, distances among them and between each of them and the observer). Robertson, Sergent, and some others (Gazzaniga, Ivry, & Mangun, 2002; Volberg & Hubner, 2004) suggested the local–global dichotomy, in which the left hemisphere is better at local stimuli processing, whereas the right hemisphere is superior in global information processing. Ivry and Robertson (1999) differentiate the two hemispheres in the way frequency data are represented, rather than along the lines of comprehensive cognitive functions (e.g., music or language). Their hypothesis refers to auditory and visual (spatial) data (to be discussed later).
Lateralization and Localization of Functions
101
Another dichotomy emerged recently. Behavioral tests have revealed indirectly that the right hemisphere registers events as they happen, but the left hemisphere analyzes, interprets and draws inferences about them. The left hemisphere uses an interpreter to construct theories by means of which it assimilates newly perceived data into a meaningful whole—this is basically learning and experience forming. The process makes it slower and less precise when the task is mere recognition. On the contrary, the right hemisphere is very precise in registering the stimuli because it engages in no other operation on them (Gazzaniga et al., 2002). Naturally, in an intact brain these two procedures and strategies are simultaneous and complementary, resulting in satisfactory precision and analysis. In their critical review of functional hemispheric asymmetry studies that had been carried out during a 100-year period, Friedman and Polson (1981) summarized several key issues: 1. Relatively small changes in stimuli, instructions to the subjects or other task properties, may result in the opposite direction of increased activity or change the dominant peripheral organ (e.g., ear or visual field). 2. Most studies cannot be replicated in other laboratories or by different methods. 3. There is great variability in subjects’ responses on tasks that are expected to reveal laterality effects, even in homogenous groups. 4. There are intrasubject inconsistencies in the exhibited degree of laterality on different tasks that are used to study the same function. 5. There is no acceptable global theory that would explain even the established regularities. Malamed and Zaidel (1993) also pointed to the influence of language and task effects on lateralized word recognition. Despite the fact that even the most fervent advocates of division of labor between the two hemispheres have not without a doubt proven that the two hemispheres are in opposition to each other, it has become popular to categorize people into left-hemisphere or right-hemisphere groups. In other words, it is common to separate those who, with respect to their cognitive style, rely more on the type of functioning characteristic of one or the other hemisphere. Many self-help and popular psychology books are full of advice on how to use the neuroscientific facts and the “facts” in everyday behavior and communication (Howard, 2000). One of the pitfalls overlooked by the authors who tend to overemphasize the differences between the hemispheres with respect to their information processing characteristics, is attaching cause-and-effect properties to relations. In fact what may be revealed is correlations, and not necessarily the influence of one property, activity, or behavior on another. The right hemisphere seems to be organized more diffusely than the left, so the functions are not as clearly localized. Due to this, some sensory or motor disorders resulting from focal lesions to the right hemisphere are not as severe as when they affect homotopic areas in the left hemisphere. Some authors have attributed this to the fact that the ratio of the white to gray matter is higher in the right
102
The Cognitive Neuroscience of Human Communication
hemisphere than in the left, which supposedly accounts for better inter-area connections in the right hemisphere, as opposed to the better intra-area connections in the left. Accordingly, damage to one area in the right hemisphere causes many different disorders, and injuries to different areas may cause identical disorders. In a healthy population the two hemispheres do not act independently. Differences occurring in response times or some other differences between visual fields, ears or hands may be interpreted as delayed transfer from one hemisphere to the other, as disintegration of stimuli during transfer, as relative functional superiority of one hemisphere for a particular mode of action or as a result of attention. Ringo and colleagues (cited in Gazzaniga et al., 2002) have shown, by using a neural network model, that slow interhemispheric transfer of information results in relatively independent activity of the two hemispheres. Each hemisphere in the model processes the task relatively successfully and quickly, but the communication between them is slow, resulting in greater probability that different processing abilities will develop. Since it takes about 30 ms for information to cross from one hemisphere into the other, and that is relatively slow, Ringo and coworkers believe this to be the basis for functional asymmetries. What are the advantages of having a functionally asymmetrical brain? Hemispheric specialization for different types of processing, or specific activities, may be interpreted as a way to increase functional capacity of the brain: redundancy is reduced, but capabilities to act are considerably increased (Gazzaniga et al., 2002). According to Kertesz (as cited in Kent & Tjaden, 1997) asymmetric representation ensures adaptability, due to complementarity of functions. In speech, this complementarity may be found, for example, between segmental and suprasegmental (prosodic) features, between short-term and long-term operations, between local and global procedures, or between high-frequency and lowfrequency characteristics. Each hemisphere processes the type of data that are optimal for its structure. This increases processing precision and efficacy, but still allows final integration of processed data. Functional asymmetry is desirable in motor control as well, particularly in fast and precise movements characteristic of speech. Peters (1992) believes that symmetrical control would require precise coordination between the two hemispheres, and account for different lengths of the left and the right n. recurrens that innervate larynx muscles. In fact, some authors claim that bilateral control of speech is one of the causes of stuttering and other speech production disorders (see chapter 8, this volume). According to some theories of lateralization of cerebral functions, there is some sort of cooperation and interaction between the two hemispheres, and according to others, each hemisphere has unique functions of its own. The problem with both conflicting views is that they rely on speculation rather than on hard evidence. Advocates of the former group (interactive) share the idea that both hemispheres are able to perform all functions, but do not. As to why that might be so, different versions of interactive theories have different answers. Some claim that the hemispheres function simultaneously, but are engaged in different aspects of processing. Others say that one hemisphere inhibits the activity of the other. For
Lateralization and Localization of Functions
103
example, the left hemisphere inhibits language processing in the right hemisphere. Although from the developmental point of view, these models are acceptable, the problem is that physiological mechanisms of inhibition are not clearly defined. The third group of answers is based on the models of information processing and they predict that the hemispheres receive different kinds of information and, consequently, perform different analyses, or that some mechanism enables each hemisphere to pay attention to a specific type of information. However, these models fail to define mechanisms that would be responsible for selective attention (Kolb & Whishaw, 1996). The second line of thought gave rise to the notion of hemispheres as representatives of different types of cognitive processing (analytic vs. holistic) and in charge of processing different types of material (verbal vs. nonverbal).
VERBAL VERSUS NONVERBAL AND LANGUAGE VERSUS SPATIAL INFORMATION In spite of similarities in general perceptual functions between the left and the right hemisphere, there are subtle differences in the type of processing the stimuli features. Both hemispheres are active in all tasks, including language and spatial ones, but they use the information available in sensory data in different ways. The most frequent dichotomy used to describe functional cerebral asymmetry is the verbal versus nonverbal or language versus spatial data processing. Studies of brain-damaged patients, as well as healthy subjects, have emphasized the role of the left hemisphere in language (more at the motor than at the receptive level). Auditory experience is not essential for left-hemisphere specialization for language. Evidence can be found in profoundly hearing-impaired subjects who had never heard speech. A right-ear advantage (i.e., left-hemisphere dominance) was found for all verbal material, meaningful as well as meaningless, even for linguistic tone. Left-ear advantage (i.e., right-hemisphere dominance) was found for simple, nonverbal acoustic stimuli, such as pitch, harmony, intensity, musical chords, tunes (in some cases) including humming, environmental sounds, nonverbal vocalizations, such as laughing, crying and sighing, sonar signals, emotional tones and intonation curves (Bradshaw & Nettleton, 1983; Kolb & Whishaw, 1996; Springer & Deutsch, 1997; Webster, 1995). Such interpretations of cerebral asymmetry were supported by studies of memory and intelligence carried out independently of asymmetry research itself. Within theories of memory, the concept of a dual coding system was developed: verbal and iconic. Advocates of the theory claim that these two types of material are represented and processed through two separate and functionally independent, albeit interconnected, symbolic systems. According to them, the iconic system operates directly, analogously, and its output has a synchronous and spatial character that allows dynamic transformations with respect to visual and spatial features such as size, location, orientation, rotation, reflexion, distortion, and substitution of imagined objects. The verbal system is specialized for processing language and sequential information involved in speech production. The authors found support for their theory in studies that have shown that verbal and iconic
104
The Cognitive Neuroscience of Human Communication
information may be acquired, coded, stored and retrieved independently, with minimal interference, and involving independent and different types of forgetting (Bradshaw & Nettleton, 1983; Kalat, 1995; Kolb & Whishaw, 1996). Intelligence testing has for quite some time now recognized the two main sets of abilities: verbal and performance. Performance sets in general include abilities related to pattern and space perception (Bradshaw & Nettleton, 1983). Studies with nonhuman primates as well as PET and fMRI results of human studies on processing spatial information have revealed that spatial tasks activated the right hemisphere considerably more than the left, whereas nonspatial tasks (linguistic and nonlinguistic alike) activated both hemispheres, with somewhat greater activation of the left than the right. Verbal tasks did not elicit such clear results. Contrary to humans, monkeys exhibited smaller differences between the two hemispheres with respect to the nature of the task (Goldman-Rakic, 2000). Disconnection of the two hemispheres usually results in the inability to access language (generally left-lateralized) to described percepts and events in right hemispace. This, however, does not change nonverbal listening aspects. Such patients are able to localize the sounds as long as they are not speech stimuli. If the stimuli are words or just parts of words, comprehension or memory is impaired (Aitkin, 1990). Processing of speech sounds, even of meaningless syllables such as pa, ta, ka, ba, da, ga, reveals great differences between hemispheres in split-brain patients. Efron (1990) does not attribute these differences (nor the ones found in dichotic listening studies in healthy subjects) to physiological suppression at the cerebral level as proposed by Kimura (1961a, 1961b), or to the difference between direct and indirect access, but to the tendency to pay attention to one side or the other. This tendency is stronger in split-brain patients than in people with intact corpus callosum. It might be concluded that left–right asymmetry in dichotic tests does not reveal hemisphere specialization for particular kinds of stimuli (e.g., music or language). Efron (1990) found, in his studies with visual stimuli, that people start scanning the stimulus from right to left more frequently than in the other direction, and concluded that this might underlie the right-ear advantage in dichotic tests. However, he failed to account for left-ear advantage for musical stimuli in dichotic listening tests. Finally, he concluded that his hypothesis allowed leftvisual-field advantage as well, because it is not based on the assumption of asymmetry between the left and the right hemisphere functioning. The left temporal lobe is responsible for the perception of short auditory stimuli (shorter than 90 ms), sound distinction (patients with damage to this region need longer pauses between stimuli in order to be able to distinguish them), and for recognition of sound sequences. Speech sounds are also distinguished better in the left hemisphere than in the right (see chap. 8, this volume). The left hemisphere is dominant for selective auditory attention as well. On the other hand, the right hemisphere seems to be dominant for music (Kolb & Whishaw, 1996). It appears that musical elements—pitch, duration, height, loudness, and rhythm—are separate entities, because injuries affect them in different ways
Lateralization and Localization of Functions
105
(Bradshaw & Nettleton, 1983). Since corresponding elements exist in speech as well, analogous division of functions may be expected. For auditory stimuli in general, it has been found that the right ear (left hemisphere) has an advantage over the left ear (right hemisphere) in dealing with verbal information, and that the opposite is true for nonverbal information. Ipsilateral pathways are partly or completely suppressed by contralateral activation. Verbal information presented to the left ear is degraded in the right hemisphere (which is reached first) or during transfer across the corpus callosum into the left hemisphere where verbal information is eventually processed. As it has been said before (see chap. 1, this volume), there are more nerve fibers in contralateral auditory pathways according to neurophysiologic evidence, and contralateral stimulation elicits greater cortical activity. In dichotic listening tests Kimura (1961b) found right-ear advantage for repetition of digits, and concluded that the left hemisphere is dominant for numbers. After that, many dichotic listening studies have been carried out for verbal and nonverbal stimuli. Behavioral tests with very small children have shown that most of them, just like adults, exhibit right-ear advantage (left-hemisphere dominance) for verbal stimuli, and left-ear advantage (right-hemisphere dominance) for music (Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Activity was recorded in both hemispheres, contrary to some other studies in which greater blood flow was found in the right hemisphere while distinguishing tones with respect to their temporal patterns, as opposed to the increased blood flow in the left hemisphere when the stimuli were words. This is in agreement with the neuromagnetic reactions to vowel onsets that have been recorded in both hemispheres. It seems that the difference is quantitative rather than qualitative: the amount of tissue activated by speech sounds is greater in the left hemisphere than in the right (Aitkin, 1990). Sidtis (as cited in Bradshaw & Nettleton, 1983) concludes that the right hemisphere is specialized for analysis of harmony information during steady states of complex tones, and not for musical perception as such. Left-ear advantage is greater for more complex (with more overtones) musical signals. Studies with visual stimuli have shown that the left hemisphere is dominant for letters and digits. For letters, right-visual-field advantage is greatest for stops, followed by fricatives, and it is the smallest for vowels. The left hemisphere is dominant when the letters are nominally identical (e.g., Aa or Bb), whereas the right hemisphere is dominant when the pairs contain physically identical letters (e.g., AA or bb). Lohmann et al. (2000) found, by using fTCD, marked double dissociation in lateralization of language and spatial attention in 10 out of 12 of their subjects: in subjects with left-hemisphere dominance for language spatial attention increased activity in the right hemisphere, whereas in subjects with right-hemisphere dominance for language, spatial attention elicited activity in the left hemisphere. In two subjects both functions were lateralized to the right. In a study with Croatian– English unbalanced bilinguals Mildner (2002) found a clear dissociation between languages and preferred hemispace: when the auditory stimuli were in the native
106
The Cognitive Neuroscience of Human Communication
language (Croatian) the subjects preferred the left hemispace, when the stimuli were in the foreign language (English) they preferred the right hemispace. Right-handers with right-hemisphere damage have difficulty performing spatial tasks. They lose their way, forget the paths they had taken many times, and they have problems understanding complex diagrams and pictures (Thompson, 1993). Besides its role in spatial abilities, the right hemisphere is involved in nonspeech communication—facial expressions, body language, and intonation. In a way, the corpus callosum eliminates the need for bilateral multiplication of structures; in evolutionary terms, this might have enabled the development of higher functions such as spoken communication (Levy, Koeppen, & Stanton, 2005). At the simple visuospatial level, the right hemisphere is almost exclusively the active domain. This is also true of recognition of objects by separating the figure from the background, or on the basis of incomplete pictures. At the level of visual construction, when parts are integrated into a whole, both hemispheres may be involved. In general, the more complex the task, the greater the involvement of both hemispheres. The simpler the task, the abilities necessary for its performance rely more on the right hemisphere. Left temporal lobe damage causes disorders in verbal memory and verbal intelligence. Damage to the corresponding area in the right temporal lobe results in difficulties in reading, recall, and recognition of nonverbal visual and auditory patterns, as well as a decrease in nonverbal IQ. Injuries to the left frontal and fronto-medial area decrease verbal fluency. Identical injuries in the right hemisphere affect rapid production of abstract, meaningless patterns. One of the functions of the corpus callosum, besides information transfer, is allocation of attention in order to ensure focal unity. Disconnection results in problems with unilateral directing of attention, and the consequences are poorer information processing capabilities (Posner & Dehaene, 1994). It is, therefore, possible that the corpus callosum also plays an inhibitory role, which is manifested as prevention of synkinesia. It may also inhibit the development of language functions in the right hemisphere. It has been found that people with bilateral speech control often stutter, probably due to rival access to the single speech apparatus from two hemispheres. It has also been suggested that bilateral representation of language may diminish spatial abilities, probably because right-hemisphere language ability inhibits spatial functioning (Bradshaw & Nettleton, 1983). Split-brain patients’ behavior suggests that the right hemisphere may comprehend simple speech, read simple names, and that the left hand may comply with verbal commands (if they are not too complex), reach for named objects and select the appropriate object. The right hand is still dominant for writing, as in healthy subjects, but the left hand is used for drawing and model construction. The right hemisphere is superior in many spatial tasks, including copying of drawings, facial expression recognition, and so forth. The right hemisphere is better than the left in recognition of faces, but the left is superior when purposeful facial expressions have to be made (Gazzaniga et al., 2002). That hemisphere can recognize and identify social relations, pictures of people, family members, acquaintances, pets, and personal belongings, as well as pictures of historical or
Lateralization and Localization of Functions
107
public persons. Both hemispheres have general awareness of body states, such as hunger or fatigue (Kolb & Whishaw, 1996). In general, consequences of damage to the right hemisphere suggest that the following may be some of its functions (Bradshaw & Nettleton, 1983; Gali´c, 2002; Kalat, 1995; Myers, 1999): understanding spatial relations among objects and within objects in extrapersonal space; recognition and processing of complex visual patterns; precise perception, memory and recognition of abstract or meaningless forms that are too complex to be described verbally; ability to copy given forms and block design; face recognition; ability to understand and produce different musical aspects (although rhythm is usually unaffected by injury); appreciation of humor; control of emotions (particularly in the frontal and medial temporal lobe); recognition of emotional aspects of the stimulus; facial expression and recognition of emotions. Experiments with tachistoscopically presented visual and visuospatial and language stimuli also support the claim that the right visual field (hence, left cerebral hemisphere) has advantage over the left for language stimuli, and that the left-visual-field advantage in visuospatial stimuli is the sign of right-hemisphere dominance for that type of material. However, it has been shown that retest results were in poor correlation with the original ones. Namely, only 42% of subjects exhibited right-visual-field advantage on both tests. When distinguishing the so-called spatial, nonspatial and verbal tasks, one should keep in mind that it is impossible to exclude the verbal element entirely, even in the “clearly” spatial tasks, because it is impossible to control the amount of verbalization that goes on during the task. It is equally difficult to conceive of a nonspatial task with absolutely no spatial processing involved.
ANALYTIC VERSUS HOLISTIC APPROACH TO PROCESSING Analytic versus holistic dichotomy is based on the premise that there are some advantages to locating the areas of the brain that perform similar functions into the same hemisphere, and that there are two basic types of reasoning: analytic and synthetic, each requiring different neural circuits. During evolution, functions that require analytic reasoning separated from those requiring synthetic thought, so that analytic and synthetic mechanisms positioned themselves in the left and right hemisphere, respectively. The left hemisphere operates in a more logical, analytic and computer-like mode, analyzing the stimulus sequentially, extracting relevant details, and giving them verbal labels. The right hemisphere is primarily a synthesizer that deals predominantly with the overall stimulus configuration, and organizes and processes information as gestalts or wholes (Levy, 1969; Pinel, 2005). It seems that the right hemisphere is aware of the cognitive potential of the left, but not the other way around. With respect to visuospatial abilities, it has been shown that the right hemisphere is capable of storing and producing drawings and patterns that are acceptable as a whole, but lacking in detail. Results of the left-hemisphere activity are rich in detail, but without these details making
108
The Cognitive Neuroscience of Human Communication
any sense as a whole. The left hemisphere is better at analyzing a whole into constituent parts, whereas the right is more successful at recognizing the overall stimulus configuration and matching parts with appropriate wholes. In general, it has been shown that the right hemisphere is better at tasks that involve holistic, gestalt approach, whereas the left is better at analytic tasks, regardless of sensory modality. It has also been found that laterality differences are more marked at the motor than at the sensory level, in verbal and nonverbal tasks alike (Bradshaw & Nettleton, 1983). Consequences of left-hemisphere damage indicate that it controls voluntary movements (injuries to specific areas frequently cause apraxia) as well as the ability to find one’s way in a building or city (i.e., the position of a body in space). Contrary to that, the right hemisphere controls spatial perception in the sense of understanding space regardless of the subject’s position in it. Patients suffering from left-hemisphere damage omit details in their drawings, but keep the spatial relations among elements, and in general preserve the overall coherence of the form. On the other hand, right-hemisphere patients have no difficulties with details, but have too many elements that are poorly connected into a coherent whole, with relations among elements and their spatial arrangement typically full of errors (Bradshaw & Nettleton, 1983; Gali´c, 2002; Kalat, 1995; Kolb & Whishaw, 1996; Webster, 1995). The right hemisphere is dominant in face recognition (measured as response times and accuracy) if the responses are supplied manually. If verbal response is required, response time superiority is lost. In some individuals, recognition of familiar faces activates the left hemisphere more than the right. If the faces are to be distinguished by only one feature (obviously an analytic procedure), the left hemisphere is better. Yovel, Levy, Grabowecky, and Paller (2003) concluded on the basis of their ERP results that the two hemispheres exchange information at early stages of face processing. Facial representation is better if facial information is presented directly to the right hemisphere than to the left. It is best when both hemispheres receive facial information. In general, for visual stimuli, in terms of pattern recognition, the superiority of the right visual field is more probable (i.e., left-hemisphere dominance) for familiar and simple forms that are easily named. Left-visual-field advantage (right-hemisphere dominance) was found for letters or words that are difficult, unfamiliar and presented very briefly to the subjects. This again suggests that the differences between the two hemispheres are relative and quantitative rather than absolute or qualitative, which in turn is in agreement with the fact that there is no structure, at any level of analysis, that might exist in only one half of the intact brain (Gazzaniga, 1994).
SERIAL OR SEQUENTIAL VERSUS PARALLEL PROCESSING From the evolutionary point of view, it is probably more correct to look at the serial, segmental motor specialization of the left hemisphere as the origin, and the right hemisphere as basically acting in a less specialized manner. The right hemisphere may mediate in different relatively unspecialized, more primitive,
Lateralization and Localization of Functions
109
and possibly unconnected functions that have been “expelled” from the left hemisphere. Even activities that the right hemisphere is believed to be specialized in are less clearly defined, less well supported by evidence, and less unique. However, the contribution of the right hemisphere to social, affective, esthetic, and even cognitive competence should not be neglected (Bradshaw & Nettleton, 1983; Myers, 1999). In line with this reasoning is Sussman’s paradigm (Bradshaw & Nettleton, 1983; Sussman, 1994), according to which the left hemisphere is specialized for sensorimotor integration of movements related to speech as well as for movements of arms and legs, hands, fingers, and orofacial muscles, even when they are not associated with phoneme production. Most likely it is not a coincidence that the cortical representations of hands, mouth, and ears are so close to each other in humans and that all of them are relatively much larger than the corresponding areas in apes. Only humans are capable of using complex tools and verbal and gestural communication, and all of these activities require sequential, timed syntactic mechanisms in order to flexibly generate new rule-based sequences. Problems arise when brain functions are studied using simple verbal-nonverbal oppositions because: (a) the right hemisphere has considerable language abilities, albeit more receptive than expressive; (b) the left hemisphere is superior in some visuospatial tasks that rely on strategies involving analytical extraction of significant or distinctive features or elements; (c) the right hemisphere is not dominant for music in all aspects, particularly in cases where the sequential or time-dependent nature of sound stimuli is relevant (e.g., in judging temporal sequence, duration, simultaneity, rhythm, or in categorical perception). Moreover, clinical and healthy-subject studies indicate that the left hemisphere is perhaps uniquely specialized with respect to temporal order, sequencing, and segmentation at sensory and motor levels alike. Motor level involves control of rapid changes in the position of hands, legs, fingers and articulators. It seems that the left hemisphere acts as a dynamic, kinesthetic and proprioceptive, time-dependent control system for attaining goals within the body, whereas the right hemisphere acts through static, exteroceptive, visuospatial functions, mainly outside the body. This notion is supported by work with dysphasic and dyslexic children who seem to suffer from deficits in perception of temporal order of nonverbal auditory stimuli (Bradshaw & Nettleton, 1983). In other words, left-hemisphere mediation in language is possibly based on the need for analytic, time-dependent, and sequential coding (primarily at the expressive level, but to some extent at the receptive level as well), rather than on its symbolic or phonological attributes.
LOCAL VERSUS GLOBAL DATA REPRESENTATION Robertson and her coworkers (as cited in Gazzaniga et al., 2002) studied patients with unilateral brain damage and concluded that the left hemisphere is superior to the right in representation of local data, whereas the right hemisphere is better in processing global data. One of the common methods of research into differences between the local and global approach involves presentation of shapes of capital
110
The Cognitive Neuroscience of Human Communication
letters consisting of capital letter of smaller size (as in the examples that follow). The local elements may be identical to the global ones or they may be different (i.e., the same or a different letter). Example 1: Local and global elements are identical (T). TTTTTTTTTTT T T T T T T Example 2: Local (T) and global (L) elements are different. T T T T T T TTTT Robertson (1982, as cited in Gazzaniga et al., 2002), got similar results working with split-brain patients using language (letters) and nonlanguage forms (geometric shapes), as did Sergent in tests with healthy subjects (as cited in Gazzaniga et al., 2002).
HIGH FREQUENCIES VERSUS LOW FREQUENCIES Ivry and Robertson (1999) claim that there are small but important differences between the hemispheres in the early stages of data processing. Interaction of these initial asymmetries with higher order systems enhances these differences, and the final result of data processing seems qualitatively asymmetrical. Dichotic tests had indicated (based on the left-ear advantage) that the right hemisphere is mainly dominant for pure tones. However, Efron (1990) cautions that in some subjects, ear advantage changes with the pitch of tones used in experiments. Differences among presented tones expressed as a simple pitch difference between the higher and the lower tone or as pitch ratio between the two presented tones calculated as their average (central frequency) affect ear advantage in different ways. In approximately 43% of subjects the ear advantage shifts from the left to the right ear with the increase of the simple frequency difference between the presented tones. In about 24% the situation is exactly the opposite and in about 33% of subjects there is no difference. Similar results were obtained for the relation between central frequency and ear advantage: Some subjects exhibited
Lateralization and Localization of Functions
111
right-ear advantage for all central frequencies (the tones used in the tests were between 200 Hz and 4,800 Hz); some exhibited left-ear advantage for all central frequencies; and in others higher central frequency caused the right ear to take over. In subjects with very strong ear advantage (regardless of side) a very large intensity difference between the two presented tones was needed to eliminate the ear advantage effect. Efron and coworkers interpreted this ear advantage in pitch processing as being the property of the peripheral part of the auditory system—up to the level of the brainstem (see Efron, 1990, for review). Their conclusions were based on tests with healthy subjects and on the behavior of split-brain patients. Ear advantage for pitch may occur in the inner ear, as the difference between the two cochleae in frequency tuning, or in the brainstem, where the frequency data arriving from the two ears are combined. Ivry and Robertson (1999) also studied the influence of relative pitch on lateralization. According to their theory of hemispheric asymmetries in perception (double filtering by frequency, DFF, theory), the differences between the two hemispheres are based on frequency characteristics of the stimulus. The right hemisphere prefers the low frequencies, and the left hemisphere the high ones. These differences may be found in auditory stimuli (where different-frequency tones are psychoacoustically perceived as different pitch), but in spatial and temporal data as well. Processing of spatial data is mediated by vision. High frequencies in the temporal domain refer to rapid changes, preferred by the left hemisphere or slow changes to which the right hemisphere is more inclined. Regardless of modality, asymmetries are caused by attention. Both hemispheres are equally capable of processing and representation of the entire data spectrum, which means that there are no considerable differences between them at the level of early sensory processes. At a higher level the nature of the task will direct attention to particular features of information at hand and at that point the effect of lateralization based on relative frequency will occur. On the basis of their observations Ivry and Robertson (1999) concluded that the left hemisphere enhances information on a relatively finer scale (i.e., high frequencies) whereas the right hemisphere does it on a larger scale (i.e., low frequencies). The theory deals with the perception of visual and auditory stimuli (visual perception actually refers to spatial data processing). It is founded on three principal ideas: (a) frequency-based representations include information about different multiple scales; (b) frequency information is asymmetrically represented in the left and the right cerebral hemisphere; and (c) selective attention causes laterality that is elicited by relative rather than absolute frequency. According to their theory, perception proceeds through three stages. Stage 1: Sensory Representation of Frequency Information At this stage there are no consistent differences between the two hemispheres. They can both represent any frequency information from the environment. Sensory representation is mainly symmetrical. This stage corresponds to detection, which means that both hemispheres are capable of determining whether the signal was present, regardless of its features and side of presentation (e.g., to which
112
The Cognitive Neuroscience of Human Communication
visual field it was presented). In auditory stimuli some differences were found in sensitivity between the left and the right ear, but only in men—the right ear was more sensitive than the left, particularly for frequencies between 2,000 and 6,000 Hz. Ivry and Robertson ignored the difference, because it did not refer to relative frequency. Stage 2: The First Frequency Filtering Stage— Selection of Task-Relevant Information At this stage the attention is selectively allocated to stimulus aspects that are the most relevant for the goal at hand. This process of selection of relevant stimulus features causes laterality effects that occur in the form of relative frequency. This enhances the segment of the auditory or spatial spectrum that is relevant for the current situation or task. Stage 3: The Second Frequency Filtering Stage—Asymmetric Processing of Selected Information in Cerebral Hemispheres After being selected at Stage 2, relevant information undergoes additional filtering at Stage 3. Identification and discrimination, which are in fact higher order analyses, take place at this stage. During these procedures the differences between the hemispheres become obvious. Right-hemisphere processing is characterized by low-pass filtering (i.e., passing the low frequencies with simultaneous attenuation of the high ones), and left-hemisphere processing is characterized by high-pass filtering (i.e., passing the high frequencies, with simultaneous attenuation of the high ones). The outcomes of such different filtering procedures actually underlie many observed laterality effects in perception. Although filtering is involved in both the second and the third stage, these two procedures are independent. They may be impaired independently of each other and their neural substrates are separate. Ivry and Robertson (1999) believe that both hemispheres are capable of representing all elements of an image, but the quality of these representations is different. It seems that the visual system of humans and other animals has developed different strategies for each hemisphere: The right hemisphere favors global features of an image, and the left prefers the details. Global features correspond to low frequencies, and local features correspond to detailed, or high frequencies. Hence, the differences in representation occur above the sensory level, due to the different types of processing spectral information in the left and the right hemisphere. It may be concluded that simple detection is not lateralized—both hemispheres are equally capable of it, regardless of the nature of the stimulus. However, when further processing (be it discrimination or identification, as more complex and demanding segments of perception) requires selection of a frequency, functional asymmetry occurs. With respect to speech, the dichotomy between high-frequency and low-frequency stimuli is best illustrated by the fact that comprehension of individual sounds requires analysis and presence of high frequencies (left hemisphere), whereas recognition of prosody requires low frequencies (right hemisphere).
Lateralization and Localization of Functions
113
Many independent studies have revealed just such a division of dominance. The local–global dichotomy fits here as well. Obviously, recognition of individual sounds, particularly of consonants, involves processing of local features, as opposed to prosody, that involves processing of global features.
CATEGORICAL VERSUS COORDINATE Kosslyn (1987) compared hemispheric specialization with a snowball: The initial asymmetry is manifested as specialization for a particular type of problem (categorical or coordinate); then this specialization is further enhanced depending on specific task requirements. He believes that left-hemisphere specialization for speech production was crucial for evolutionary development of the left hemisphere as the one specialized for categorical tasks. Namely, speech production must be fast and therefore coordinated by a single center (one hemisphere). Since speech is by its very nature categorical, its lateralization into the left hemisphere has made that hemisphere specialized for categorical types of processing. The need for a single control center that can rapidly and successfully manage sequential movements is not limited solely to speech movements. Such control is necessary for nonspeech movements of articulators, and for any type of motor activity in general. Moreover, Corballis (as cited in Gazzaniga et al., 2002) proposes that the development and use of tools have contributed to the specialization of the left hemisphere for control of skilled movements (e.g., speech).
DEVELOPMENTAL ASPECTS OF LATERALIZATION Two different theories have tried to explain the notion of lateralization of functions. According to Lenneberg (1967) cerebral hemispheres of children from birth to about the second year of life have equal potentials to become a neural substrate for language or other kinds of lateralized functions. Damage to either hemisphere may be equally incapacitating for speech acquisition in children. From the second year until puberty the left hemisphere becomes increasingly more important for language. As the right hemisphere loses its initial language capabilities, cortical plasticity that enables recovery after damage, decreases. After puberty, all throughout adulthood, left-lateralized control of language functions remains relatively unchanged. This theory is based on clinical cases of recovery after aphasia in childhood that have shown that children who had suffered left-hemisphere damage before age 2, have right-lateralized speech. Children who had suffered brain damage between the ages of 2 and 6 had bilaterally represented speech, and those who were injured after the age of 6 had permanent speech disorders similar to those found in adult aphasics. The other theory is based on empirical data about early behavioral and anatomic asymmetries (Hynd, Obrzut, Weed, & Hynd, 1979; Kinsbourne, 1976). According to it, lateralized functioning does not develop gradually: cerebral hemispheres are programmed to function asymmetrically from the moment of birth in such a way that the left hemisphere controls language and motor functions
114
The Cognitive Neuroscience of Human Communication
of most people. Krashen (1973) reviewed clinical and dichotic literature and concluded that hemispheres are not equipotent for language functioning at birth and that behavioral asymmetries do not increase with age, reaching their maximum in puberty. He believed that the ratio of speech disorders in children age 5 or older was the same as in adults, and only right-hemisphere lesions occurring earlier than the age of 5 had some impact on speech abilities. This could have been the consequence of a general cognitive deficit impacting speech. However, Satz, Bakker, Teunissen, Goebel, and Van der Vlugt (1975) did find in their dichotic tests of children that the degree of laterality increased until approximately age 11, with some asymmetry for language functions noticeable around age 5. A dichotic listening study of elementary schoolchildren—unbalanced Croatian-English bilinguals—revealed that around ages 10 to 11 most children do not exhibit clear laterality effects (Mildner & Podnar, unpublished data). It is safe to assume that faster and more satisfactory (albeit almost never complete) recovery of language processes occurs after an early left-hemisphere lesion more so than has been found when damage occurs at an older age; this can be explained by the gradual decrease in neural plasticity. According to Kertesz and McCabe (1977), the inverse relation between the age of aphasia onset and the degree of recovery persists in adults as well. Such a progressive decrease of plasticity is not related to development of lateralization and some areas of the same hemisphere that had not been allocated to any particular function before damage may be activated in the course of recovery (Rasmussen & Milner, 1977). Morphologic brain asymmetries can be seen at birth or even before, although they may simply indicate preprogrammed structural bias underlying functional asymmetries that develop later. Newborns exhibit marked asymmetry of responses to sensory stimulation, with a lower threshold on their right side and a greater tendency to turn toward the right. These effects do not seem to be dependent on previous asymmetrical head position. It is not quite clear whether this asymmetry can be attributed to sensory sensitivity or to motor turning bias (Bradshaw & Nettleton, 1983). It is possible that hemispheric specialization for language functioning is not related to motor asymmetries. This is supported by the considerable percentage of left-lateralized language in left-handers, and some cases of bilateral or righthemisphere language representation in right-handers. Previc (as cited in Gazzaniga et al., 2002) proposed a theory that left-hemisphere dominance for language is primarily related to structural asymmetries in the skull. Most people have somewhat larger orofacial bones on the left side of the face, which indirectly influences ear structure and has a negative impact on the projection of auditory information in the right hemisphere, particularly in the frequency range important for speech. Handedness is determined by the position of the fetus in the womb, and since most fetuses are turned with their right side toward the mother’s front, the left vestibular organ (specifically the left utricle) receives more stimulation. This results in the better developed vestibular system in the right hemisphere. As a consequence, children rely more on their left side for balance and body posture, leaving the right side (i.e., right hand) free for investigation. This theory is not
Lateralization and Localization of Functions
115
quite supported by exact measurements, and the correlations have not been established for individuals, but rather averaged across the entire sample. According to some authors, functional asymmetry of the brain is the prerequisite for full realization of human language potentials, and the absence of such asymmetry is believed to be the cause of numerous difficulties in various intellectual areas. However, Anneken et al. (2000) found no direct or firm connection between the degree of language lateralization (measured by fTCD during word generation task) and success on standard language tests. Lohmann et al. (2000) found by the same method that normal development does not require lateralization of language and spatial functions to different hemispheres. Amunts, Schleicher, Ditterich, and Zilles (2000) applied a postmortem staining technique on the brains of 34 individuals who died at ages 4 months to 85 years, to carry out cytoarchitectonic analysis of Brodmann’s areas 44 and 45 in both hemispheres. They found that during postnatal development there are considerable changes in those areas that are manifested as different laminar patterns. Differences in laminar structure imply different organization of input–output structures in Broca’s area in the two hemispheres. Asymmetry in Broca’s area was found in newborns. Although the differences failed to reach statistical significance, there was a tendency toward greater asymmetry in male brains than in female ones. Considerable changes in asymmetry were observed between newborns and children ages 1 to 5, and between children ages 5 to 12 and adults. However, no changes were found between younger and older children. Developmental changes were more noticeable in right than in left Brodmann’s areas 44 and 45, even in adult brains. This provides one explanation for the smaller asymmetry in children than in adults for language tasks. The overall architecture of neural circuits is probably present at birth and subsequent events are related to establishing particular local connections and eventual branching, as well as to the use of specific receptor architecture in neural networks (Gazzaniga, 1994). ERP studies of 20-month-old babies revealed that lateralization is correlated with the number of words in their repertoire, not their age. Children with rich vocabulary had left-lateralized brain activity while listening to words, whereas those with a relatively smaller vocabulary exhibited bilateral activation in identical conditions (Gazzaniga et al., 2002). Showing that some linguistic knowledge exists at birth would suggest that the key issue is establishing the appropriate number of neurons and connections. With the exception of some smaller cortical sulci, the newborns’ brains are impossible to distinguish from adult ones with respect to convolutions. Besides, it seems that architectonic differentiation is already present at birth, in the sense that it is possible to distinguish all adult cytoarchitectonic areas. Neuronal density is considerably greater than in the adult brain, but myelination of cortico-cortical pathways has not reached full maturity. Asymmetry of the future language areas is noticeable between the 20th and 30th week of gestation, which means that it precedes exposure to language. In any case, the near-adult differentiation is present at birth, only to be perfected later. Corballis (as cited in Gazzaniga et al., 2002) proposes the existence of a laterality gene that is uniquely human, and that determines left-hemisphere dominance
116
The Cognitive Neuroscience of Human Communication
for language and hand use. He also believes that postnatal development, during which lateralization is fixed, is programmed to be asymmetrical. He also points out that equipotentiality of the right hemisphere for language acquisition has to be interpreted as an unrealized possibility that has been inhibited by some processes. Studies of hand preference have yielded many contradictory results, frequently due to poor methodology and criteria. As already mentioned, one suggestion is that the hemispheres differ in maturation rates, so that there are periods when one hemisphere is faster than the other, and then the roles are reversed. All this is combined with a great deal of individual differences. Right-handedness is fixed by age 5, possibly even by the end of the first year of life, and may be related to speech acquisition. The results may depend on whether bimanual or unimanual activities have been assessed. There is evidence of very early asymmetries in holding, reaching and dropping, although their significance is not very clear. There is also evidence of correlations between head posture asymmetries and later handedness (Bradshaw & Nettleton, 1983; Kolb & Whishaw, 1996). However, developmental patterns in hand preference may reflect the way in which the child learns how to lift and manipulate objects. Hand performance measurements offer no clear evidence of developmental changes, and such changes cannot be assessed before the third year of life (Bradshaw & Nettleton, 1983). Registering evoked potentials during visual and auditory stimulation has shown that even in very small children there are stimulus-dependent asymmetries in brain electrical activity. Auditory stimuli elicited greater activity in the left temporal lobe, and visual stimuli resulted in greater activity in the right occipital lobe (Spreen et al., 1984). Molfese and Molfese (1979) showed that newborns are capable of differentiating acoustic cues and that children have an adult-like ability to use phonemically relevant cues in a categorical manner, which suggests some sort of hemispheric specialization for language in early childhood. There is evidence that perceptual mechanisms needed for speech sound discrimination relying on formant transitions and voice onset time (VOT) are lateralized at birth (Bradshaw & Nettleton, 1983). Contrary to Lenneberg’s claims, cases of childhood aphasia after right-hemisphere damage are much less frequent than after left-hemisphere damage. Many potential artifacts may enhance what seems to be right-hemisphere aphasia in very young people: the nature and degree of brain damage in early childhood; the limitations of testing methods suitable for children and their interaction with the environment; and speech may be one of very rare available organized responses in children’s repertoire. As opposed to Lenneberg, Wada’s tests suggest that it is not probable that early left-hemisphere damage will result in speech being taken over by the right hemisphere. The left hemisphere, unless completely removed, will probably become and/or remain dominant for language, in spite of substantial damage. In other words, whenever possible there is intrahemispheric rather than interhemispheric functional reorganization. Left-hemisphere damage in childhood actually does generally result in some degree of aphasia, although it is more transient than in adults. In some cases, left-hemisphere damage occurring before 5 to 7 years of age may initiate a shift of language function to the otherwise
Lateralization and Localization of Functions
117
nondominant right hemisphere, but after that age intrahemispheric reorganization will be more frequent. When the damaged left hemisphere is incapable of plastic reorganization (which is the real key to recovery), the right hemisphere takes control over speech, but subsequent testing in adulthood reveals that such speech is never as perfect as when speech processes are managed by the left hemisphere. The residual permanent deficits reveal limitations of early plasticity of interhemispheric reorganization (Bradshaw & Nettleton, 1983). Cases of childhood hemispherectomy imply that at birth both hemispheres are capable of acquiring language, and that injury to either hemisphere does not impede development of seemingly normal speech. However, the right hemisphere is never quite as successful as the left. During the periods of fastest development, the left hemisphere actively participates in production and perception of speech, and the linguistic potential of the right hemisphere gradually decreases. The left hemisphere becomes firmly dedicated to speech, and these functions can no longer be “squeezed out” by rival functions that may try to shift into it from the injured right hemisphere. On the other hand, the remaining language potential of the right hemisphere indicates that it is not irreversibly dedicated to nonlanguage functions, although transfer of language functions to the right may be at the expense of its nonlanguage, primarily spatial capabilities (Kimura, 1975). Khan, Frisk, and Taylor (1999) found that children with very low birth weight, when they reach ages 8 to 10, use their cortical resources differently from control children: They exhibit smaller functional asymmetry. However, response speed and accuracy do not differ significantly between the two groups. By puberty, the right-hemisphere potential for language acquisition decreases to the point where it is able to mediate little or no propositional speech after left hemispherectomy. This ontogenetic decrease of neural plasticity may be achieved by active suppression by the left hemisphere via the maturing corpus callosum. It is therefore possible that acallosal patients have bilateral language representation, due to the absence of inhibitory pathways. We could even suppose that in healthy children laterality would decrease with maturity, were it not for mutual inhibition of secondary abilities exerted by one hemisphere on the other by means of the developing connections. Some skills may undergo a decrease in asymmetric organization as they become fixed and integrated into the child’s repertoire. Witelson (as cited in Bradshaw & Nettleton, 1983) claims that it is even possible that environmental factors, for example, language experience, influence the degree of left-hemisphere control. She presented evidence of less frequent cases of aphasia in the illiterate and severely hearing impaired population, and in support of the idea that total language deprivation until puberty may result in subsequent right-hemisphere mediation of language processes. A case in point is Genie, who had practically no exposure to language until age 13. When she was tested for lateralization of language functions after years of socialization and deliberate language coaching, she exhibited right-hemisphere language. Healthy subjects, even as young as 3, gesticulate more freely with their right hand while talking, and exhibit greater interference with right-hand than with
118
The Cognitive Neuroscience of Human Communication
left-hand tapping while performing a concurrent verbal task. Right-ear advantage occurs around age 4. Although some authors speak of developmental aspects of the advantage, many more do not. Among the artifacts in such developmental studies is the type of testing that varies greatly across studies, but also the kind of required distinctions. Memory measures, rather than perceptual functions, are used with older children. On the other hand, with younger subjects perceptual measures have to be used, despite the probability of weaker laterality effects that have been found at the perceptual level. Ceiling and floor effects may blur laterality differences. Ability to direct attention develops with age, and this particular factor has been shown to increase ear advantage. When attention aspects are controlled, there is no developmental increase in right-ear advantage. In the visual modality, young subjects may not want or be able to keep their gaze at the fixation point. Some functions may not have developed in young children, and cannot be in their processing or response repertoires. When novel, unknown, or difficult letters or words are being read for the first time, left-visual-field (right hemisphere) advantage may initially occur even in adults. This is the consequence of involvement of the right hemisphere in preprocessing novel tasks, and reading is in itself a novel task for very young children—unfamiliar and difficult as well. It is difficult to design suitable tests for nonverbal right-hemisphere processing in young subjects. Even in adults it is usually more difficult to demonstrate nonverbal right-hemisphere superiorities. But even in that area there is no hard evidence of developmental effects in lateralization. Some authors claim, on the basis of behavioral tests, that the right hemisphere matures more slowly than the left, although other evidence (e.g., anatomic) suggests just the opposite. However, there is plenty of anatomic evidence for continuous developmental myelination of the corpus callosum and possibly also of the inferior part of the parietal lobe until puberty (Bradshaw & Nettleton, 1983). It may be concluded that the hemispheres are not equipotential at birth and there is no solid evidence in favor of the developmental nature of asymmetries, which would suggest that asymmetries might reach their peak in puberty or even at age 5. A developmental decrease of plasticity can explain better recovery from early aphasia, but that phenomenon is not correlated with the development of lateralization. Later on, until puberty, the left hemisphere becomes firmly involved in language, mostly as a consequence of the considerably decreased neural plasticity. Hemispheric specialization increases with age only with respect to the child’s repertoire of asymmetrically organized functions. These functions probably emerge at some crucial developmental periods, such as puberty (hormone-dependent) and possibly around the fifth year of life (when language is stabilized). It is possible that there is different hemispheric development related to these (and possibly some others) critical periods, which may also interact with a general decrease in neural plasticity (Gazzaniga et al., 2002). Lateralization of functions has its neural substrate as well.
Lateralization and Localization of Functions
119
NEUROANATOMIC ASYMMETRIES Even in 29-week-old fetuses the left planum temporale is larger than the right in about 90% of cases (Wada, as cited in Spreen at al., 1984), and the Sylvian fissure is longer on the left side than on the right. Magnetic resonance imaging revealed a high correlation between the absence of this asymmetry and the occurrence of developmental dyslexia in children (Gazzaniga et al., 2002). In the left hemisphere the Sylvian fissure has a less prominent upward curl than in the right, and this is noticeable in 16-week-old fetuses. As a consequence, the parieto-temporal region located ventrally to the Sylvian fissure is larger on the right, and such an enlarged area has a particular role in sensory integration of spatial characteristics. The inferior part of the postcentral gyrus (parietal operculum) that belongs to Wernicke’s area and receives kinesthetic input from the articulators is also larger in the left hemisphere. Habib, Robichon, Levrier, Khalil, and Salamon (1995) found that, ontogenetically, parietal operculum asymmetry precedes planum asymmetry and that the highest found positive correlation with right-handedness is parallel enlargement of the parietal operculum and planum temporale in the left hemisphere. It has to be pointed out, however, that modern three-dimensional imaging techniques (MR) have revealed that the surface of that cortical region is larger in the left hemisphere in 5 out of 10 subjects, and in 5 out of 10 it is larger in the right hemisphere (Gazzaniga et al., 2002). In the right hemisphere there is frequently an additional Heschl’s gyrus, whereas in the left hemisphere there is only one. Kolb and Whishaw (1996) consider this complementary asymmetry between the two hemispheres to be a possible anatomic foundation for the functional division of labor between the temporal lobes into language (left hemisphere) and music (right hemisphere) functions. The location, direction, extent, and consistency of these morphologic asymmetries indicate that they may be the neurological substrate of language (Bradshaw & Nettleton, 1983). Prefrontal areas, which include Broca’s area for speech production, seem smaller on the left side, because the visible brain surface is larger on the right by about one third. However, given the greater degree of folding in the left hemisphere it is highly probable that the total cortical surface is greater in the left than in the right hemisphere (Bradshaw & Nettleton, 1983; Kolb & Whishaw, 1996). This different organization probably reflects functional differences between homologous areas of the left and the right hemisphere, the left being involved in speech production, and the right in tone of voice (Kolb & Whishaw, 1996). There is more tissue in the right hemisphere and its mass is greater. The left occipital pole protrudes toward the back more than the right. Anterior parietal and posterior occipital areas are commonly larger on the left. The right frontal pole and the central (prefrontal) part of the right hemisphere are commonly wider than on the left, and the right frontal pole is usually longer than the left. Wider and longer anterior parts of the right hemisphere and posterior parts of the left give the human brain a torqued look in counterclockwise direction (Bradshaw & Nettleton, 1983).
120
The Cognitive Neuroscience of Human Communication
Density of the left hemisphere is greater than that of the right and the ratio of gray and white matter is greater in favor of gray in the left hemisphere, particularly in frontal and precentral areas. This means that one of the right-hemisphere characteristics may be better connections among different areas, as opposed to better within-area organization in the left hemisphere. It is believed that a consequence of these differences is the left hemisphere’s greater capability for unimodal processing and specific representations of sensory and motor functions, contrary to the right hemisphere that has more association cortex. This ensures multimodal processing; hence, one of the most prominent differences between the hemispheres is their mode of action: The right is adapted for holistic processing, and the left for analytical (Bradshaw & Nettleton, 1983). Asymmetries are visible at lower levels as well. For example, in about 80% of the cases, more neuronal fibers connect motor cortical areas in the left hemisphere via contralateral pathways with right extremities, as well as motor areas of the right hemisphere via ipsilateral pathways also with the right extremities, than is the case with the left. In other words, the right hand is better innervated by motor fibers than the left, which may explain such a high proportion of righthanders in the population (Bradshaw & Nettleton, 1983). The main artery in the left hemisphere is located near speech and language areas, whereas in the right hemisphere it is in the superior part of the parietal lobe that is involved in spatial information processing. Distribution of various neurotransmitters is asymmetric at cortical and subcortical levels (Kolb & Whishaw, 1996) as is the cell appearance, as for example, their columnar pattern (Gazzaniga et al., 2002). At present there is not much evidence about systematic asymmetries at subcortical levels, with the exception of the thalamus. There is electrophysiological evidence of the asymmetric participation of the thalamus in language functions (Bradshaw & Nettleton, 1983; Kolb & Whishaw, 1996; Sternberg, 2003). Having listed the most obvious neuroanatomic (and cytoarchitectonic) asymmetries, we should emphasize several points of caution: (a) a doubtless correlation between the size of an area and its functional significance cannot always be established; (b) successful performance of a function does not necessarily require more tissue, larger area, and so forth; (c) it seems that cortical mass is not as important as the specialized circuits that are localized in one hemisphere or the other; (d) boundaries and limitations of cortical asymmetries are not straightforward or easy to define. Finally, if the three-dimensional reconstruction algorithm that takes into consideration the natural curvature of the cortical surface is applied to the posterior temporal lobe, the area is not in fact asymmetrical (Gazzaniga, 1994).
SENSORY ASYMMETRIES Olfaction is the only predominantly ipsilateral sensory modality. All other senses are bilaterally represented in the brain.
Lateralization and Localization of Functions
121
In audition there is the greatest degree of doubling of hemispheric functions—the stimulus from each ear projects to both sides of the brain, so that approximately two thirds of the information is relayed to the contralateral hemisphere and one third to the ipsilateral one. Chen, Halpern, Bly, Edelman, and Schlaug (2000) found a significant correlation between asymmetry in planum temporale and absolute pitch. If exposure to music and music training start early (preferably before age 7) the probability of developing absolute pitch is greater in individuals with greater asymmetry in favor of the left hemisphere (i.e., in those whose right planum temporale is considerably smaller than the left). Vision is more clearly lateralized to the side opposite to the peripheral sensory organ. Somatosensory modality is mostly contralateral, with pain and temperature having both ipsi- and contralateral components. In the tactile modality, it has been found that the left side of the body (hand, sole, etc.) has a lower absolute threshold for pressure sensitivity than the right. Studies have revealed superiority of the right hand for sequential tasks and of the left for simultaneous tasks. If two tasks controlled by the same hemisphere are performed concurrently (e.g., verbalization and right-hand movements) there is interference (Bradshaw & Nettleton, 1983; Mildner, 2000). Some exceptions have been reported for female subjects and left-handers (Hicks, 1975). Left-hand movements may even improve concurrent verbalizations (Kinsbourne & Cook, 1971). Motor control is mainly contralateral, especially for precise finger movements.
MOTOR ASYMMETRIES Archeological findings indicate, judging by the earliest traces of tools and paintings dating to the Paleolithic time, that most humans were right-handed (Bradshaw & Nettleton, 1983). The ontogenetically earliest examples of motor asymmetry can be seen in the walking reflex of newborns. They may also be manifested as more frequent head turning to the right while lying on their back. It has also been found that more newborns turn their head to the right when touched on their right cheek, than to the left as response to touching of their left cheek. Children prefer one hand for reaching and holding objects. This hand preference is fixed after the third year of life; earlier fixed asymmetry may be a sign of some pathology (Bradshaw & Nettleton, 1983; Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Different theories of hand preference may be grouped into those that stress impact of the environment, structure, hormones, or heritage (Kolb & Whishaw, 1996). A very high correlation was found between the preferred (dominant) hand and the preferred (dominant) foot. Such dominance may be recorded by asking subjects to fill out questionnaires about which hand or foot they prefer in some everyday activities (brushing teeth, lighting a match, kicking the ball, etc.) or by observing them perform these activities. Searleman (1980) found that the correlation between language function lateralization and the dominant (preferred) foot is higher than is the case with the dominant (preferred) hand. He attributed this to the fact that left-handedness is still stigmatized in some societies, and foot dominance
122
The Cognitive Neuroscience of Human Communication
does not attract as much attention. It is, therefore, foot dominance that is a true measure of motor asymmetry, because it is not influenced by social conventions and expectations. He corroborated his claim with the fact that subjects who had reported left-foot preference had the greatest percentage of left-ear advantage. Various methods have shown that motor asymmetries in favor of left extremities do not necessarily change language function lateralization. Left-hemisphere dominance for language is more widespread than right-handedness, and reversed dominance is rarer than left-handedness. However, right-handers exhibit more consistent lateralization of language functions to the left hemisphere, and lefthanders exhibit less stability of left-hemisphere dominance in test–retest situations involving visually or auditorily presented language material. In general, the contribution of the right hemisphere is more probable in left-handers than in right-handers. Their language functions recover faster after left-hemisphere injury but they are more prone to dysphasia after right-hemisphere damage. Some authors have found that the degree of right-handedness is more important for prediction of language lateralization than its direction. Data on the influence of familial sinistrality on language lateralization are inconclusive. If such influence exists, it is more noticeable in left-handers than in right-handers. Similarly, a reliable correlation between parents’ dominance of extremities and that of their children has not been established. Levy and Reid’s (1976) hypothesis about hand posture during writing as an indication of language representation has not been definitively confirmed either.
ASYMMETRIES IN OTHER SPECIES Many animals exhibit foot or paw preferences. These asymmetries are mostly unsystematic with respect to members of a species (the ratio is approximately 50:50), and frequently within one and the same member across tasks: In some activities the animal will prefer the right foot or paw, in others the left. With respect to the structural cerebral asymmetries, it was found that rats have a thicker cortex in the right hemisphere than in the left. Various experiments have shown that their right hemisphere is important for complex tasks, new situations and aggression, and that it is more under the influence of environmental factors. Studies of birds have revealed that left-hemisphere damage impairs their song, but that their brains are plastic as well, because their right hemispheres can take over control of song if damage occurred before it has developed completely. Plasticity is also manifested in their ability to acquire a new song repertoire each year (Bradshaw & Nettleton, 1983; Nottebohm, 1994). Chickens and pigeons solve different types of tasks if they are presented only to the left or to the right eye (Gazzaniga et al., 2002). In apes, neuroanatomic asymmetries similar to humans’ have been found: a longer and less steeply sloping left Sylvian fissure, and in other mammals the right hemisphere is also heavier than the left. Anatomic asymmetries found in other species seem to support the claim that human brain asymmetry is not directly related to language or the preferred hand (Kolb & Whishaw, 1996).
Lateralization and Localization of Functions
123
FACTORS INFLUENCING FUNCTIONAL CEREBRAL ASYMMETRY Many factors cause and modify laterality. They may increase or decrease effects of laterality, change their direction or eliminate them entirely. Efron (1990) listed about 30 such factors and grouped them along criteria of subjects (e.g., age, sex, dominant hand, education, culture, language, processing strategies, motivation, attention, skill, etc.), stimuli (size, complexity, amount, difficulty, duration, contrast, degree to which they can be verbalized, etc.), or research method (detection, response time, recognition, event-related potentials, etc.). It has been shown that laterality effects (i.e., greater activity of one or the other hemisphere in a particular task) depend on many other details: familiarity with the task, instructions, type of material, kind of response required, sensory modality, and many others. Their very number and diversity, and the fact that so many of them are difficult or impossible to control in any given task, even when we are aware of them, makes one wonder how much experiments aimed at revealing hemispheric specialization for that particular task in fact tell us about the activity and not about some of the listed factors. Besides, each specific task or area of research brings in additional variables (more on that in the section on bilingualism). Carpentier, Pugh, Studholme, Spencer, and Constable (2000a) conducted fMRI studies of brain areas that might be affected by the sensory modality. They defined common, modality-independent areas that probably perform more abstract calculations (e.g., syntactic or pragmatic processing), and those that are modality-specific. These specialized areas (e.g., Wernicke’s area in the right hemisphere, left fusiform gyrus, Broca’s area in Brodmann’s area 44, supramarginal gyrus) seem to carry out procedures that are important for establishing connections between abstract language dimensions, which increases knowledge and skill of the entire language network. Their results indicate that the area in the temporal lobe of the right hemisphere that is homotopic to Wernicke’s area in the left hemisphere plays a special part in speech recognition. Furthermore, they have shown that, depending on whether the stimulus was auditory or visual, processing is different in Brodmann’s areas 44 and 45 (although they both constitute Broca’s area) and in some other regions. In short, they have shown that in the visual modality, functional asymmetry, manifested as language lateralization, is more marked than in the auditory modality. A general conclusion that may be drawn from their research might be that language processing does not proceed within a unique network, but in several networks whose action is combined in order to improve sentence processing. In other words, language is built of simpler components whose combinations determine specific types of complexity and diversity. Gender differences in cognitive tasks have been studied and described extensively, particularly with respect to verbal and spatial processing. Men seem to be better at spatial tasks and women outperform men on phonological tests. Earlier studies using fMRI have revealed differences in brain activation during phonological processing: bilateral activation was found in women and primarily left-lateralized activity was recorded in men. Gur et al. (2000) compared men and women
124
The Cognitive Neuroscience of Human Communication
on two tasks—verbal (producing analogies to presented stimuli) and spatial (estimation of line orientation). During verbal processing, expected changes in lefthemisphere activity, particularly in the inferior temporal lobe and in the planum temporale, were found in all subjects, but only men exhibited increased activation of these areas in the right hemisphere during spatial tasks. The network of cortical regions that were activated by these tasks included lateral and medial frontal, medial temporal, parieto-occipital and occipital regions. Activation was greater in the left hemisphere during verbal tasks and in the right during spatial ones. Men differed from women in their activity of the left hemisphere during spatial tasks. Making the tasks more difficult resulted in wider activation during verbal tasks, but in more limited activation during spatial tasks. The authors concluded that insufficient activation in a particular hemisphere in regions directly involved in task performance accounts for sex differences in various tasks, and sex differences in spatial tasks are based on bilateral activation in a distributed cognitive system. Sex differences are discussed in more detail in a separate chapter. Differences in ear or visual field advantage may occur (even in the same subjects and for the same stimuli) as a consequence of a forced or self-imposed processing strategy, which implies that the manner of stimulus processing is more important than the stimulus itself. However, characteristics of the stimulus may also be crucial. Differences in visual-field advantage may depend on image sharpness, duration of exposure, retinal eccentricity, frequency range of its constituents, and so forth. In the auditory modality, Hagberg et al. (2000) have recorded by PET that listening to a regular series of pulses (e.g., drum beating) elicited activation in the left superior temporal gyrus (Brodmann’s area 40), as opposed to the irregular series that elicited bilateral activation in Brodmann’s areas 40 and 43, as well as in the right cerebellar hemisphere. The importance of attention in dichotic listening became clear as the first studies were done in 1950s. It is discussed more comprehensively by Mondor and Bryden (1992) and Hugdahl et al. (2000). They checked the effects of attention by PET recordings of changes in brain activity of healthy subjects during listening to music (instrumental) and speech (CV syllables) stimuli in different attention conditions: when it was directed to the right ear, to the left ear, or to both ears simultaneously (divided attention). Directing attention to one ear or the other significantly reduced the activity in both temporal lobes, compared with the condition of divided attention. Syllables activated the classical language areas (Broca’s and Wernicke’s) and music stimuli activated areas in the visual association cortex, cerebellum, and hippocampus. They interpreted these results as a facilitatory effect of attention on auditory processing because the activity of the primary auditory cortex is decreased when attention is consciously manipulated. Increased activity in the parietal lobe, which was also recorded during the tasks, was attributed to activation of the attentional network that is common to all modalities. The significant effect of attention was also found by Mildner (1999) in a dichotic listening task. ERP studies have shown that auditory stimuli that are attended to elicit signals of higher amplitude than those that have been ignored. This difference is
Lateralization and Localization of Functions
125
manifested as the first large negative polarity (auditory N1 potential). Its latency is somewhat less than 90 ms, but even shorter latencies have been recorded. Hillyard and coworkers (as cited in Gazzaniga et al. 2002) found, using MRI, greater sensory activity elicited by auditory stimuli in the primary auditory cortex that receives stimuli from the attended side as early as 20 to 50 ms after stimulus onset. These results support early selection attention theories. According to these theories, the stimuli to which attention has been directed are facilitated, and the others are inhibited in early processing stages, before more complex stimulus features are to be processed in higher cortical areas. The time it takes the auditory stimulus to reach the cerebral cortex is approximately 20 ms, and the greatest activity was recorded 20 to 50 ms after stimulus onset in Heschl’s gyrus. It has to be emphasized that intentional, conscious manipulation of attention does not affect the subcortical level of the auditory pathway. Alho et al. (1999) also reported the influence of attention on brain activation. Their PET scans showed that attending to the right ear activates the auditory cortex mainly in the left hemisphere, and vice versa. This selective activation of one or the other hemisphere, depending on the side of stimulus presentation, is mediated by executive control mechanisms in the frontal lobes, where increased activity related to auditory attention was in fact recorded. Manipulation of attention was found to be important in some concurrent verbal-motor tasks with auditorily presented language material, more so for native than for foreign language (Mildner, Stankovi´c, & Petkovi´c, 2005). Similar results were obtained for visual stimuli. Although they were present, differences in early activity recorded in the primary visual cortex (Brodmann’s area 17) with 50 to 80 ms latencies, were less affected by spatial attention than differences in later (80 to 200 ms) activity found in extrastriate visual areas. Attending to other, nonspatial, object features (e.g., to color or orientation) produced no reliable results, but it has shown that directing attention to a particular isolated object feature (e.g., color) may affect the areas in the extrastriate cortex that are specialized for processing these features. This is more evidence in support of the claim that selective attention influences perception before the analysis of all features has been completed. Despite the effects of selectivity, attention facilitates processing of all features of the attended object. This is explained by the fact that attention facilitates synchronization of neuronal firing, which makes processing more efficient. Clinical electrical cortical stimulation of human patients has confirmed that selective attention affects the activity of the corresponding sensory modality. Even reflex attention has similar effects to intentional manipulation of attention, but the neural mechanisms are not quite the same (Gazzaniga et al., 2002). PET and fMRI techniques have also revealed attention-dependent activity in areas other than those where particular sensory modalities are represented: in the thalamus, basal ganglia, insula, frontal cortex, anterior cingulate, posterior parietal lobe, and temporal lobe. The parietal lobe was found to play a very important role in attention manipulation. It is the seat of representations of spatial relations, it controls intentional orientation toward a relevant location in space, and it connects subcortical structures (particularly pulvinar) with the temporal lobes.
126
The Cognitive Neuroscience of Human Communication
The most frequent attentional deficits are neglect and extinction. They are typically a consequence of brain damage (particularly of the parietal lobe) and affect the side contralateral to the side of the injury. It has been shown that even stimuli presented to the otherwise neglected field are processed without awareness about it. For example, a meaningful word presented to the neglected visual field will be processed at the semantic level without the patient’s awareness of even seeing the word. Details of a familiar scene or space (a room or a city square) that the patient may not have remembered, from his/her perspective at the moment of description, on the neglected side, will be included in the description if the patient’s imagined position is changed. These examples corroborate the claim that neglect is an attention deficit rather than a memory or processing disorder. Extinction is a milder form of attention deficit. It refers to the state when a person is able to register stimuli in either side of the space, regardless of injury side, but if two stimuli occur at the same time, one on each side, the patient does not attend to the one contralateral to the injured side (Judaš & Kostovi´c, 1997). It should be pointed out, however, that the term extinction may be defined differently in some psychology textbooks, where it refers to the gradual decrease in conditioned response intensity under certain circumstances (Zarevski, 1994). Garavan et al. (2002) found that similar brain areas are activated when switching attention within one modality (verbal or visuospatial) and from one modality to another. These areas include the dorsolateral prefrontal cortex, anterior cingulate, the area immediately anterior to the supplementary motor area, bilateral premotor cortex, bilateral inferior temporal lobulus and the precuneus. In addition to these areas, intermodality switching of attention activates the right middle frontal gyrus (Brodmann’s areas 9 and 10). Dubost, Beachet, Najafi, Aminian, and Mourey (2002) concluded that allocation of attention is different for working memory than for semantic memory. Signal processing is an active process at almost all levels, and it depends considerably on the nature and requirements of the task at hand. Perceptual mechanisms, regardless of modality, are not mere processors of sensory data—processing is modified according to the situation. The final result, be it behavior or a percept, is a whole. It is not quite clear whether this whole is a product of constant interaction between the two hemispheres or whether each hemisphere processes the stimuli on its own, according to its own strategies, style and preferences, with the final outcome being the result of the activity of the hemisphere that was more successful in the competition. In any case, hemispheric specialization is one of the essential characteristics of cognitive processes. All evidence suggests that the differences between the hemispheres are relative, quantitative, and a matter of degree, rather than absolute or qualitative. The degree of specialization may vary from one person to the next and may depend on gender, preferred hand, or literacy, among other factors. Visible consequences of injuries seem to support the model of exclusive hemispheric specialization, whereas laterality differences in healthy subjects probably reflect the time needed for interhemispheric transfer and/or signal degradation during such transfer. Severe clinical consequences may be a result of the existence of intact interhemispheric
Lateralization and Localization of Functions
127
connections that are inhibitory rather than facilitatory. For example, after removing the damaged hemisphere or a portion of it, or after severing the connections with the left hemisphere, the right hemisphere may realize its language potential (that is in fact considerable) more efficiently. In healthy subjects, the inhibitory influences probably take care of rivalry between the hemispheres in processing identical types of stimuli. Split-brain studies reveal the quantitative and relative nature of functional differences: each hemisphere is capable of functioning in any processing mode, but at different competence levels and in different directions. For example, even left-hemisphere superiority in processing stops decreases if the timing is slowed down. In healthy subjects, the intensity or temporal advantage given to the inferior ear or visual field does not eliminate or change laterality effects, as would be expected if laterality differences depended on interhemispheric transfer from the incompetent to the exclusively functional hemisphere. Quantitative asymmetry may seem qualitative if it is large enough, if analytic processing requires greater processing capability than holistic, and if the left hemisphere is more powerful in arithmetic operations than the right in its early development, regardless of the reasons for such ontogenetic differences. Finally, it should be stressed that many brain functions are symmetrical, particularly in the primary sensory and motor areas. In many functions, intrahemispheric differences are greater than the interhemispheric ones. For example, although frontal lobes are asymmetrical, the functions of the left and the right frontal lobe are more similar than the functions of different lobes in the same hemisphere. Furthermore, the differences between the two hemispheres are relative rather than absolute. It should be pointed out again, that asymmetry does not mean exclusivity. Both hemispheres are generally capable of and involved in performing any cognitive task, but their contributions may differ. Although experience and the environment may influence the manner of stimulus processing, there is no evidence that the environment changes the organization of cerebral hemispheres. Having discussed the pitfalls of oversimplification and the nuances of functional asymmetries, we can nevertheless summarize that in the gross cerebral organization, with respect to lateralization, the left hemisphere is primarily dominant for language and the right is dominant for spatial processing.
LOCALIZATION OF FUNCTIONS Localization of functions refers to the possible neurophysiologic substrate of sensations or behavior. The notion closely related to it is that of cerebral representation of sensations and/or functions, in other words, defining regions of greatest cerebral activity while performing an action or experiencing a sensation as a result of an external stimulus. For example, it is commonly said that the phonological store is represented at the junction of the parietal, occipital, and temporal lobes. Different brain study techniques, among the first the cortical stimulation method, have revealed organized, topographic maps in the cortex. For example, body surface is represented in the postcentral gyrus; the tonotopic organization present in the cochlea is preserved in the temporal lobe so that high frequencies
128
The Cognitive Neuroscience of Human Communication
# # "
$ #
! $ Lateral
Medial
FIGURE 6.1. Human primary somatosensory cortex—mapping of periphery onto the postcentral gyrus. (From Purves et al., 2001. With permision.)
are represented more posteriorly in relation to the more anteriorly represented low frequencies, and so forth (Fischbach, 1992; Levy et al., 2005; Pinel, 2005). The area of the corresponding primary sensory cerebral cortex is directly proportional to the density of innervation of peripheral areas (Figure 6.1; the homunculus represents the relative size of the particular cortical areas devoted to corresponding body parts; Purves et al., 2001). In other words, the most densely innervated peripheral areas (e.g., lips, palms, and soles) take up the relatively greatest portion of the primary sensory cerebral cortex (Penfield & Rasmussen, 1950). The effectors are represented in the primary motor areas located in the precentral gyrus. The size of the area in which an effector is represented does not reflect its size, but rather its importance and the amount of control necessary to manage it. That is why, for example, fingers are represented in a relatively greater
Lateralization and Localization of Functions
129
# "
$ #
! $
Lateral
Medial
FIGURE 6.2. Human primary motor cortex—mapping of periphery onto the precentral gyrus. (From Purves et al., 2001. With permission.)
cortical area than toes (Figure 6.2; Purves et al., 2001). Each cerebral hemisphere controls the opposite side of the body, and each cerebellar hemisphere controls the ipsilateral side, due to the double crossing of pathways. Different authors disagree about the number of identifiable brain areas. The number varies between several to about 200. All agree, however, on the basic sensory (auditory, visual, and areas onto which body surface is mapped) and motor areas. Whereas the anatomy of the principal motor and sensory systems is quite well known, the patterns of connections among association cortices and large subcortical nuclei have not been determined. It has become clear that primary sensory areas are not clearly separated, with association areas between them, as previously thought. The majority of these so-called association areas include, in fact, many specific sensory representations (Bradshaw & Nettleton, 1983; Kalat,
130
The Cognitive Neuroscience of Human Communication
1995; Levy, Koeppen, & Stanton, 2005). Sensory and motor processes involve surprisingly many areas, richly connected with each other and with the thalamus. It is interesting that the basic organization of primary sensory and motor areas in the cerebral cortex is very similar in all mammals. However, with the higher place on the mammalian evolutionary scale, the absolute brain size and the relative amount of association cortex increase (Thompson, 1993). Cortical maps are not fixed. Their appearance and organization may be affected by onset and course of illness, as well as recovery of function after injury, normal learning, involving changes in peripheral receptors or nerves (more on that in the section on plasticity). Cortical areas that no longer receive stimuli from the original peripheral organ or modality are taken over (flooded) by representations from neighboring organs or other modalities. For example, disconnection between a finger and its cortical representation results in the reaction of that area to stimuli arriving from the neighboring finger; tonotopic organization may be modified by increased exposure to selected tones, so that the representation of the practiced tones will cover a larger area; partial injuries to the retina may result in reorganization of the visual cortex. The most obvious cross-modal changes can be found in the multimodal areas of the superior temporal lobe that receives visual and auditory stimuli anyway (Gazzaniga et al., 2002), but functional reorganization is possible in other sensory combinations (vibrotactile aids in hearing impaired persons are based on this). Neuroimaging methods (e.g., PET, SPECT) enable us to visualize how many different areas in both hemispheres are metabolically active during cognitive processes. The maps obtained for particular activities reveal that neural circuits underlying the cognitive processes are located in anatomically separate areas, but also that these areas may be activated by very different cognitive functions. Even before such methods became available for research in healthy subjects, there had been clinical reports of identical functional disorder after damage to different parts of the brain in different patients, as well as cases where focal lesions to a specific area had not necessarily caused identical functional deficits in different patients. Obviously, complex cognitive activities, such as language or spatial data processing, cannot be simply and straightforwardly related to strictly localized areas in the brain. These and similar results have given rise to attempts to discover and interpret different contributions of the two hemispheres to the common task, rather than to rigidly categorize activities into left-hemisphere and right-hemisphere. The highest correlation between lesion location and cognitive dysfunction was found for language. More than 95% of patients who have suffered a stroke in the left fronto-parietal region are bound to present with some kind of aphasia. This high correlation definitely points to the importance of the region for language, but it does not necessarily imply that it is the exact or the only seat of language functions. This argument is supported by cases of aphasics without injury to language areas and cases of patients with extensive damage to the regions without aphasic symptoms. Obviously, despite the importance of this region that has been confirmed by numerous cases, other brain areas that are active in the process should not be neglected. Similarly, it has been shown that spatial representation
Lateralization and Localization of Functions
131
is a result of joint functioning of both hemispheres. Due to these and similar examples, Efron (1990) proposes the term support rather than specialization, it is more accurate to say that certain parts of the brain or neural circuits, for which the correlation between location and function has been established, support that particular function—it is not a matter of specialization. The human auditory cortex is very sensitive to small changes in the auditory stimulus. Rupp, Hack, Schneider, Stippich, and Scherg (2000) found, using the MEG technique, that interruptions of the auditory signal as brief as 3 ms have their neural representations in the brain. In their fMRI studies of healthy subjects Hall et al. (2000) found that complex acoustic stimuli activate secondary auditory areas more than the simple ones, just as it had been found in electrophysiological animal studies. They compared differences in activation between modulated and static tones and between complex and simple tones and found that in all cases the primary auditory cortex and neighboring areas, including the planum temporale, were activated bilaterally, although differences associated with signal complexity were not statistically significant. Ducommun et al. (2000) found in their ERP study that sound movement and location are processed in different cerebral areas. Of the 22 maps obtained by measuring the brain’s electrical activity, 15 represented auditory stimulation in general, three represented sound location and were located mainly in the temporal lobe, and four represented movement and were located mainly in the frontal lobes with some activity in the temporal lobe. Stimulation of the anterior part of Heschl’s gyrus elicits simple auditory sensations and damage to that area may cause deafness. When stimuli are presented posteriorly, anteriorly, or laterally to the auditory area, different sensations occur. Sounds are received more frequently by the language nondominant temporal lobe (usually right). Stimulation of the speech cortex immediately next to the auditory cortex actually inhibited speech (aphasia) but did not cause any vocalizations. Listening to word lists, simple sounds, including the 1,000 Hz pure tone and music, and silent reading of visually presented words elicited activity in the temporal gyrus. It was found that: (a) responses of all neural units were speech-specific, namely, they were not evoked by simple stimuli; (b) semantic content was irrelevant, in other words, the unit could respond to various words, differing in temporal and spectral content; (c) attention was an important variable as opposed to comprehension, and soft speech evoked no response. Finally, music was found to suppress the activity of the examined population of units. Motor cortex functioning was studied in primates by electrical cortical stimulation. The studies revealed that their cerebral cortex was organized into efferent microzones, each of which represented an individual muscle or a basic move, but also that the response of individual muscles or muscle groups that work together (in synergy) may be elicited by stimulation of several different nonneighboring areas of the motor cortex. Similar results were obtained when brain activity during voluntary hand movements was recorded by fMRI. Contrary to traditional views that motor functions of different parts of the body were systematically mapped onto the motor cortex (Figure 6.2), overlapping cortical representations of finger and wrist movements were found, as well as multiple representations of all fingers
132
The Cognitive Neuroscience of Human Communication
and the wrist in the precentral gyrus. Multiple efferent microzones were found in regions of the primates’ motor cortex where mouth and face muscles were represented. Electric cortical stimulation revealed two very important facts: (a) face representation in the motor cortex partly overlaps with and surrounds the area where jaw and tongue movements are represented; and (b) besides the expected contralateral facial movements, microstimulations elicited ipsilateral movements as well, which suggests a certain degree of bilateral control (for a brief review see Kent & Tjaden, 1997). By analogy to human behavior, particularly with respect to the consequences of brain injury, these findings may explain observations that a spatially circumscribed injury does not necessarily result in a severe and irreversible functional disorder. Fabri, Polonara, Salvolini, and Manzoni (2002) showed that in humans, similarly to nonhuman primates, parts of the skin along the body midline are represented bilaterally in the primary somatosensory cortex. A very noticeable mapping of peripheral organs onto the cerebral cortex was found in rodents. Each vibrissa in the snout is represented by a column of its own in the somatosensory cortex (Cybulska-Klosowicz & Kossut, 2002). Butler et al. (2000) found by PET that reaching for objects that had previously been presented kinesthetically (by touch) or visually, activates cerebral structures that are associated with these modalities in both hemispheres: the sensorimotor cortex, supplementary motor area, and the cerebellum for the kinesthetically presented stimuli, and in the primary and the secondary visual cortex for the visually presented stimuli. Interhemispheric differences were recorded with respect to the hemispace in which the stimulus was located. Regardless of modality, when the stimulus was in the left visual field (i.e., on the left side of the body) sensorymotor, parietal, and occipital areas in the right hemisphere (and the left cerebellar hemisphere) were activated. When the stimulus was on the right side of the body (i.e., in the right visual field) activation was more bilateral. This confirmed the importance of the right hemisphere in processing kinesthetic spatial information and right-hemisphere dominance in visuospatial processing. As stated previously, spatial data are processed mainly in the right hemisphere (in the inferior temporal and superior frontal lobe, specifically in the angular and superior precentral gyrus). Bilateral activation was recorded in the lingual/parahippocampal region only when the task was to determine the position of a given object with respect to a reference point in the surroundings (allocentric processing), but not in other spatial tasks (Committeri et al., 2002). Studies of metabolism showed that the spatial representation mechanism includes neural networks with inputs from cortical and subcortical structures (Myers, 1999). Grube, Ruebsamen, and Von Cramon (2002) found that, regardless of a sound spectral composition, left-hemisphere injured persons had only partially impaired ability to determine its location in space, whereas right-hemisphere patients had serious difficulties in such tasks. This confirmed the functional importance of the right primary auditory cortex in sound localization. Clarke et al. (2002), like many other authors, have shown that in healthy subjects identification and discrimination of auditory stimuli are supported by
Lateralization and Localization of Functions
133
neural networks different from those supporting sound localization. During recognition (identification and discrimination alike), the middle temporal gyrus and the precuneus in both hemispheres, as well as the posterior part of the left inferior frontal gyrus, were more active than during sound localization. Apparently, these structures constitute the “what” path. Contrary to that, sound localization activated the inferior part of the temporal lobe and posterior parts of the middle and inferior frontal gyri in both hemispheres more than its recognition, which implies involvement of these structures in the “where” path. In patients with right-hemisphere focal injuries, auditory stimuli activated the spared areas of the right hemisphere. Both types of tasks activated Heschl’s gyrus and varying amounts of the surrounding cortex in the intact left hemisphere (albeit less than in healthy subjects), but not the pathways specialized for one or the other type of task. All patients had difficulties in sound localization. It seems that right-hemisphere damage significantly impairs auditory processing of the opposite, healthy, left hemisphere, particularly its specialized networks. Nirkko, Baader, Loevblad, Milani, and Wiesendanger (2000) found that violin playing, and possibly music production in general, involves the area of frontal operculum in both hemispheres. Parsons et al. (2002) found, by using PET, that music production (e.g., finishing a tune, harmonizing) in nonmusicians activates the midbrain, left insula, cerebellum, basal ganglia, left middle temporal lobe (Brodmann’s areas 21 and 39), parts of the medial frontal lobe (Brodmann’s areas 6 and 25), and the right inferior frontal gyrus (Brodmann’s area 47). Harmonizing activated some specific areas in the right hemisphere (Brodmann’s areas 20, 40, and 45). Neither the control speech task (finishing incomplete sentences) nor mere repetition of melodies has activated these areas. Most patients who suffer from prosopagnosia (inability to recognize faces) have lesions in the occipital and/or temporal lobes, as revealed by fMRI and other neuroimaging techniques. There are two areas in the temporal lobe that are activated more during recognition of faces than of any other objects: superior temporal sulcus and inferior temporal gyrus. Furthermore, the ventral surface of the temporal lobe (fusiform gyrus, particularly in the right hemisphere) is activated more by faces than by inanimate objects or meaningless forms (patterns). In the left hemisphere there was no difference between reactions to faces and reactions to inanimate objects. Although some other stimuli (e.g., pictures of birds or cars) elicited activity in the area that is considered responsible for face recognition, a series of letters never evoked any considerable activation in the area. These stimuli activated primarily the left hemisphere, its inferior part of the occipital lobe, along the border between the occipital and the temporal lobe (Gazzaniga et al., 2002). Patients with frontal lobe and white matter damage subsequent to traumatic brain injury have severe problems with attention and concentration. This is particularly obvious when concentrated attention is required during a longer period of time. A 0.58 correlation was found between the severity of the injury and problems related to inability of sustained attention at the level necessary to continually perform a task successfully. Different kinds of memory are represented differently. This issue is discussed in more detail in chapter 7, this volume.
134
The Cognitive Neuroscience of Human Communication
On the basis of the observations of autistic children, Baron-Cohen concluded that theory of mind probably has a cerebral representation of its own. His model proposes that it may be distributed, involving processing of emotions (probably in the amygdala) and some aspects of face processing (probably the superior temporal sulcus). Two studies (one SPECT and one PET) using different tasks suggest the activity in the right orbito-frontal cortex and the left medial frontal cortex (Gazzaniga et al., 2002).
LATERALIZATION AND LOCALIZATION OF EMOTIONS Expressing one’s own emotions and reacting to the emotions of others are important components of speech communication. Early 20th-century authors who studied emotional functioning emphasized the separation of emotions from other functions—primarily cognitive ones—and believed that emotional activity may be limited to the limbic system. These views were modified in the 1980s, due to an increasing body of evidence suggesting that some parts of the limbic system are functionally closer to cognitive than to emotional activities (e.g., the relationship between the hippocampus and learning and memory), and that processing of emotions is not restricted to the limbic system. The analysis of neural substrates of emotions revealed that neural systems supporting emotional functioning are both dependent and independent of those supporting cognitive actions. Certain structures are specific to emotional functioning, particularly the amygdala and the orbito-frontal cortex. The amygdala are a small almond-shaped group of neurons, immediately anterior to the hippocampus in both hemispheres. They are involved primarily in processing negative emotions (e.g., fear), but they also participate, to a somewhat lesser degree, in responses to positive and neutral stimuli. They play a very important role in learning and memory. The orbito-frontal cortex is a part of the prefrontal cortex that constitutes the base of the frontal cortex and is adjacent to the upper orbital wall just above the eyes (Brodmann’s area 47). Its main task is to manage and control our actions in social and emotional situations. Berthoz et al. (2000) listed medial and middle prefrontal and cingulate regions in both hemispheres as key structures in the processing of emotions. They claim that inability to recognize and express emotions, that is believed to be a personality trait, may be related to the differences in cerebral activity during processing of emotional stimuli that depends on their (un)pleasantness. Iidaka et al. (2000) studied (by fMRI) brain activity during recognition of faces with neutral, negative (angry or disgusted) and positive (happy) expressions. In addition to the common cortical and subcortical regions that constitute a unique neural network, they found areas specific for negative and positive expressions. The common network comprises the bilateral prefrontal region (Brodmann’s area 46), the fusiform gyrus (Brodmann’s area 37), and the amygdala. Areas specific to negative, as opposed to neutral, expressions were the left amygdala, the parahippocampal gyrus, and the right middle temporal gyrus (Brodmann’s area 21). Areas specific to positive, as opposed to neutral, expressions were the prefrontal
Lateralization and Localization of Functions
135
area (Brodmann’s area 46), and the right inferior and middle temporal gyrus (Brodmann’s areas 37 and 21). Direct comparison of activity elicited by negative and positive facial expressions revealed functional hemispheric asymmetry: negative expressions activated mainly the left hemisphere (Brodmann’s areas 45/46, 6, and 21), whereas positive expressions activated predominantly the right hemisphere (Brodmann’s areas 10 and 46, the fusiform gyrus in Brodmann’s area 37, and Brodmann’s area 7). Finally, the authors concluded that positive facial expressions are processed holistically, in the right hemisphere, whereas the processing of negative expressions requires a more analytical processing in the left hemisphere. However, Simon-Thomas, Role, and Knight (2005) found that emotional responses to aversive pictures selectively facilitated right-hemisphere processing during higher cognitive task performance. In line with some earlier studies that reported modality-specific cerebral activation in response to emotional stimuli, Heining et al. (2000) found (using fMRI) that identical emotions (anger, disgust, fear, sadness) elicit greater cerebral activation when presented auditorily than when presented visually. The common region activated in most emotions, regardless of presentation modality, was the cerebellum. It may, therefore, be inferred that it is essential in processing emotional content in general. In the auditory modality all emotions activated the auditory cortex in both hemispheres and the anterior cingulate. The cingulate activity is related to its role in attention, because it is obvious that the auditory modality requires a greater amount of attention. Almost all visually presented emotions activated the visual cortex. The activity of some emotion-specific structures was recorded as well: visually presented emotions expressing fear and auditorily presented anger activated the amygdala; auditorily presented expressions of disgust activated the left insula (in accordance with earlier studies). However, contrary to earlier studies, no activity was recorded in the amygdala during presentation of emotions expressing sadness, regardless of modality. With respect to lateralization in brain-damaged patients, a double dissociation was found between understanding meaning based on language and emotional prosody: the left hemisphere was crucial for the linguistic aspect, and the right hemisphere for prosody (more on that in the last chapter). Similarly to recognition of emotions, their expression depends mainly on the right hemisphere. However, it seems that two different neural pathways control emotional facial expressions, depending on whether they are intentional or spontaneous. The former are controlled by the left hemisphere, and they involve the cortex, whereas the latter are under bilateral control, with no cortical activity. Healthy subjects’ data do not suggest such pronounced right-hemisphere dominance as found in clinical patients, but they do confirm the relatively greater involvement of the right hemisphere, compared with the left, during emotional functions (Gazzaniga et al., 2002). An interesting correlation between affective style and differences in the left and right hemisphere electrical activity during rest was found by EEG. Individuals that rate themselves more positively (e.g., as happier) on the scale of positive and negative personality traits exhibit relatively greater activity in the medial part of the left frontal lobe, and those who rate themselves more negatively (e.g., anxious)
136
The Cognitive Neuroscience of Human Communication
exhibit relatively greater activity in the corresponding area of the right frontal lobe. Asymmetries have been found in newborns as well. Babies with relatively higher activity in the right hemisphere cry more when separated from their mothers, as opposed to those who had greater activity in the left hemisphere (Gazzaniga et al., 2002). Clinical and experimental data have indicated that the right hemisphere is more closely related to processing of emotional content than the left. Judaš and Kostovi´c (1997) offered the following summary: 1. Emotional facial expressions presented to the left visual field (right hemisphere) are recognized faster and more accurately. 2. Emotional facial movements are usually more prominent on the left side (i.e., contralaterally to the hemisphere controlling the appropriate muscles). 3. Right-hemisphere injury causes more difficulties in recognizing emotional facial expressions than left-hemisphere injury. 4. Split-brain patients recognize emotionally relevant events only if presented to their left visual field. Many clinical cases confirmed that emotions and mood in general are not exclusively associated with the limbic system, their primary neurophysiological substrate, but with cortical structures as well. The theory of right-hemisphere dominance for affective behavior was proposed on the basis of studies of righthemisphere patients and healthy subjects that included behavioral tests and EEG measurements, among other methods. According to the theory, the right hemisphere is dominant for perception, recognition, and expression of emotions. Some authors have found that affective problems are encountered in patients with either left or right hemisphere damage, but that left-hemisphere damage results in cognitive deficits, whereas right-hemisphere damage causes emotional problems. Another hypothesis related to the possible laterality of affective functions proposes that the importance of the two hemispheres in emotional behavior is different, depending on the valence (emotional value) of the stimulus (Ortigue et al., 2004). According to that hypothesis, the right hemisphere is better at processing negative emotions, whereas the left hemisphere is superior at processing positive emotions. Although this dichotomy might account for cases of right-hemisphere patients who respond to negative emotions less strongly and see everything in a more positive light than left-hemisphere patients, clinical cases are not conclusive in that respect (for review, see Myers, 1999). According to Myers (1999), affective and emotional disorders caused by righthemisphere damage are manifested as: 1. Impaired recognition and production of nonverbal emotional information (decreased use of facial expressions in expressing emotions, decreased sensitivity to facial expressions of others, decreased use of gestures, inappropriate body posture).
Lateralization and Localization of Functions
137
2. Impaired understanding of verbally expressed emotions (difficulties in understanding emotions conveyed verbally in stories, pictures and films, as well as decreased recognition of emotional prosody). 3. Impaired verbal expression of emotions (impoverished use of emotions in conversation and storytelling, decreased use of prosody in expressing emotions, and decreased recall of emotionally colored events and memories). Emotional and affective problems may take different forms and intensity, or they may be absent entirely. The most frequent manifestations of right-hemisphere damage at that level are depression, states of confusion—which may include restlessness—disorientation, delirium, psychoses, indifference, and lack of awareness of one’s own condition. In this volume, when we talk about consequences of right-hemisphere damage in affective and emotional aspects of communication, we refer to individuals who do not present with actual psychological problems, but only to those who have cognitive difficulties.
SUMMARY In a great majority of people, the left hemisphere is without a doubt dominant for language processes. Asymmetric brain morphology and consequences of lesions confirm that. Are earlier analytic and segmental specializations of the left hemisphere accountable for that? In other words, have language mechanisms been drawn to the left hemisphere during evolution or even ontogenetic development and then subjected to further neural development in the left hemisphere due to its already existing analytic segmental specialization? If we were to adopt such a viewpoint, we would consider the left hemisphere as specialized, not for symbolic functions as such, but for carrying out some kinds of motor activity that may be used in communication. On the other hand, are all observed language superiorities of the left hemisphere really just manifestations of analytic and sequential processing of the left hemisphere? There is plenty of evidence of the left-hemisphere analytic or segmental superiority, regardless of the phonological or verbal component. Some authors point out the advantage of speech and verbal processes with respect to left-hemisphere dominance, particularly unilateral motor control of the speech apparatus. According to them, speech is a species-specific ability. It is, therefore, controlled by the specialized left-hemisphere mechanism that enables humans to overcome limited resolution capabilities of the ears and the lack of acoustic invariance in the speech signal. It is believed that this mechanism is responsible for the phonological and the sound system that determines basic speech sounds and implements rules for their combinations into meaningful words, as well as for syntactic rules for combining these words into phrases or sentences. This leaves the right hemisphere free to engage in another important aspect of survival—visuospatial orientation and functioning.
7 Learning and Memory One of the most impressive characteristics of the brain is its ability to learn how to create internal representations that make possible various skills: speech, recognition of faces and objects, sports, arts, and so forth. At the neurophysiologic level, learning is based on changes in synaptic activity induced by the influence of one neuron on another (Hinton, 1992; Rose, 1995). It seems that in these learning processes, the brain uses so-called population codes, in which the information is represented by an entire population of active neurons. For example, as the brain coordinates eye movements, the process is coded by the activity of a population of nerve cells, each of which represents a somewhat different movement. The final outcome, the actual movement, is the result of all constituent movements coded by active cells. If for some reason some of the cells are not active, the movement will be different than anticipated, but not impossible. All learning is based on one of the key principles of the structure, development, and functioning of the nervous system: plasticity.
PLASTICITY As previously discussed in chapter 5 on the principles, models, and theories, (neural) plasticity may be manifested in several ways. Developmental plasticity involves changes in connections that occur after birth as a consequence of interactions with the environment. These changes make possible the process of learning and acquiring new experiences but are also a result of these processes and are present until old age. Another type of plasticity refers to the ability to recover functions after brain damage (or more generally, after damage to the central nervous system). Several decades ago the development of the nervous system was thought to be unchangeable: it was believed that neural connections were genetically determined with little influence from the environment on synaptogenesis. In other words, the connections among the neurons making up the nervous system were thought to be fixed once established. However, it eventually became clear that the genetic code determines just a set of rules for brain structure. Two principal rules are genetically determined: (a) the time when different cell groups appear (e.g., the ones that are predetermined to form the cortex or the hypothalamus), and (b) relative positions of cells and their terminals at the end of the process. The brain develops in an orderly and organized fashion: cells in certain areas occur and establish connections before cells in other areas (Changeaux, 1983). Neuronal activity cannot be chaotic either. It is defined temporally and spatially 139
140
The Cognitive Neuroscience of Human Communication
and is associated with particular synapse types. Temporal determination of action potentials is crucial in determining which synaptic connections are to be maintained and strengthened, and which are to be eliminated (Shatz, 1992). Obviously, distribution and connection patterns of earlier cells determine distribution and connection patterns of those that appear later on. Such a developmental mechanism is efficient, but probably not very precise. It provides similar overall brain structure within the species but not a well-formed mature nervous system. Under certain circumstances some synaptic connections may change, and this capacity to adapt is the basis of neural plasticity. Plasticity underlies the continuity of mental life (Aitkin, 1990; Fischbach, 1992). From the neurophysiologic point of view, action potentials do not just encode information—their metabolic consequences change pathways along which they travel. Plasticity increases the complexity of molecular characteristics or functions of cells (Fischbach, 1992). Grachev and Apkarian (2000) studied chemical changes related to establishment of new synaptic connections and found that the thalamus, sensorimotor cortex, and the cingulate might play a key role in plastic changes associated with normal aging. The fact that cortical connections in the somatosensory system are modified constantly, based on relevant activities is also related to plasticity. There is ample evidence that considerable reorganizations may occur in some cortical areas as consequences of the changes in peripheral receptors. For example, early sensory hearing loss will result in the reorganization of the representation of the auditory periphery in the primary auditory cortex in such a way that the intact (peripheral) hearing cells are represented in a much larger area than is the case in people with normal hearing (Harrison, Smith, Nagasawa, Stanton, & Mound, 1992). This is the basis of auditory hypersensitivity (i.e., recruitment), which typically accompanies sensory hearing loss. Somatosensory cortical maps seem very plastic, at least in primates, although there are limits to the expansion of a particular cortical area after peripheral changes. In the primary sensory areas, the brain refines the connections to the level of high precision during critical periods in early life, by means of unsupervised learning. During these critical periods (but not before or after) synaptic connections among neurons are very plastic and they react vigorously to changes in the patterns of neural activity (Stryker, 1995). In other words, cortical maps (for more on cortical maps see chapter 6, this volume) change constantly under the influence of the use of sensory pathways. It is believed that fine-tuning of connections in later developmental stages may require an activitydependent associative synaptic mechanism (Kandel & Hawkins, 1992; see also Levy, Koeppen, & Stanton, 2005). Singer (1995) proposed two complementary strategies that the brain uses to analyze and form representations of an infinite number of possible relations among the components of sensory and motor patterns. The first strategy is the analysis and representation of relationships that occur frequently, by means of broadly tuned cells that selectively respond to certain features. The other is a dynamic combination of these cells into functional units. Both strategies depend on experience, but the former is predominant in the prenatal and early postnatal period, whereas the latter reflects adaptability and probably underlies the ability
Learning and Memory
141
to form representations of new perceptual and motor patterns throughout life. Attention and relevance of the activity for the individual are important prerequisites, especially for the latter strategy (Maunsell, 1995; Singer, 1995): attention paid to certain aspects of the stimulus may increase neuronal responses by as much as 50% to 100%. The brain starts to function even before all circuits and connections have been completely developed. Neuronal connections are elaborated and perfected from the initial immature structure that only in general terms resembles the patterns of the adult brain. Although people are born with almost all neurons that they are ever going to have, brain mass at birth is only about one fourth of the adult brain. The brain enlarges because the neurons grow, the number of axons and dendrites increases, as well as the number of their connections (Shatz, 1992; Strange, 1995). Developmental plasticity is, obviously, fine-tuning of connections within the nervous system and it depends on experience during postnatal period. At first, there is an excess of connections among cells, and then their number gradually decreases. This overabundance is probably manifested as overreacting of cells to stimuli in early life periods. This responsiveness gradually decreases as a function of later experience. Neuronal activity is a prerequisite for developing the precision characteristic of the adult brain—the brain has to be stimulated. Functional properties of neurons, as well as their structure acquire their specificities during postnatal development. The sequence of developmental changes in arborization indicates that the mature pattern of connections is established by means of reshaping the axons through selective growth or withdrawal of different branches (Shatz, 1992). The greatest changes in the primate brain occur in its cortex during the postnatal period, especially until puberty. This is attributed to the fact that the cortex is the most susceptible to postnatal experience. The relatively large size of the primate cortex (especially in humans) compared with other species, is associated with its higher cognitive functions (Johnson, 2000). This is particularly true of the frontal regions that are the last to fully develop in the postnatal period. Developmental changes that may be found in children during the early postnatal period (before 12 months of age) are a consequence of the maturation of the frontal lobes. Neural development results in cognitive changes, but the opposite is true as well. Some of its aspects may be a function of the interaction between neurocognitive systems of the organism and the environment. In addition to context, knowledge, and other factors, past experience can also have a powerful effect on the reception and perception of new stimuli, as well as on the reorganization of the cortical regions that used to be considered exclusively primary sensory areas (Gilbert, Das, Ito, Kapadia, & Westheimer, 2000; Kosslyn, Thompson, Kim, & Alpert, 2000). Interestingly, the effect of experience is first manifested as decreased activation of sensory and motor cortical areas. This is explained by the selectivity of neuronal populations that participate in the representation of familiar stimuli, because experience has “taught” them which features are relevant. After the
142
The Cognitive Neuroscience of Human Communication
initial decrease, additional cells become gradually active (Turner & Frackowiak, 1995; Ungerleider, 1995). At the same time, changes in activation of other regions (cerebellum, prefrontal cortex) occur, which may be compared with the previously mentioned results of Raichle (1994). According to Changeux (1983), the connections among cell groups are genetically determined but not stable. Unstable synapses may be reinforced or weakened—depending on the overall activity of the postsynaptic cell, which is input-dependent. The input may be a result of spontaneous activity in the network that may be enhanced by environmental effects. This is selective stabilization, which means at the neuronal level learning is actually the process of elimination. A manifestation of learning is cortical reorganization in the absence of sensory stimuli. For example, congenitally deaf individuals were found to be more sensitive to events occurring at the periphery of their visual fields than hearing subjects; furthermore, visual stimuli activated parts of their cerebral cortex that are activated by auditory stimuli in hearing persons (in the temporal lobe). These results of ERP studies are interpreted as stabilization or reinforcement of visual afferent connections in the areas that would otherwise remain unused due to the absence of auditory stimulation ordinarily represented in these regions. In this context, Neville (1991) concludes that development of language competence (she is specifically talking about grammar) is necessary for the left hemisphere to become specialized for language. As it is discussed elsewhere in this volume, this is not necessarily applicable exclusively to oral language: the left hemisphere is specialized for language in congenitally deaf individuals who use sign language. One of the important facts resulting from studies of connections between neural development and behavior is that the relationship is two-way: behavior is not just a consequence of neural development, it is an active contributor to it. Orientation of sensory organs toward particular types of stimuli within the developing nervous system makes possible specialization for processing of information about biologically relevant stimuli. Le Carret, Petrescu, Amieva, Fabrigoule, and Allard (2002) used fMRI to compare the brain activity of young and old adults while they were solving tasks that required selective attention. In young subjects, they found bilateral activity in the occipital and superior parietal areas, which are responsible for visuospatial processing. In older subjects, they found decreased activity in the posterior parietal regions and bilateral activation of the dorsolateral prefrontal cortex. This indicates that, in order for cognitive abilities to be maintained, functional changes occur in cortical network activity: the areas that become less efficient with age (e.g., visuospatial) are functionally replaced or aided by other resources in the more frontal regions. Hagoort, Wassenaar, and Brown (2003) provide evidence from an ERP study of agrammatic patients, nonagrammatic aphasics and controls for the compensation of a syntactic deficit by a greater reliance on another route in mapping sound onto meaning. This form of plasticity is referred to as multipleroute plasticity. As discussed at the beginning of the chapter, plasticity is also manifested as the ability to recover after injury or some other nervous system dysfunction.
Learning and Memory
143
Several variables may affect recovery after brain damage (Kalat, 1995; Legg, 1989). An example of relatively rapid reorganization is vestibular compensation. Because of this mechanism, asymmetry in the sense of balance that occurs as a consequence of unequal responsiveness of the vestibular system in the left or the right inner ear (manifested as ataxia) is very quickly compensated, primarily in the cerebellum, but also at the level of the vestibular nucleus in the brainstem (Dutia, 2002). On the other hand, there are cases of very slow reorganization: Gazzaniga et al. (2002) reported on patients who started “speaking” from their right hemispheres as long as 10 years after callosotomy. After a stroke in one hemisphere, sensorimotor representations of arms or legs are frequently reorganized in both hemispheres. In the intact hemisphere, this is attributed to the fact that the extremities that had not been affected are used more, which changes their cerebral representation accordingly (Rossini, 2002). Such activation of the cortical regions in the intact hemisphere has been recorded by PET and MRI after various types of cortical and subcortical lesions. Weiler (as cited in Runji´c, 1996) calls this activation the auxiliary mechanism. The age at which the brain injury occurs plays an important role in recovery, but that influence is not straightforward. On the one hand, the developing brain has greater capabilities for compensating for the deficits occurring after localized damage. On the other hand, precisely because development has not been completed by the time of injury in young people, the risk of other disorders in subsequent development is increased. After a left hemispherectomy in early childhood, language functions are likely to transfer to the right hemisphere with minimal functional deficits (Boatman et al., 1999; Lassonde, Sauerwein, Chicoine, & Geoffroy, 1991). Obviously, the site of injury is closely related to the prognosis for recovery, but apart from location itself, the rate at which damage occurred is also important. For example, a gradually developing tumor will cause less severe functional disorders than a sudden focal injury or stroke. The extent of damage is frequently more important than its location. Weiler (as cited in Runji´c, 1996) concluded that the potential for functional reorganization is smaller and the outcome poorer if the affected motor area is larger. It has also been found that appropriate care and attention immediately after surgery resulted in alleviation or complete disappearance of symptoms, which suggests that the time and quality of the postoperative experience also play important roles in recovery. It is presumed that the mechanism of recovery is based primarily on the fact that intact systems go back to normal activity levels, rather than a damaged system being replaced by another one, but this hypothesis lacks sufficient evidence. LeVere (as cited by Legg, 1989) concluded that even partial damage of the system may change its function to the degree that a patient may even stop using it entirely in spite of the fact that some residual function is still possible. The consequence of this is overrating the extent of damage. Other mechanisms of recovery that have been proposed in the past (with varying degrees of empirical support) include: taking over the function of the damaged region by the intact ones that had not had that particular function before; adopting new problem-solving strategies; and limited regeneration that is primarily manifested
144
The Cognitive Neuroscience of Human Communication
as sprouting of new axon terminals at undamaged axons. In a great many cases the reorganization occurs in the same hemisphere (Milner, 1994). The brain has the ability to reorganize after peripheral injury or deprivation so that it allows neighboring (intact) areas to expand to the area normally occupied by the input from the (now) deprived sensory organs. In blind individuals, it has been found that additional cortical areas participate in active recognition of auditory and somatosensory stimuli. There is also some indication of enhanced automatic processing of changes in auditory stimuli. Data suggest compensatory changes in the auditory and somatosensory modality after early visual deprivation. This plasticity is possibly not limited to particular developmental periods but may to some extent be feasible throughout life (Kujala et al., 1995; Rauschecker, 1995). Bellman Thiran et al. (2002) recorded by fMRI the functional reorganization in the brain of their 60-year-old patient suffering from progressive aphasia due to atrophy of the left temporal lobe. Plasticity that is related to learning and experience, but which also enables functional reorganization after injury, is associated with notion of the so-called critical periods.
CRITICAL PERIODS Critical period is the time span between the onset of an anatomically or functionally determined biobehavioral system and its maturation. During this immature phase the system may be under the influence of positive or negative exogenous stimuli with more or less permanent consequences (Colombo, 1982; Spreen, Tupper, Risser, Tuokko, & Edgell, 1984). Critical periods occur even prenatally. The most sensitive at a particular point are the organs and systems that undergo the most intensive cell growth. The systems that have not yet reached their level of rapid growth and development as well as those that have already completed that stage are less affected by external influences (e.g., teratogenous drugs). It is believed that the initial phases of axon growth and selection of the paths of migration are independent of their activity. The genetically determined part of the program can be seen in the high degree of structural completeness that is reached during the embryonic stage. However, when axon terminals reach their designated areas, the selection of specific targets is under the influence of nerve impulses originating in the brain or stimulated by exogenous events. Synaptogenesis during a critical period may be determined by the type of competition among axons; the axons that have been activated in the optimal manner have priority (Fischbach, 1992; Shatz, 1992). According to Spreen et al. (1984), the following criteria have to be met in order for a period to be called critical: (a) the beginning and the end that can be identified; (b) an intrinsic component (i.e., sensitivity of the organism to some maturational event); and (c) an extrinsic component (i.e., an external stimulus to which the organism is sensitive). It is possible to compare the developmental periods of the brain with the ˇ critical, sensitive (Spreen et al., 1984), or optimal (Andrilovi´c & Cudina, 1990)
Learning and Memory
145
periods in the development of the organism. These periods are defined as windows of time during which the organism is particularly susceptible and/or vulnerable (i.e., more sensitive) to the environmental influences, and that involve prenatal and postnatal period. Coren and Ward (1989) define the critical period as “… an interval during which sensory experience is essential if perceptual development is to proceed normally” (p. 509). In postnatal development the importance of these periods is manifested in acquiring early experiences (Shatz, 1992), establishing basic social relationships, and learning. Moreover, developmental stages of the brain, which are closely associated with the notion of plasticity, are the foundation of behavior that is explained by the onset or offset of a critical period. Studies in anatomy, biology, physiology, and developmental neurobiology have shown that the brain does not develop continuously or evenly, but rather in spurts. These brain growth spurts are chronologically predictable and alternate with plateaus. About 85% to 90% of children of average and above-average abilities go through several phases of brain growth spurts: between the 3rd and 10th months of life, between the 2nd and 4th years, between the 6th and 8th years, between the 10th and 12th years, and between the 14th and 16th years of age (or later; Epstein, 1980). Some authors question the methodology of Epstein’s studies and challenge the replicability of his results, but there is agreement as to the existence of brain growth spurts, with a word of caution that these phases are not necessarily simultaneous in all brain areas: some areas are at a plateau, whereas others are undergoing rapid growth (Greenough, Black, & Wallace, 1987). However, Chugani, Phelps, and Mazziotta (1987) measured glucose consumption (as an indicator of activity) in eight cortical and three subcortical areas, as well as in the cerebellar cortex, and in the brainstem and found that by the age of 2, children reach adult (19 to 30 years of age) levels in all measured areas; these values increase until ages 3 to 8, when they are about twice the adult levels, after which time there is a decrease. At ages 9 to 15 the levels are still somewhat higher than in adults. One of the key issues is the relationship between brain growth and development on the one hand, and mental functions and learning on the other. The visual cortex is a good example of the close connection between anatomic changes and functional development. Synaptogenesis is closely related to the development of visual functions: at birth, there are only about 10% of connections, whereas rich synaptic connections are established rapidly at the age of approximately 4 months corresponding to the time of development of visual acuity and the ability to track objects (Huttenlocher, 1990). Huttenlocher’s studies confirmed Epstein’s (1980) claims that brain size increases are due to the increase in the amount of protein. In neurons, this increase in the amount of protein is manifested as elongation and arborization of axons and dendrites, despite the decrease in the number of synapses per neuron between late childhood and age 16 (Huttenlocher, 1990). The process results in a remarkable increase in the complexity of neural networks during later developmental stages, which is manifested as greater complexity of cognitive processes in the brain. Lassonde et al. (1991) associate the maturation of the corpus callosum with periods of greater plasticity. On the other hand, during plateau phases in brain growth and development there is practically no increase in brain
146
The Cognitive Neuroscience of Human Communication
mass or complexity of neural networks (Epstein, 1980). The importance of the age factor in learning has been confirmed for various skills—from sports to arts. Chen, Halpern, Bly, Edelman, and Schlaug (2000) found in their fMRI studies that early exposure to music and commencement of music education is important for the development of absolute pitch. However, they also stressed the importance of the necessary anatomic prerequisite—smaller right than left planum temporale. Locke (1994) believes that the sensitive period for language acquisition is a developmental framework that includes several phases. These phases are sensitive to external and internal events and vary in the degree of sensitivity. The constituent systems of the overall language system are turned on and supported by different processes, and each of them has its own subsystem and course. The consequence of these variations is a series of dynamically variable interactions that cannot be recognized or explained by the existing theories of language development. The notion of sensitive periods in language acquisition is supported by evidence from acquisition of sign language by the deaf. Lieberman (1991) found that individuals who acquired sign language earlier performed better at the morphological level than those who acquired it at a later age. Foreign language learning literature provides ample evidence that the ability to learn a foreign language to the level of near-native competence decreases sharply after ages 6 to 8. Around age 14 the ability reaches and remains at the adult level. Most of these data do not, unfortunately, tell us much about the acquisition of the first (native) language, because they do not address the initial activation and the development of the learning systems and processes involved in mastering a language. Another flaw in most approaches to language acquisition and learning is that they treat language as a unitary ability, not as different systems—e.g., phonology, morphology, syntax, lexicon—which would be more appropriate, because the abilities that support these systems are different and do not develop simultaneously (Hyltenstam, 1992; Locke, 1994). Language development begins with the orientation of the child toward language manifestations—the physical activity of people who talk—which provides children with information about language. Vocal learning and the first words are preceded by (socially) cognitive operations: studying and interpreting eye movements with respect to utterances, learning and application of the rules of turn-taking in vocalizations, orientation toward and imitation of prosody, communicative pointing, assimilation of phonetic patterns, and interpretation of the mental activities of others. Combined with perceptual experience and motor development, these procedures enable children to manage (native) language situations in spite of their limited language capacity (Locke, 1994). What Lenneberg neglected in his claims that language acquisition starts around the second year of life is that the crucial part of the critical period for language is the preverbal phase. The amount of attention the children get from their mothers or other care providers and the extent of the imitations they produce by their own vocalizations were found to be in positive correlation with the size of their expressive vocabulary at 21 months of age. According to Locke (1994), there is a vocal learning phase and three linguistic phases in language acquisition. They occur in specific order and
Learning and Memory
147
partly overlap. Each phase affects the next but is also under the influence of the preceding one, and they are all affected by neuromaturational events and social stimulation. The vocal learning phase and the first linguistic phase (acquisition of utterances) are primarily affective and social. Their goal is to master the prosody of the language and form a collection of utterances. The function of the second linguistic phase (structure analysis and computation) is analytical. In this phase, previously stored forms are analyzed into syllables and segments. This process enables discovery of regularities and acquisition of grammar, including phonology, morphology, and syntax. The third linguistic phase (integration and elaboration) extends throughout the lifetime and is in fact the basis for lexical learning (i.e., enrichment of vocabulary). Preliminary results of Plumet, Gil, and Gaonac (2002) suggest that practice may improve and maintain cognitive functions (e.g., verbal fluency and memory) well into old age. Contrary to the critical periods discussed previously, referring primarily to different phases before and including puberty, Lenneberg’s (1967) notion of the critical period in language acquisition includes the entire preadolescent period, as the optimal time for language acquisition as opposed to the (post)adolescent period. On the basis of evidence from stroke patients, children with Down syndrome, and the experiences of foreign-language learners, Hurford (1991) also places the critical period for language acquisition in the time before puberty. Deane (1992) associates language acquisition with the rate of development of the parietal lobe, which is, according to him, the seat of language function as an integral process at three levels (phonological, syntactic, and cognitive, or semantic–pragmatic). According to his hypothesis, grammatical competence is represented in brain regions whose primary function is to represent body image, which means that they are to be found in the areas where bits of information from different sensory modalities converge and are integrated: in the inferior parietal lobe, at the border between the temporal, occipital, and parietal lobes. Since neurological measurements reveal rapid development of the parietal lobe between birth and the 2nd year of life (with further, somewhat slower maturation until age 10), Deane believes that it is possible that “… the stages of development in first language acquisition reflect stages in the maturation of the inferior parietal lobe …” (p. 297). Johnson and Newport (1989) tested the critical period hypothesis (in addition to numerous other tests of motivation, identification, etc.) on Korean and Chinese subjects who had arrived in the United States at different ages. Their knowledge of American English was measured by comprehensive syntactic and morphological tests. With respect to critical periods they concluded that: (a) there is a significant correlation between the age of arrival in the United States and the results on all types of syntactic and morphological tests; (b) subjects who had arrived in the United States before the age of 7 had results comparable to native speakers of American English, whereas those who arrived after the age of 7 exhibited a noticeable drop in competence until puberty; (c) subjects who had arrived as adults scored on average significantly more poorly than those who had arrived before puberty, but competence did not decrease as a function of increasing postadolescent age of arrival. Their general conclusions (which they found support for in many studies by other authors) are that the notion
148
The Cognitive Neuroscience of Human Communication
of the critical period may be extended from the native language acquisition proper to foreign language learning, but that the capacities diminish differently than defined by Lenneberg (1967). A drop in competence does not occur in puberty, but as early as around 7 years of age. Interestingly, after age 7, the number of synapses in the frontal lobe starts to decrease (although the functions of that cortical region are not yet understood completely, it is clear that it plays an important role in complex intellectual processes). As it has been mentioned above, cerebral metabolism measurements have shown that metabolic activity of the human cortex increases rapidly in the early postnatal period, remains above adult levels during childhood, and decreases to adult levels during puberty (Chugani, Phelps, & Mazziotta, 1987; Huttenlocher, 1990). The importance of critical periods in language acquisition has been confirmed by (luckily rare) examples of speech and language deprivation without organic etiology. These are the cases of the so-called feral children, who for various reasons had no exposure to speech and language communication at the optimal age. The best documented example is Genie, who had lived in isolation (imposed by her parents) until she was 13. Her speech and language development (but also her social development and interactions) started when she was found, but in spite of enormous efforts and time spent in teaching and coaching, her competence never reached the level of children who had been growing up surrounded by speech. Even 4 years after she began stringing words into first sentences her speech was still confused and telegraphic. She could not distinguish passive from active verb forms and confused her pronouns. Her language and, it is interesting to note, her nonlanguage functions were lateralized to the right hemisphere. Learning is usually classified as associative and nonassociative. During associative learning the individuals learn about relationships between two events or about consequences of their own behavior. It may be simple (classical and instrumental conditioning) or complex (explicit or implicit). Nonassociative learning occurs when individuals learn and remember the features of the stimulus to which they are exposed. It may be manifested as habituation or sensitization. The process of acquiring information and skills (learning) and the result of that process (memory) regardless of whether it is conscious or unconscious, proceed through several stages (Figure 7.1). The first stage is encoding, during which the material to be learned is presented and processed. Encoding consists of two Learning
Encoding
Acquisition
Memory
Storage
Consolidation
FIGURE 7.1. Stages of learning and memory.
Retrieval
Learning and Memory
149
steps: First, the input data are registered by the senses and subjected to sensory analysis (acquisition), and then they are consolidated. The second stage is storage, during which some of the available information is stored in the memory system. During the third stage (retrieval) the stored material is retrieved from memory in the form of a conscious representation, recognition or as some form of learned behavior. Animal studies have shown that acquisition, storage, and retrieval are separate neural processes: one or more of them may proceed independently of the others (Webster, 1995). It has also been found that specific neural pathways participate in each learning task, depending on the nature of the task. In a particular situation the changes occur in the majority of the pathways, but not necessarily in all of their parts. It is, therefore, more plausible that data are stored in a wider area, rather than in a small number of structures (Levy et al., 2005).
TYPES OF MEMORY Memory is most frequently characterized according to the duration of information retention (sensory, short-term/working, and long-term; see Figure 7.2) and with respect to the type of information storage (declarative or explicit vs. nondeclarative or implicit). The latter classification is used in long-term memory. Each of these groups has characteristics and subgroups of its own. They are briefly reviewed here. For more detailed explanations and discussions of different types of memory see Zarevski (1994) and Gazzaniga et al. (2002).
SENSORY MEMORY The duration of this type of memory ranges from less than one second to several seconds. It is sometimes referred to as sensory trace(s). The traces disappear very quickly; most often we are not even aware of them, and the capacity for this type of memory is relatively large. Information representations are sensory rather than semantic and are manifested as brief traces of neural activity in cortical areas that correspond to the projections of various senses. If the information was auditory this type of memory is called echoic. The visual modality will produce iconic Memory with respect to duration
Sensory Milliseconds – seconds
Echoic (auditory)
FIGURE 7.2.
Iconic (visual)
Short-term/working Seconds-minutes
Haptic (tactile)
Types of memory with respect to duration.
Long-term Days – lifetime
150
The Cognitive Neuroscience of Human Communication
memory. Tactile information will result in haptic memory. Echoic memory may last from two to three seconds, but the traces start to dwindle after 300 to 500 ms (according to some authors even after 100 to 200 ms; Levy et al., 2005). Iconic memory lasts not longer than half a second (according to some authors not longer than 300 ms). Kölsch, Schroger, and Tervaniemi (1999) found that sensory memory, specifically echoic memory, may be modified by practice. If the stimulus is “worth” remembering, it is passed on to short-term/working memory.
SHORT-TERM/WORKING MEMORY This type of memory is measured in seconds and minutes. It enables us to remember someone’s phone number or address long enough to dial it or write it down. It is accessible to consciousness, but its capacity is very limited: five to nine individual pieces of information (the well-known Miller’s 7±2 number). One of the common ways of overcoming this limitation is grouping data before subjecting them to memory. A series of 12 digits (e.g., 257986105337) will be much easier to remember than six two-digit numbers (25, 79, 86, 10, 53, 37) or than four three-digit ones (257, 986, 105, 337). Many different mnemonic techniques are recommended in the popular literature (e.g., Howard, 2000). The serial position effect is closely related to the notion of short-term memory. It has been found that the elements at the end of the list and those at the beginning are the easiest to remember. These two effects are known as the recency effect and the primacy effect, respectively. They are based on different mechanisms. Primacy effect is based on the transfer from the short-term to long-term memory by repetition. Recency effect is based on retention in short-term memory (Gazzaniga et al., 2002). The time elapsed from the stimulus and the time allowed for response are the most important factors that determine which of the two effects will prevail (Petz, 1992). The term working memory evolved from the term short-term memory (as opposed to long-term memory). Short-term memory used to be described as a single short-term storage of limited capacity, responsible for various memory phenomena. It was also thought to be responsible for learning and recall. Several memory models proposed that storage of sensory data into long-term memory proceeds via short-term memory. However, clinical studies have shown that patients with severely impaired short-term memory may have intact long-term memory, which was in contradiction to the notion of short-term memory as a single system. It was, therefore, replaced by the term working memory, to designate short-term limited-capacity information storage where mental operations on the content of the storage may be performed as well. As opposed to the earlier models that described short-term memory only as one-way input to long-term memory, working memory may receive data from sensory memory, but from long-term memory as well, and may be a place for processing of such data. Because of this, working memory cannot be viewed as a single system. It comprises at least two components: storage and processing units. Baddeley and coworkers (as cited in Baddeley, 2000) propose a three-part model in which a central executive regulates and controls attention and is hierarchically superordinate to two active systems:
Learning and Memory
151
(a) the articulatory/phonological loop, and (b) the visuospatial sketchpad. The central executive coordinates their connections with the long-term memory. It is comparable to the supervisory attentional system of Norman and Shallice (as cited in Gazzaniga et al., 2002) that coordinates and plans activities, stops routine performance of learned behaviors when new circumstances that require different behavior occur, and so on. The phonological loop contains and maintains speech information. It comprises two components: short-term (transient) speech storage, where the trace remains for only about two seconds, and the process of articulatory control. This process may best be described as subvocal (silent) articulation or a practice that keeps the speech material in the storage and supplies new speech by the process of subvocalization, regardless of the fact that the original data input may not have been auditory or spoken. It has been shown that working memory uses acoustic rather than semantic code during practice. It is assumed that auditory speech information has automatic and obligatory access to the storage. The phonological loop is very important for the acquisition of new vocabulary, which involves longterm phonological learning. The visuospatial sketchpad involves short-term storage and control processes responsible for registering visual and spatial information and its restoration by repetition, or practice. The visuospatial sketchpad is also included in long-term memory, but these two subsystems are independent of each other. Their relative independence has support in neuropsychological research. It has been found that difficulties in phonological working memory, manifested as a decreased ability to hold a series of words in working memory, occurs because of injury to the left supramarginal gyrus (Brodmann’s area 40; see Figure 7.3). The phonological loop includes the left premotor area (Brodmann’s area 44). On the other hand, the visuospatial sketchpad is damaged after injury to the parietooccipital areas of either hemisphere. Right-hemisphere damage results in poorer nonverbal visuospatial short-term memory, whereas damage to homologous areas in the left hemisphere results in difficulties in short-term memory of visually presented language material (Gazzaniga et al., 2002). Neuroimaging studies of healthy subjects have revealed that the anatomic network of verbal working memory comprises mostly frontal regions (Brodmann’s areas 44, 10, 9, 6) during repetitions within the articulatory loop, and posterior regions (Brodmann’s areas 40 and 7) constitute the temporary phonological store. On the other hand, the anterior cingulate (Brodmann’s area 32) and the dorsolateral prefrontal cortex (Brodmann’s area 46) are active during selection, manipulation, error control, and affective regulation. During complex tasks or in some pathological conditions (e.g., in chronic fatigue syndrome), some righthemisphere areas exhibit increased activity (Lange et al., 2000). Fiez et al. (as cited in Gazzaniga et al., 2002) analyzed the results of several PET studies and concluded that tasks that include verbal practice (i.e., those that belong in the phonological loop) elicit increased blood flow in inferior regions of the left lateral prefrontal cortex (Brodmann’s areas 44 and 45) and in the right cerebellar hemisphere. They attributed this cerebellar activity to the subvocal repetition, and
152
The Cognitive Neuroscience of Human Communication
4
6 8
9
46
10
7
41
43
40
19 39 18
44
45
52
47 11
5
3 1 2
22
37
21
38
17
42
20
31
4
6
5
2
8
7
24
9
31 23
32
33
10
30 26 29
27 11
18 17
25 38
19
34 28
35 36
37
19
18
20
FIGURE 7.3. Activation of brain areas—various stages and types of memory and learning.
activity in Broca’s area to phonological (re)coding. Crosson et al. (1999) found by fMRI that various posterior regions of the left hemisphere participate in decoding and storage of information, depending on whether the data in working memory are semantic, phonological or orthographic, which was in agreement with earlier literature on the mechanisms for processing of these types of data. During tasks that required involvement of working memory they found additional activation in the left frontal lobe, including the premotor cortex, but in subcortical structures as well. They concluded that there were different forms of verbal working memory, and that it was necessary to reconsider the premotor cortex functions. Walter et al. (2000) studied working memory during verbal (recognition of visually presented letters) and visuospatial (determining the position of visually presented letters) tasks by fMRI and found that verbal tasks activated left-hemisphere structures (inferior frontal gyrus, superior temporal gyrus, and fusiform gyrus) more than
Learning and Memory
153
the visuospatial tasks. These tasks activated medial precuneus, the right superior frontal lobe and the left superior temporal lobe. D’Arcy, Ryner, Richter, Service, and Connolly (2004) also confirmed left-hemisphere dominance for verbal memory. Suchan, Gayk, and Daum (2002) used the ERP method to study the visuospatial working memory, and recorded mainly the left lateralized positive peak with the 600 ms latency, over frontal lobes and a pronounced negativity over the right parietal lobe. They also concluded that different neural circuits are activated during processing of visuospatial information, depending on memory and attention requirements. Schmidt et al. (2002) reported similar results. In their fMRI study they recorded activity in the frontal, parietal, and temporal lobes of both hemispheres during manipulation of three-dimensional objects in virtual space. Chambers, Stokes, and Mattingly (2004) reported modality-specific attentional processing in the parietal cortex. Saur, Erb, Grodd, and Kammer (2002) studied the differences between visual processing of colors and spatial configuration with the aim of discovering possible differences that would correspond to the ventral (“what”) and the dorsal (“where”) pathways. Their data have suggested domain-specific processing of stimuli within working memory in the frontal lobes: during memorizing colors (the “what” pathway) there was increased activity in the middle frontal gyrus, whereas during memorizing the spatial arrangement of dots (the “where” pathway) there was increased activity in the premotor regions. Moreover, in addition to the frontal “what” pathway they recorded simultaneous activation of the posterior “what” pathway (ventral), and in addition to the frontal “where” pathway there was simultaneous activity in the posterior “where” pathway (dorsal). These differences were noticed at cellular level as well (Gazzaniga et al., 2002). Schizophrenic patients and healthy controls have similar activation patterns during visuospatial and verbal tasks related to working memory (Brodmann’s areas 44/45, 9/46, 6, 7, and 40, and the cerebellum). However, contrary to control subjects, in whom greater activation was found mainly in the left hemisphere during verbal but not for visuospatial tasks, they do not exhibit domain-specific lateralization, only overall greater activity (Walter et al., 2000). The central executive function is perhaps the most important, but also the least investigated segment of the working memory. Clinical studies suggest that its neurophysiologic substrate is in the frontal lobes of both hemispheres, particularly in the lateral prefrontal areas. Prefrontal areas are considered to be a temporary repository of representations and information that are held here while they are essential for performing the task at hand. When a new stimulus arrives, a temporary representation is retrieved and established in prefrontal cortex by means of connections with posterior regions where the representations are stored permanently. Patients with damage to these regions cannot control the direction of attention, which is manifested as perseveration on one and the same response, or as a reaction to the first stimulus that comes along, regardless of its relevance to the organism in general or to the actual situation. Alzheimer’s patients exhibit a pronounced disorder of the attention direction mechanism. In these patients it
154
The Cognitive Neuroscience of Human Communication
is manifested as the inability to coordinate two concurrent tasks from different subsystems, and the disorder becomes more serious as the condition progresses. These data support the description of working memory as the ability to simultaneously store and process information (Baddeley, 2000). On the other hand, Goldman-Rakic (2000) concluded on the basis of registering brain activity in monkeys that the central executive is distributed in the same areas where information is stored. According to her, it comprises specific modules for information processing, each with its own sensory and mnemonic properties and elements for motor and motivational control, rather than being a single control system with hierarchically lower slave subsystems. In other words, in the prefrontal cortex, there are several working memory domains, and not a single working memory with different ongoing processes. On the basis of clinical, experimental, and neurobiological data she concludes that the prefrontal cortex has a specialized function that is replicated in all or almost all cortical subparts, particularly in the cerebellum and basal ganglia. At the same time, interactions between these working-memory centers and other areas in specific cortical networks make up the brain structure for higher cognitive functions. Prefrontal cortical damage does not impede acquiring knowledge of the world, but it does prevent its use in control of behavior. Although the term working memory has its roots in the classical term shortterm memory, it seems that the more general model of the relationship between memory and cognition has been adopted. Working memory is involved in processing language data, in formation and understanding of utterances, but also in cognitive processes in general.
LONG-TERM MEMORY This type of memory is measured by days, months, and years. With respect to the characteristics of the stored information it is usually grouped into declarative and nondeclarative, and it has been found that these two types of memory are differently represented in the brain (Kandel & Hawkins, 1992). Declarative memory is also called explicit or direct, and it refers to knowledge (and experiences) that can be consciously accessed. It may be further divided into episodic and semantic memory. Episodic memory is knowledge of events that concern us personally (memory of the first day of school, wedding, the most embarrassing situation, etc.); it is therefore frequently referred to as autobiographic memory, and it is temporally marked. Semantic memory contains knowledge of facts that we have read, learned in school, seen on television, and so forth, without particular details about the process of acquiring such knowledge, which makes it temporally unmarked. Semantic memory underlies the mental lexicon (see chap. 8, this volume). Nondeclarative memory is also called implicit or indirect. It is sometimes used synonymously with the term procedural memory, but most authors agree that nondeclarative memory comprises subtypes other than procedural memory (Kalat, 1995; Thompson, 1993). Besides procedural memory, nondeclarative
Learning and Memory
155
memory includes the perceptual representation system, classical conditioning, and nonassociative learning (Gazzaniga et al., 2002). What is common to all these subtypes is that they all involve knowledge that has been acquired and stored without explicit memory of that process. It is an unintentional, unconscious form of knowledge retention. Procedural memory involves the ability to perform a motor or cognitive skill (e.g., skiing or reading) without conscious knowledge or deliberation about how the skill is performed. Knowledge retained in the perceptual representation system may be discovered by priming and memory tests. Classical conditioning is also referred to as Pavlovian conditioning. As mentioned previously, nonassociative learning involves habituation and sensitization. Explicit memory is typically studied by means of recall and recognition tests that require intentional recall of information from some specific earlier learning situation. On the other hand, implicit memory is tested by tasks that do not require conscious recall of specific episodes or events. PET studies have shown that implicit memory may be further categorized into perceptual and conceptual implicit memory, because perceptual and conceptual priming tasks affected different parts of the brain (Baddeley, 2000; Schacter, 2000). The above classification of long-term memory is based mainly on neuropsychological tests and observations of brain-damaged patients. It has been noticed that different types of memory are affected differently by injury: some types may be completely disordered without deficits in the others. Amnesic patients are believed to have deficits in the episodic learning mechanism, which is manifested as difficulties in adding on to the existing semantic memory, namely, its updating. In such patients implicit procedural learning capabilities are usually preserved, which is explained by the fact that such learning is possible without reliance on episodic learning (memory; Baddeley, 2000). Posthypnotic amnesia affects episodic but not semantic memory and it dissociates explicit from implicit memory (Sternberg, 2003). Drubach (2000) pointed out another difference between declarative and nondeclarative memory: Plasticity of the brain for declarative learning remains relatively constant throughout the lifetime (with the exception of very old people), whereas plasticity for nondeclarative learning is the greatest in early childhood. According to some authors, native (first) language and foreign (second) language are acquired or learned by means of different types of memory. The former rely on nondeclarative memory, and the latter on declarative (Paradis, 1994, 2004). In Figure 7.4 different types of long-term memory are shown.
NEURAL SUBSTRATES OF MEMORY The most important structures involved in declarative memory (episodic and semantic) are: (a) the medial part of the thalamus with the associated connections within the internal medullar lamina, and (b) the hippocampal formation with the neighboring anatomically related parts of the cortex (medial temporal lobe). Since most patients suffering amnesia caused by injury have poor episodic and semantic memory, it would seem logical that both types of memory depend on the same brain area. However, it has been found that episodic and semantic memories
156
The Cognitive Neuroscience of Human Communication
Long-term memory
Declarative (explicit)
Episodic
Events, personal experiences related to specific time and place
Nondeclarative (implicit)
Semantic
Conditioning
Perceptual representation system
Procedural
Nonassociative learning
Facts, knowledge
Classical, instrumental
Priming and interference
Motor and cognitive skills
Habituation, sensitization
FIGURE 7.4. Long-term memory.
are represented in neighboring brain regions that are commonly affected in amnesiacs (Baddeley, 2000). Although no differences were found between the consequences on episodic and semantic memory after injury to these parts, it seems that severe frontal lobe damage has a negative effect on episodic, but not on semantic memory (Squire & Zola, 2000). Rajah and Mcintosh (2005) suggested, on the basis of their combined ERP and PET studies, that activation differences between episodic and semantic retrieval may reflect variation along a continuum of processing during task performance within a single memory system. The hippocampus turned out to be the most active when explicit recall involved experience with the test material, rather than an attempt to remember information. Several different studies have confirmed that the hippocampus is the most important structure in coding new data and retrieval of recent information that requires explicit recall (Gazzaniga et al., 2002). Guillem, Rougier, and Claverie (1999) have found in their ERP studies that previous exposure to a stimulus facilitates each subsequent processing of an identical stimulus. They recorded electrical activity in different parts of the brain during a continuous memorization task in order to determine which parts contribute to short-term and long-term effects of stimulus repetition. Their results indicate that there are multiple anatomic and functional memory systems in the human brain (a short-term semantic activation system and a long-term episodic memory system) and structures that connect and coordinate the functioning of memory. Saykin et al. (2000) studied Alzheimer’s patients and found double dissociation between structural and functional changes in the systems that support semantic as opposed to episodic memory. Namely, atrophy in the left inferior frontal gyrus was associated with deficits in semantic processing, whereas atrophy of the hippocampus was associated with episodic memory disorders. Episodic memory is the cognitive function that is affected first and foremost in Alzheimer’s patients, similarly to memory changes that occur as a function of normal aging (Bernard, Desgranges, Eustache, & Baron, 2002). The theoretical hypotheses about the importance of the inferior temporal lobe for semantic memory were supported by Papps, Best, and O’Carroll (2000). They studied a patient who had suffered considerable damage to the left temporal and
Learning and Memory
157
parietal lobes. Many comprehensive tests have revealed good functioning of her semantic memory and even relatively intact neurological and neuropsychological functioning, probably owing to the spared left inferior and medial temporal lobe. Rossion et al. (2000) found by measuring cerebral blood flow during PET scans that recall of semantic information about visual characteristics of objects and their categorization are located in the left occipito-temporal regions. Their study has not confirmed prefrontal activity, which they attributed to the fact that their study involved direct processing of visual stimuli. Simultaneous activation of the left prefrontal areas may be elicited by a similar task, but through mediation of language operations. Semantic coding and information recall involve the left prefrontal cortex, primarily Broca’s area (Brodmann’s area 44) and neighboring parts (Brodmann’s areas 45 and 46). This lateralization is not affected by the type of the task (verbal vs. nonverbal) (Gazzaniga et al., 2002). In their PET study, Krause et al. (1999) found that an important role in episodic memory, regardless of modality (visual or auditory) or word characteristics (abstract or imageable), is played by precuneus, a part of the multimodal association area in the medial temporal lobe above its border with the occipital lobe. Activity in the right prefrontal cortex is greater when the subjects are trying to retrieve knowledge related to episodic memory than when they try to access other types of memory (Eysenck & Keane, 2000). The results of clinical and animal studies were compared with the PET and fMRI studies on healthy subjects. Tasks that tapped into episodic memory have shown that at the encoding stage (i.e., during acquisition of new knowledge) the right hippocampus and the left prefrontal cortex are activated, whereas recall of stored information activates the right prefrontal cortex. Several authors have found, using different tasks, that (neo)cortical areas are the most active, with task-dependent functional asymmetries: encoding activated the left hemisphere more than the right, and recall activated the right hemisphere more than the left. These asymmetries were affected by the nature of the task as well—language and pictorial tasks elicited opposite results (Bernard et al., 2002; Gazzaniga et al., 2002). Episodic memory is better for emotionally charged stimuli than for neutral. Furthermore, the results of the study by Beauregard, Breault, and Bourgouin (2000) suggest that the emotional valence of information may affect neural networks involved in the recall of memorized material (i.e., negative and positive events do not activate identical areas) as opposed to working memory that is not influenced by emotional content (Perlstein, Bradley, & Lang, 2000). Moreover, they found that men and women differed in the neural substrates responsible for memorizing pleasant and unpleasant stimuli, and that greater areas were activated in men than in women, regardless of whether the stimuli were pleasant or unpleasant. In women, memory of unpleasant stimuli was located predominantly in the right hemisphere (Brodmann’s areas 37 and 47), whereas in men it included Brodmann’s area 37 in both hemispheres, Brodmann’s area 18 in the left hemisphere, Brodmann’s area 36 in the right hemisphere and the right insula. Recall of pleasant stimuli activated Brodmann’s areas 21 and 37 in the right hemisphere and Brodmann’s area 19 in the left hemisphere in women, whereas in men, under
158
The Cognitive Neuroscience of Human Communication A
4
6 8 9 10 46 45 44
5 3 1 2 43
52
47 11
B
41
7
39
18
10
43 41 52
47 11
37
4
8
31 2 5
33
10 11 25
38
7
19
17
22 42
37
34 35 28 36
19 18 17
37
19
18
10
31
24
9 30
26 27 29
7
8
31
20
39
18
31 2 5
4
6
23
32
40
20
24
9
7
21
38
20
6
5
3 1 2
9 45 44
46
17
22 42
21
38
19
40
4
6
8
32
33
11 25 38
23 30
26 27 29 34 35 28 36
19 18 17
37
19
18
20
FIGURE 7.5. Brain areas responsible for processing emotional stimuli (A = unpleasant stimuli; B = pleasant stimuli).
identical conditions, Brodmann’s areas 10 and 47 of the right hemisphere and bilateral Brodmann’s areas 37 and 39 were activated (see Figure 7.5). The motor cortex was found to be important for implicit procedural learning of movement patterns. Activation was found in the putamen (basal ganglia) and in the supplementary motor cortex. All these areas make a common network—the cortico-subcortical motor loop—that regulates voluntary movements. On the other hand, when the subjects were aware of learning movements (explicit learning, or memory), the increased activity was found in: the right premotor cortex, the dorsolateral premotor cortex typically associated with working memory, the anterior cingulate, the parts of the parietal lobe that control attention, and the lateral parts of the temporal lobe believed to contain records of explicit memory (Gazzaniga et al., 2002). Declarative memory is associated with temporal lobe activity, including the hippocampus, whereas procedural memory includes specific sensory and motor systems that are active during a particular task (Kandel, 2002). Amygdala are important for learning social behavior, which is related to their role in processing emotional information (see chap. 6, this volume). The perceptual representation system, which is also a subtype of implicit learning, does not activate the hippocampus. In general, it is believed that normal memory functions rely heavily on the hippocampus. This is especially true for memorizing new data and associative
Learning and Memory
159
memory. One of the best known and most frequently quoted cases in neuroscientific literature is H. M., who had undergone a bilateral resection of the medial temporal lobe, and subsequently could not acquire any new knowledge. He could not even learn how to get to his new home although it was on the same street, just a couple of houses away from the address at which he used to live before the operation. He also suffered from retrograde amnesia reaching back 3 years prior to surgery. Apart from such severe cases, there are many milder or transient forms of amnesia. There is a positive correlation between the amount of damaged hippocampal tissue and the severity of amnesia (Scoville & Milner, 2000). Namely, only bilateral removal of these parts caused such serious deficits, whereas removal of the medial part of the right temporal lobe (including the hippocampus and the hippocampal gyrus) had no negative effects on the patients’ memory. Further studies have established that the medial part of the temporal lobe is not crucial for short-term memory. Due to the temporal characteristics of retrograde amnesia (which affects the period closest to the onset of amnesia and extends back into the past up to several years at the most), it is believed that damage to the hippocampus affects consolidation of new knowledge, but not access to the already consolidated information. It follows that the neocortex is the seat of long-term memory, whereas the hippocampus, albeit crucial for the process of consolidation, is not necessary for storage or recall of consolidated knowledge (Gazzaniga et al., 2002). On the other hand, Kali and Dayan (2002) proposed a model of neocortical functioning according to which both the hippocampus and the medial temporal lobe are essential for acquisition and recall of declarative data. The most prominent difference between types of memory, with respect to their neural substrates, was found between short-term and long-term memory. These differences are not limited to different brain areas that are selectively activated during processes belonging to one or the other type—they are present at the cellular level as well. Short-term/working memory is associated with rapid and temporary changes at the cellular level, namely, the internal structure of nerve cells; with the strengthening of synaptic connections, and only if the number of synapses actually increases, knowledge will be consolidated (Drubach, 2000; Webster, 1995). Long-term memory requires gene activation, establishing new connections and, according to Kandel (2002), synthesis of new proteins. The difference between short-term and long-term memory is manifested by strong double dissociation: Individuals suffering from short-term memory disorders have intact long-term memory and vice versa. Consequences of brain damage that are manifested as amnesia or some types of language disorders (e.g., aphasia) suggest that learning and memory have both the temporal and the categorial structure. It seems that all chunks of knowledge acquired within the same time period are stored together, and that information is organized and stored in categories (e.g., tools, fruits, animals). However, it seems less plausible that information is stored in separate compartments, as suggested by Drubach (2000). We are probably dealing with multiple representations of the same concept or information formed during acquisition, storage, and manipulation of knowledge.
160
The Cognitive Neuroscience of Human Communication
In summary, it can be said that: 1. The hippocampus participates in formation and consolidation of new episodic and possibly semantic memories. 2. The prefrontal cortex is responsible for encoding and recall of information. 3. Episodic and semantic information is stored in the temporal lobe. 4. The association cortex is the seat of perceptual priming. Other cortical and subcortical areas participate in acquisition of skills and habits as well. Although practically the entire brain is active during encoding, storing, and recall of information (i.e., in learning and memory) each type of memory relies on a specific structure. Kopp, Schroger, and Lipka (2006) found different networks to be engaged in different recall modes too. These differences are best manifested as selective deficits in different types of memory as consequences of damage or degenerative changes occurring in the brain, but they can also be revealed by neuroimaging methods in healthy subjects.
8 Speech and Language Language seems to have appeared on the evolution scale when people became skillful in movement and categorizing actions, and in forming and categorizing mental representations of objects, events, and relationships. Ontogenetic development mimics these phylogenies, which can be seen in speech acquisition in children: Children’s brains form and recall concepts and initiate actions before they utter their first