Neuroinformatics for Neuropsychology

Neuroinformatics for Neuropsychology Vinoth Jagaroo Neuroinformatics for Neuropsychology 123 Vinoth Jagaroo Depar...

Author: Vinoth Jagaroo

34 downloads 926 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Neuroinformatics for Neuropsychology

Vinoth Jagaroo

Neuroinformatics for Neuropsychology

123

Vinoth Jagaroo Department of Communication Sciences & Disorders Emerson College 120 Boylston Street Boston, MA 02116 USA [email protected] and Department of Psychiatry and the Behavioral Neuroscience Program Boston University School of Medicine 715 Albany Street Boston, MA 02118 USA [email protected]

ISBN 978-1-4419-0059-3 e-ISBN 978-1-4419-0060-9 DOI 10.1007/978-1-4419-0060-9 Springer Dordrecht Heidelberg London New York Library of Congress Control Number: 2009930050 © Springer Science+Business Media, LLC 2009 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

I dedicate this book to my parents, Barath and Sona

Preface

The idea for this book was conceived over many years and through many influences. The fields of neuropsychology, general neuroscience, and information technology were certainly among the main influences. It was in particular an unusual context in which I was on the one hand exposed to academic and clinical neuropsychology and on the other to information technology that gave rise to the ideas that would eventually lead to this work. I began thinking about informatics for neuropsychology more than a decade ago as a graduate student in behavioral neuroscience at Boston University School of Medicine. My track in this broad interdisciplinary area cut across neuropsychology, neuroanatomy and neurobiology, and my focus was visual cognitive neuroscience. I had concurrently held a position in a large information technology unit at the university where I gained experience in computer networks and database programming. The neuropsychology component of my training involved neuropsychological assessment, which was carried out at the Boston Veterans Administration Hospital, one of the teaching hospitals of Boston University Medical School. It was at these institutions that many legendary neuropsychologists had pioneered their craft and where some famous assessment instruments were developed. As I engaged in carrying out neuropsychological assessment, I could not help being struck by how comfortably this subspecialty of neuropsychology had contained critical problems tied to its origins and its development. Psychometric testing had played a huge part in the shaping of neuropsychological batteries and in some cases assessment batteries were nothing more than modified psychometric tests. When tests were developed from scratch in clinical neuropsychology, they were typically developed around symptom clusters or operational tasks. Assessment tools bore little tie to highly defined neuroanatomic systems or to rich conceptual frameworks of cognition. Where was the alignment between neuropsychological assessment tools, which were developed in earlier generations, and that rich body of theory on neurocognitive principles that had arisen through cognitive neuroscience and cognitive neurobiology, in a more recent generation? The “decade of the brain” had brought forth so many neural systems and modules that related, with relative precision, cognitive processes to the brain. In comparison, neuropsychological assessment tools and neuropsychological models of cognition appeared rather unsophisticated. It would have been possible to strive for reconciliation between vii

viii

Preface

assessment tools and functional neuroanatomic/neurocognitive systems if assessment took on a more computational dimension, but again, this consideration was absent in neuropsychology. My interest in the representational model of spatial neglect had me combing through primate neuroscience literature on posterior parietal mechanisms for coordinate-based spatiotopic transformations. The conventional assessment tools for neglect, e.g., line bisection, letter cancellation, and clock and figure drawings, by virtue of their simplicity, could generate dramatic pictures of neglect, but had no potential to relate to neural models of neglect. It was this problem that made me look to computerized methods, which in this case could be devised to tap into the complexities of neglect. I began work on an informatics system involving a grid-based screen interfaced with a database. The coordinates of presented visual stimuli and the gradients of neglect could be recorded and subjected to various kinds of analysis (this is described in a subsection of this book). Exploring informatics systems for neuropsychological applications inevitably had me surveying the larger field of biological informatics (bioinformatics) and its subspecialty in the neurosciences (neuroinformatics). The levels of sophistication attained by these disciplines were astounding as was the unique and transformative potential that they conferred. It was evident that modern biomedical science was inseparable from bioinformatics. The Human Genome Project was in large part a bioinformatics project and so much of the Human Brain Project centered on neuroinformatics. The absence of neuropsychology on the vast and flourishing landscape of neuroinformatics was stark and striking. The scenario was that most of the subdisciplines in neuroscience had discovered a powerful new technology, enabling novel methods of research, data analysis, problem solving, and knowledge building. With neuroinformatics, they could capture, manipulate, and visualize data in ways never before conceived. Neuropsychology, however, remained quite oblivious to this informatics-based revolution in the neurosciences. Neuropsychology, especially clinical neuropsychology, had by the 1980 s solidified an identity that had been shaped over many decades. It had developed a modus operandi that was intimately tied to its tools and models, most of which were rooted in periods that long preceded the modern era of cognitive-brain sciences. By the late 1990 s, informatics had become a tour de force in neuroscience, but neuropsychology, lying snug under its canopy of conventions, showed almost no awareness or understanding of the potential that was spelled by neuroinformatics. In February 2005, I presented a paper at the US annual meeting of the International Neuropsychological Society, in St. Louis, Missouri. The paper described the impact of neuroinformatics in neuroscience, and a case was laid out for neuroinformatics in neuropsychology. I soon after began to structure the paper as a manuscript for a review publication. Research for the paper brought me into contact with a small but steadily increasing number of individuals whose work in neuropsychology tied in with informatics. They shared valuable data with me and were also keen about a larger account of neuroinformatics in neuropsychology. During this period, the Internet had also been transitioning from its first generation to its second, marked

Preface

ix

by a host of web-based technologies for data modeling and collective knowledge building. Needless to say, with all these factors, what began as manuscript for a review publication quickly evolved into a book. This book introduces the field of neuroinformatics to neuropsychologists. It tours the field of neuroinformatics and articulates ways by which neuroinformatics can be integrated with neuropsychological research and practice. It describes various applications for neuropsychology. The book is an ambitious first account of neuroinformatics for neuropsychology – it discusses the kinds of changes required in the discipline for a successful integration with neuroinformatics, and it also lays out various issues that are likely to arise as neuroinformatics becomes an everyday part of neuropsychology. It presents a vision of 21st century neuropsychology defined by neuroinformatics. The book is aimed at neuropsychologists and to those in related disciplines – behavioral neurology, psychiatry, clinical psychology, speech-language pathology, cognitive psychology, and cognitive neuroscience. The introduction offered by this book is non-technical. The reader does not require a background in computer science or computational neuroscience. A reader of general neuropsychological literature will have no problem understanding the material presented. Numerous possibilities for the realization of neuroinformatics in neuropsychology are conveyed by this book. It is the author’s hope that the book will help accelerate discussion and enhance awareness of neuroinformatics for neuropsychology. A theme carried throughout the book is that neuroinformatics for neuropsychology is not an option but an inevitability brought about by technological and theoretical advances of our time. Boston, Massachusetts

Vinoth Jagaroo

Acknowledgements

I am grateful to the many individuals who helped make this book possible. The encouragement and support I received from my colleagues, Daniel Kempler, Cynthia Bartlett, and David Maxwell in the Department of Communication Sciences and Disorders at Emerson College, was simply invaluable. The very same must be said of the many years of generous support that I have received from Marlene Oscar Berman of the Behavioral Neuroscience Program and the Department of Anatomy and Neurobiology at Boston University School of Medicine. Research for a core section of the book, on neuroinformatics applications and models for neuropsychology, could not have been completed were it not for the cooperation that I received from twelve individuals – who graciously addressed my inquiries and provided me valuable data on their work: Dennis Reeves and Joseph Bleiberg (automated neuropsychological assessment); Mark Baggett, Mark Kelley, and Daniel Christensen (Internet-enabled assessment systems pioneered in clinical research programs of the US Army); Ho-Chuan Huang (computerized cancellation test system); Frank Guenther (computational model of speech production); Hubertus Axer and Jan Jantzen (aphasia database); Carol Neidle (sign language database project); Curtis Deutsch (behavioral phenomics and dysmorphology); and Robert Bilder (cognitive and neuropsychiatric phenomics). I also wish to thank Stephen Koslow, the former director of the Office of Neuroinformatics at the US National Institutes of Health, for reviewing my synthesis of the Human Brain Project, the Neuroinformatics Program of the National Institutes of Health, and the Neuroinformatics Working Group of the Organization of Economic Cooperation and Development. Jane Emes, my diligent graduate student assistant deserves enormous credit for her assistance with research and preparation of the manuscript. Finally, I am indebted to my colleague and friend, Jon Hemperley, the former manager of Information Technology at Office of the President at Boston University, for years of mentorship on information technology systems.

xi

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 1.1 An Overview of Bioinformatics . . . . . . . . . 1.2 What Is Neuroinformatics? . . . . . . . . . . . . 1.3 Bringing Neuroinformatics to Neuropsychology .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

1 1 2 3

. . .

7

. . .

7

. . . . . . . . .

8 10 10

. . .

12

. . .

12

. . . . . .

13 14

. . . . . .

18 20

. . .

21

. . . .

25

. . . .

25

. . . .

28

. . . . . . . .

29 32

2 Current Neuroinformatics Applications and Infrastructure . 2.1 Brain Image Construction, Analysis, and Morphometric Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Examples of Image Construction, Analysis, and Morphometric Tools . . . . . . . . . . . . . 2.2 Brain Image Atlases, Databases and Repositories . . . . . 2.2.1 Examples of Image Databases . . . . . . . . . . 2.3 Tools and Databases for Mapping Neural Structure and Connectivity Patterns . . . . . . . . . . . . . . . . . 2.3.1 Examples of Tools and Databases for the Study of Neural Architecture . . . . . . . . . . . . . . 2.4 Tools and Methods for the Simulation of Neurons and Neural Circuits . . . . . . . . . . . . . . . . . . . . . 2.4.1 Examples of Tools for the Simulation of Neurons 2.5 Database and Knowledge Discovery Systems for Clinical and Academic Research . . . . . . . . . . . . . . 2.6 Neuroinformatics Management and Infrastructure . . . . . 2.6.1 Examples of NI Organizations, Infrastructure, and Management . . . . . . . . . . . . . . . . . 3 Neuroinformatics for Neuropsychology . . . . . . . . . . . . 3.1 Differentiating Between the General Computer Applications in Neuropsychology and Neuroinformatics Systems for Neuropsychology . . . . . . . . . . . . . . 3.2 Defining Neuropsychology-Specific Neuroinformatics Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Neuroinformatics Applications and Models for Neuropsychology . . . . . . . . . . . . . . . . . . . . . 3.3.1 General Neuropsychological Assessment . . . .

. . . .

. . . .

xiii

xiv

Contents

3.3.2 3.3.3 3.3.4

Visuospatial Processing, Visual Attention, and Spatial Neglect . . . . . . . . . . . . . . . . . . . . . Speech, Language, and Aphasia . . . . . . . . . . . . Phenomics and Neuropsychology . . . . . . . . . . . .

4 Obstacles and Aids to Neuroinformatics in Neuropsychology . 4.1 Data Sharing and Issues of Privacy . . . . . . . . . . . . . 4.2 Standardization, Taxonomies, and Ontologies for Neuropsychological Data . . . . . . . . . . . . . . . . 4.3 Attitudes, Re-organization, and Training . . . . . . . . . .

43 58 75

. . . . . .

85 86

. . . . . .

88 91

5 An International Society for Neuroinformatics in Neuropsychology . . . . . . . . . . . . . . . . . . . . . . . . . . . .

95

6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . 6.1 Collaborative Knowledge Building in the Next Wave of the Internet: Web 2.0, Neuropsychology 2.0, and Global Network Innovations . . . . . . . . . . . 6.2 The Promising Politics of Informatics Infrastructure 6.3 Prologue to Neuropsychology for the 21st Century .

. . . . . .

99

. . . . . . . . . . . . . . . . . .

100 102 103

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

105

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

119

Chapter 1

Introduction

1.1 An Overview of Bioinformatics Over the past 25 years, the biomedical sciences have seen an unparalleled explosion in the amount of research data generated. The mission of understanding complex systems at discrete levels of analysis, be they protein structure or gene expression, and the devising of complex tools to carry out such studies have inevitably led to massive amounts of data and the need to process the data. Large-scale undertakings in certain biomedical avenues, such as the mapping of the human genome, have meant that large data sets would be generated by multiple research teams. Often these teams would be working across disciplines and across international borders, each team dealing with an aspect of the greater research project. In such contexts, the need to rapidly code and index data, share the data in a common database, and devise flexible cross-referencing and retrieval systems would be vital to the success of the undertaking. Occurring in parallel and serving as the critical enabler of biomedical research have been the many strident advances in computer technology. Cheap and powerful relational databases, Internet-accessible database systems, server–client computer architecture, and visualization and simulation software have provided the essential tools that have greatly propelled biomedical research. So much of modern biomedical research is so integrally woven with computational technology that the computational environment is built into the very identity of biomedical research. Bioinformatics is the name given to the discipline that has been formed around this very specialized application of computer science and information technology to conceptual, investigative, and cataloguing challenges in biology and medicine. Bioinformatics, as a discipline, has carved a distinct identity between general biological science and computer science. It arises from both these disciplines but gains its distinction from its very unique crafting and application of computational technologies to biological complexities that can be effectively investigated only with the application of these computer-based methods. Numerous bioinformatics books and journals have been published over the past two decades and many undergraduate and graduate bioinformatics programs have emerged. Genetics and molecular biology are the biomedical subdisciplines that first realized a research dependence on informatics systems. Hence these disciplines were V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9_1,

1

2

1

Introduction

the first to boast elaborate informatics systems and a very well developed bioinformatics infrastructure. The most well-known bioinformatics undertaking has been the Human Genome Project (HGP), coordinated by the National Institutes of Health and the US Department of Energy (see Collins et al. 2003; http://www.genome.gov/; http://doegenomes.org/). The HGP ran from 1990 to 2003 and was a visionary project. Its primary goals were to identify all the genes of human DNA (the human genome) and the sequences of their base pairs. The project was officially completed ahead of schedule, in 2003, when its central goal – the sequencing of the human genome – was met but work continues on some remaining goals. Well represented within the HGP were numerous aspects of a large-scale bioinformatics undertaking. It grew out of the need to process massive amounts of data and elaborate data sets in order to understand a complex biological system. It encouraged innovative computerized tools designed to capture data and it shared these tools among the HGP community. The data generated were fed into centralized databases, easily accessible to research teams internationally. The data had multiple uses. Analytic tools were devised for the specialized assessment of the data (and some teams would focus only on the assessment of captured data). The HGP involved biologists, mathematicians, computer scientists, and IT specialists, melding their skills in various ways. It was an international collaborative research effort with well-defined goals and it had a centralized coordinating body. A clear understanding of the distribution of tasks and the establishment of uniform research protocols and IT platforms made for a streamlined global mission, maximizing resources while minimizing redundancy of effort. The HGP had strong organizational leadership that guaranteed funding and infrastructure for continuity of the project. As indicated by the example of the HGP, there are many levels at which bioinformatics can be practiced. Bioinformatics can refer simply to algorithms, databases, and analytic tools used to further biomedical research. At broader levels, it may encompass a large organizational system for project coordination and an IT infrastructure for integration of data sets. Genetics research and informatics systems have defined such a powerful interface that the term genomics was coined to describe this area. Similarly, the extent of informatics dependency in studying protein structure has given rise to the field of proteonomics. Numerous other subdisciplines in biology and medicine have since inevitably been drawn into bioinformatics.

1.2 What Is Neuroinformatics? The broad field of neuroscience has clearly been a burgeoning field. It has continued to face ever-complex research questions and large data volumes. It is not surprising therefore that neuroscience has taken to bioinformatics. The many applications of bioinformatics in the neurosciences have come to be labeled by the term "neuroinformatics" (Huerta and Koslow 1996; Shepherd et al. 1998). Neuroinformatics is in itself a richly constituted subdiscipline with avenues that extend from neuroimaging to neurobiology. Some respective examples are databases of fMRI images and sets of algorithms to help connect functional patterns; computerized simulation tools

1.3

Bringing Neuroinformatics to Neuropsychology

3

for neuronal conduction and dendritic branching; and a data mining system that can search published neuroscience data (electronic files) and then generate probable relational patterns between data fragments. Initially neuroscience was slow to adapt to a bioinformatics approach when compared to genetics and molecular biology (Koslow 2000). In more recent years, however, neuroinformatics has rapidly gained momentum, as will be illustrated through the many examples described in Chapter 2. A major initiative in neuroinformatics has been the Human Brain Project (detailed in following chapters). It was sponsored by the National Institutes of Health and a few other US federal agencies, with the explicit aim of furthering brain research through application of informatics (Huerta et al. 1993; Shepard et al. 1998). The project has been in essence a neuroinformatics initiative – to database the brain – and it has supported many types of projects that have been drawn into this central goal. Its significance to the brain sciences has been as great as the significance of the Human Genome Project to the field of genetics. It has spawned a number of major neuroinformatics developments. These developments together with many independently developed neuroinformatics tools and networks have come to be spread over almost every subdiscipline of neuroscience. The Digitization of Neuroscience Data: Common to many of the facets of neuroinformatics (NI) are a few essential features to do with the digitization of neuroscience data. With the aid of computers, other specialized hardware, and database applications, data are first captured at very fine levels. The data may then be stored, manipulated, analyzed, and reconstructed. This general digital format and the computational platform offer a greatly enhanced degree of complexity in describing structural or functional properties of the nervous system. The key features in the NI approach to data are summarized in Table 1.1. With the digital approach to data capture, the degree of data complexity can be more closely in tune with the actual biological complexity of the object of function being studied. The richness of the data capture is often enabled by specialized hardware. These tools are quite unique as are the kinds of analyses and relationships that can be derived from the data they generate. Conceptually and operationally, they are on a very different level when compared to conventional tools or methods. These are the features that make NI a very unique and powerful avenue in neuroscience research.

1.3 Bringing Neuroinformatics to Neuropsychology In neuropsychology, the application of NI is yet to be realized. NI perspectives in neuropsychology are presently seen only in quite a few projects in development but the field of neuropsychology lacks awareness of the importance of NI. NI has transformed neuroscience and is bound to have far reaching effects in neuropsychology. A case for the systematic incorporation of NI in neuropsychology was articulated by the author in February 2005 meeting of the International Neuropsychological

4

1

Introduction

Table 1.1 Key features of the NI approach to data Aspects of data processing

Key considerations

a. Data capture

• Emphasis on digitizing the data – capturing or coding the data in ways that can be stored and manipulated by computers. • Capturing data at multiple levels of abstraction – data about all levels of structure/function and data on as many variables as feasible. • Indexing the data in ways that are optimal for database queries/searches and pattern-seeking algorithms.

b. Data storage and sharing

• Databases with data fields to accommodate multiplicity and breadth of data to enable multiple levels of inference and analysis. • Tagging or coding data; saving text, images, and data about spatial and functional relationships. • Web-accessible centralized databases or shared databases through server networks. • Data availability for current or future use – prospective and retrospective study needs.

c. Data analysis, modeling, manipulation, rendering, and knowledge discovery

• Data mining – statistical techniques designed to mine the data for patterns and relationships; search algorithms tuned to complex relationships among the data. • Data modeling – manipulate one or more variables; an application can use real or hypothetical contingencies to model the effect of one set of data on another. • Reconstruction – using the binary data to create realistic 3D spatial renderings of structures, or probabilistic maps; semantic relationships among text data can be expressed in spatial form. • Ontology building – computer readable models of the properties and relationships of elements of a data domain; aids the knowledge discovery process

Society (INS) in St. Louis, Missouri. At the June 2008 INS meeting in Hawaii and at the February 2009 meeting, Robert Bilder laid out compelling perspectives on informatics frameworks for neuropsychology (these are discussed at length in later chapters). Both clinical and experimental neuropsychology have certainly built an increasingly close relationship with specialty subdisciplines like neuroimaging, to which NI is central. Yet, neuropsychology in itself has not yet realized the application of NI to subject areas that are definitive to the field of neuropsychology. Traditional neuropsychological research on the functional domains of language, visuospatial function, memory, etc., tests and assessment batteries used in neuropsychological assessment, and neuropsychological models for describing clinical syndromes remain starkly unengaged with the activities of NI when compared to other subdisciplines of the brain–behavioral sciences. Because neuropsychology is concerned more with cognitive behavior and deals with the brain at a level that does not delve into the minutiae of laboratory-style bench neuroscience, it could appear that it is incongruous with the very technical stance of NI. This book attempts to demonstrate

1.3

Bringing Neuroinformatics to Neuropsychology

5

instead that neuropsychology can utilize NI as richly as NI has been exploited by the other areas of neuroscience. The purpose of this book is to describe the relevance of NI to neuropsychology. With some innovation and creativity, NI systems can be tailored for every context of neuropsychology. Computer hardware and software can be modified to capture neuropsychological data in digital format especially with regard to clinical assessment. The data can automatically be fed into databases. Pattern-seeking algorithms can mine these data and seek logical relationships between data sets. Neuropsychologists can decide on how to structure centralized computer platforms for the sharing of primary data. Modeling tools can be devised to refine both neuropsychological models of normal cortical function and models of neuropsychological disorders. NI techniques can also help better integrate neuropsychological data with converging results from other subdisciplines of neuroscience. In short, NI tools and database techniques can be used to capture a much finer grain of neuropsychological data. The renderings of complexity and the computational sophistication that NI can bring to neuropsychology lie in contrast with the fundamental limits of conventional or non-computational neuropsychology. NI can add a new dimension to neuropsychology, giving the field a much enhanced level of investigative, descriptive, and conceptual power. The Structure of the Book: Demonstrating the relevance of NI to neuropsychology requires that the application of NI in other areas of neuroscience first be sampled. In making this illustration, the objective is to provide more than just a tour of the field of NI. Only by conveying some important details of various NI examples across the neurosciences can the extrapolation of NI to neuropsychology be appreciated. Chapter 2, therefore, surveys numerous NI applications in the neurosciences and describes NI management and organizational structures. Chapter 3 begins by placing neuropsychology in the current NI landscape. It critically distinguishes between (a) the general application of computers in neuropsychology (including computerized assessment) and (b) NI systems for neuropsychology. Criteria are suggested for defining tools that can be considered NI tools specific to neuropsychology. Nine examples of NI applications or models for neuropsychology are then presented. Seven of them relate to three broad areas of neuropsychology: assessment and test-battery development; research and assessment of visuospatial processes; and research on speech, language, and aphasia. Two examples relate to the field of cognitive phenomics. Brief summary of the examples of NI applications/models for neuropsychology discussed in Section 3.3: • An automated, digitized assessment battery structured in a highly modular fashion where each subtest generates a unique data file; all data files can be seamlessly uploaded to a database where queries can be applied to sort the data in various ways or search for patterns among the data. • An Internet-enabled neuropsychological assessment battery linked to a central server/database, allowing for ease of inter-subject comparisons and

6

• • • • • • •

1

Introduction

population profiling, and especially suited to assessment carried out at remote locations. A computational, screen-based grid system that captures spatial coordinates of the visual field, offering a digital approach to analyzing neglect and to introducing perceptual dynamics that can alter the gradient of neglect. A computerized tool that records visual search and visual attention patterns and applies algorithms for graphical analysis of the visual search/attentional patterns. A theoretical and computer-simulated model of speech motor skill acquisition and speech production that can intelligently use motor and acoustic cues to modulate vocal tract movement. An online database of symptomological and neuroanatomic data in aphasia, designed as a research tool to help clarify aphasic syndromes that have overlapping symptoms. A multimedia tool for transcription, analysis, and databasing of signs and gestures in visual/gestural communication. A database initiative for the mapping of relationships between craniofacial features and neuropsychological features in neurodevelopmental and neuropsychiatric disorders. A research consortium for neuropsychiatric phenomics that is developing numerous novel NI tools tied to data integration and knowledge building in neuropsychology.

A considerable degree of background is described before the examples are presented. This helps to give perspective to the gains brought by NI in each of the examples. Chapter 4 also considers various obstacles and aids neuropsychology can expect in adopting an informatics approach. These are grouped into three broad classes pertaining to data sharing/privacy, standardization of methods in neuropsychology, and general reorganization in the field. Ways of overcoming the obstacles are discussed, drawing on lessons learnt from the practice of NI in general neuroscience to date and on some trends in neuropsychology. Chapter 5 describes the mission of the newly formed organization, The Society for Neuroinformatics in Neuropsychology (SCNN). The need for a forum for discussion and communication about NI in neuropsychology is laid out, as are the plans of SCNN to address the needs. Chapter 6 offers some concluding remarks.

Chapter 2

Current Neuroinformatics Applications and Infrastructure

Neuroinformatics is a multifaceted field. It is as broad as the field of neuroscience. The various domains of NI may also share some common features such as databases, data mining systems, and data modeling tools. NI projects are often coordinated by user groups or research organizations. Large-scale infrastructure supporting NI development is also a vital aspect of the field. There are many possible ways of describing the field of NI. Descriptions of NI can vary depending on the research area and the type of NI application of interest. The areas of NI surveyed in this section are among the most commonly described clusters in the NI landscape (see Amari et al. 2002; Gorin et al. 2001; Huerta and Koslow 1996; Shepherd et al. 1998). They provide a sampling of the field. Some of the examples describe smaller projects and some describe major developments. It will be noteworthy to consider the following questions while going through this section for they may be parallels when addressing NI applications for neuropsychology: What forces in the field led to the development of the NI tool or user network? What is the research or conceptual problem being addressed and how is NI applied? What are the unique elements that NI brings in addressing these research questions? What impact has the example of NI had? How have user groups shaped the field and why is a coordinating body important when developing a new area of NI? How can conceptually similar NI applications be developed for neuropsychology? There are a few fundamental concepts and methods (e.g., data mining) that apply broadly across NI, which are also referenced throughout the book. They are briefly clarified in this section as they are encountered. There are also variations in the use of terminology in NI and these are also clarified at different points.

2.1 Brain Image Construction, Analysis, and Morphometric Tools Functional magnetic resonance images are the “end products” of an elaborate and complex sequence of experimental steps and computerized data processing. The variables involved in the process are immense (see Buxton 2001; Huettel et al. 2004). The raw data that are captured are typically in the form of numerous graphed signals tied to voxels (and they have the appearance of approximated sine wave V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9_2,

7

8

2 Current Neuroinformatics Applications and Infrastructure

or time-series graphs). The raw data look nothing like the colorful brain images that are synonymous with fMRI. The data are fed into computerized tools that filter and analyze the data in terms of signal strength, temporal properties, etc. They then necessarily use a 3D brain coordinate system so that the data can be viewed (transposed) as a neuroanatomic activation pattern in the brain. A suite of computerized tools enable the transformations from raw data to 2D images, 3D volumetric reconstructions, surface maps, etc. The generation of 3D images from 2D image slices in a single subject typically involves steps such as image slice assembly formatted for a particular computer application for image analysis; image quality checks; realignment and smoothing of the assembled slices; removal of non-brain imaged parts/structures; time-series analysis stimulus and inter-stimulus activity; corrections for artifacts (e.g., head movement); and segregation in differential patterns of activations (blobs and regions of interest). In experiments involving multiple subjects the same steps apply to each subject but the activations blobs are averaged across subjects. Each and every one of these steps is dependent on highly specialized computerized tool sets. These are powerful informatics tools without which fMRI images would not be possible. A few examples of such tools are described below (and the aim of this subsection is merely to describe some of these tools, not the specialized domain of brain imaging).

2.1.1 Examples of Image Construction, Analysis, and Morphometric Tools Analysis of Functional Neuroimages (AFNI): AFNI is a set of programs for fMRI image processing, analysis, and visualization. Altogether, these programs also constitute a technique and an environment for mapping and displaying functional brain activity. AFNI is primarily a research tool. Robert Cox leads the development of AFNI at the National Institutes of Health (he began writing the AFNI suite in 1994 at the Medical College of Wisconsin). The initial impetus for the development of AFNI was the need to view (transform) fMRI results in “Talairach space,” a 3D coordinate system used to describe the locus of brain structures and activation patterns.1 Various programs and data sets were added and by 1996 AFNI was established as

1 Brain coordinate systems are used to define a “standard brain” against which to describe the locus

of neuroanatomic structures and activation patterns. They are necessary to imaging – data sets are “normalized” to the space defined by a particular coordinate system (a process of stereotaxic transformation) so that voxel-based computations can be carried out. One such system is the Talairach system that derives from the Talairach and Tournoux Atlas, which was based on the brain of a single individual (see Talairach and Tournoux 1988; 1993). Another is the MNI system developed at the Montreal Neurological Institute in collaboration with the International Consortium for Brain Mapping (see Section 2.2). This system was developed from a sample of 152 individuals and is currently the most commonly used template for brain imaging. Some imaging software suites can convert data between the Talairach and MNI systems.

2.1

Brain Image Construction, Analysis, and Morphometric Tools

9

a comprehensive suite for fMRI data processing. AFNI’s basic unit of data is a set of 3D volumetric data – an array of voxels, for which AFNI codes various voxel values. AFNI provides a number of ways of viewing the imaged data, for example, in axial, sagittal, and coronal views; one slice at a time; many slices at once; with functional overlay (merging two data sets); with anatomical overlay (merging a subject’s functional data with anatomic data from a database); as volumetric renderings; as time-series graphs; etc. AFNI has evolved into a very comprehensive image analysis package, currently consisting of about 300 external programs and plug-ins that work with the core application. Each deals with a very specific aspect of image analysis or rendering. AFNI has been able to achieve this level of sophistication in part because it is also an open source platform for development of new software. Users can download and compile AFNI source code to program new features. Core developers of AFNI at the NIH and the AFNI user community work together to iron out bugs and address questions via the AFNI website. AFNI is a prime example of a dynamic platform for neuroinformatics development. (URL: http://afni.nimh.nih.gov/afni) Computerized Anatomical Reconstruction and Editing Toolkit (CARET): Developed at Washington University School of Medicine (St. Louis), CARET is a powerful software application for generating and manipulating cortical surface reconstructions. It is an open source software available to researchers and is accompanied by comprehensive online tutorials. CARET is designed to accommodate various kinds of experimental data (neural connectivity patterns, anatomically or functionally defined cortical geography, neuroimaging scans, etc.), which it can use to produce surface reconstructions of cortical gyri. For example, CARET can use a few neuronal connectivity maps of a gyral region or a few functional images of the gyral region and from the data, it can project a gyral surface contour map. It can also piece together individual image sections to create an entire gyral surface. It can further manipulate the reconstruction to generate, for example, flat maps (an unfolded gyrus) and inflated maps. Surface maps can be thought of as a generic form of expressing brain images and neural organization. The advantage of these maps is that they can represent the integration of various data types arising from different studies. In accommodating different data types, CARET also serves as a repository that successfully brings together and compares the data. (URL: http://brainmap.wustl.edu/) 3D Slicer and FreeSurfer: Both 3D Slicer and FreeSurfer are software packages freely downloadable from the Biomedical Informatics Research Network (BIRN), an informatics consortium. The 3D Slicer utility has similarities with CARET in that it enables neuroanatomic visualization and image segmentation. It can, for example, process a segment from each of three brain scan slices (axial, sagittal, and coronal planes) and then produce a 3D surface model highlighting the segmented region, a function particularly important in the visualization of tumors or other lesions. The development of this software is ongoing. It is the product of collaboration between the Artificial Intelligence Laboratory at Massachusetts Institute of Technology and the Surgical Planning Laboratory at the Brigham & Women’s Hospital (Boston). FreeSurfer is a set of software tools that can also reconstruct a cortical

10

2 Current Neuroinformatics Applications and Infrastructure

surface using imaging data. Using data about functional characteristics of a cortical region it can then overlay these data on the reconstructed cortical surface. This functionality is well geared to extract cortical regional correspondence across large data sets. FreeSurfer has been developed at the Center for Biomedical Imaging at Massachusetts General Hospital with support from the BIRN, the NIH, and other agencies. The examples described above are a sample of the range of tools available for imaging analysis. Without these tools, the elaborate forms of data image manipulation and data modeling that stem from imaging data would not be possible. Recognizing that “[I]nformatics tools are key at all stages of neuroimaging, allowing scientists to control highly sophisticated imaging instruments and to make sense of the vast amounts of complex data generated by them” (http://nifti.nimh.nih.gov/background), the NIH launched the Neuroimaging Informatics Technology Initiative (NIfTI). Its mission is to help researchers enhance the utility of such tools and to foster unification of tool platforms between different research groups using a community-based approach. (URLs: http://nbirn.net/research/morphometry/visualization_tools.shtm http://surfer.nmr.mgh.harvard.edu/)

2.2 Brain Image Atlases, Databases and Repositories With the advent of computerized neuroimaging and the churning out of brain images came the need to store the images. Brain image databases would serve as image repositories, enable rapid retrieval of images (scans), and provide a means to compare images. The emergence of functional neuroimaging only solidified the need for imaging databases. Brain image databases have also meant that increased attention would be given to the processing of imaging data, post acquisition. As Wong and Huang (1996) have highlighted, (a) this post-acquisition image analysis capability allows for the maximal analysis of the data, even long after the data have been acquired, and (b) the informatics tools for managing and sharing these data have the potential to reshape the clinical environment in the decades ahead.

2.2.1 Examples of Image Databases The Whole Brain Atlas: The Whole Brain Atlas is an example of a relatively simple open access online utility. It offers a variety of images and some useful ways of manipulating neuroanatomic and scanning variables. Scans obtained through CT, PET, SPECT, and MRI are organized along five categories: normal brains, cerebrovascular disease, neoplastic disease, degenerative disease, and inflammatory or infectious disease. More than a hundred brain structures are highlighted in the images of normal brains. With many of the images, the screen interface allows the user to adjust the position of the scan axis to produce a corresponding image slice. This allows for navigation through successive slices across a

2.2

Brain Image Atlases, Databases and Repositories

11

particular axis (horizontal, coronal, and sagittal). Some of the images can be regenerated by changing the weighting parameters (e.g., T1 vs. T2), thereby reflecting differences in tissue-image characteristics. A well-organized database of images presented through a simple online interface makes the Whole Brain Atlas a compact and powerful utility. The image data can be used simply to tour the brain slice by slice or as a brain scan reference for normal and diseased brains. (URL: http://www.med.harvard.edu/AANLIB/home.html) The International Consortium for Brain Mapping (ICBM) Subject Database: The ICBM Subject Database is one component of a multifaceted neuroinformatics project based at the University of California, Los Angeles. It is a web-interfaced database of images and is at the center of a much broader NI infrastructure. Among the goals of this NI infrastructure is the fostering of unification in (a) the collection of images from various sources and (b) the organization these images and their dissemination among researchers. The ICMB database is also a large and continually growing image database. Its sophistication lies in its powerful functionality: Variables and attributes relating to the subject (individual) and to the actual scan can be set in order to refine a query. The query interface offers, for example, search fields for the sex and weight of the subject and image attribute search fields for modality (MRI, PET, etc.), weighting, pulse thickness, slice thickness, and acquisition plane. A collaborator registered with the ICBM can use this unified set of data fields to upload images and view customized representations of the larger data set. By bringing organization and structure to image data set collection, the ICMB subject database provides an effective and convenient system for collaborators to archive, share, and retrieve image data. (URL: http://www.loni.ucla.edu/ICBM/Databases/) The Database of the NIH MRI Study of Normal Brain Development: The NIH MRI Study of Normal Brain Development is a large-scale, multi-centered study that tracks normal brain development in healthy children and adolescents. The greater goal of the study is to provide a large reference data set on normal brain maturation that will aid in understanding a range of neurodevelopmental conditions. The study has recorded MRI and cognitive/neuropsychological data from approximately 500 children, from newborns to age 18, across the United States. Data collection began in 2001 and was completed in 2007. It is, to date, the most extensive MRI study of normal brain development in children. The study is sponsored by the NIH and involves seven participating pediatric centers in the United States and the Montreal Neurological Institute (MNI) in Canada, which acts as the data coordinating center (see Almli et al. 2007; Evans 2004; Evans and the Brain Development Cooperative Group 2006). A study of this scale overcomes problems of small sample sizes and problems of different data types being spread across different (incompatible) databases. The pediatric MRI data repository that has resulted from the study resides at MNI and contains a unique collection of multi-spectral structural MRI images. The images were fed into the databases via the web network linking the participating centers. Since the database also contains behavioral data, it allows researchers to relate cognitive and motor changes in growth to changes in brain anatomy. The database is web accessible and is available internationally to qualified researchers.

12

2 Current Neuroinformatics Applications and Infrastructure

As with any NI project of this type and scale, new NI methods are developed in the process. The NIH/MNI MRI data repository demonstrated new methods for MRI segmentation, web-accessible image databasing, and a workable architecture for large-scale collaboration. (URL: http://www.bic.mni.mcgill.ca/ nihpd/info/index.html)

2.3 Tools and Databases for Mapping Neural Structure and Connectivity Patterns The study of neural connectivity patterns and neuronal structure using staining and tracing methods constitutes one of the most major areas of neuroscience research. Many decades of work on the neural mapping of mammalian (and non-mammalian) brains have produced masses of data on neural architecture. A commonly occurring problem in this research has to do with correspondence between maps produced by different research teams – also known as the parcellation problem. There has been a lack of shared coordinate reference system when describing neural patterns at the microanatomic level. If, for example, there are numerous research findings about the connection patterns between two regions of the cortex, there needs to be a way of extracting a pattern common to all the findings. This is particularly so if the findings are to assist in functional interpretations. Other challenges have been encountered in the study of ultrastructural neurocytology. Here, many of the technical problems have had to do with the parsing out and visualization of extremely small cellular structures – often in the range of less than 1 nanometer to a few nanometers. It is no surprise then that with the advent of neuroinformatics, the study of neural architecture is seeing rapid reorganization, sprouting off an array of tools and databases.

2.3.1 Examples of Tools and Databases for the Study of Neural Architecture Collations of Connectivity Data on the Macaque Brain (CoCoMac): CoCoMac is a set of databases aimed at collating all known neural wiring patterns in the primate brain. It is run by the Computational Systems Neuroscience Group at the Vogt Brain Research Institute in Düsseldorf, Germany, with institutional collaborations across Germany, the US, and the UK. Tracer is the main CoCoMac database containing data on cortical neural connectivity patterns extracted from many thousands of studies. In addition to data about labeled (stained) cortical sites, their laminar patterns, etc., Tracer also stores information about the experimental conditions and methodologies of the studies that produced the data. A significant feature of CoCoMac is the mathematical method it uses to overcome the parcellation problem. Data on a particular cortical region collected from divergent brain maps are converted into a data set that is independent of the coordinates of each map. This is done through algorithms that take into account the relative arrangement of structures described in the divergent brain maps. CoCoMac data can also be exported to CARAT. Such interface between NI tools signals an unimaginable potential of NI systems developing meta-NI networks in the future.

2.4

Tools and Methods for the Simulation of Neurons and Neural Circuits

13

The parcellation problem is also attended to by other computer applications and graphical databases such as XANAT (http://redwood.psych.cornell.edu/ bruno/xanat/xanat.html), which provides a standardized format for putting the results of tracer studies and a means to query the net result of connections to a cortical region. (URL: http://cocomac.org/home.asp) BrainMaps.org: BrainMaps.org provides an extensive compilation of very high resolution histological images of the rhesus monkey brain. Numerous stained preparations sectioned at very close serial intervals across the coronal, sagittal, and horizontal planes are provided through a convenient web interface. Before being input into the database, the images are magnified greatly and then scanned at very high resolution. The scale of the resolution of the images (ranging roughly from 0.5 to 8 microns per pixel) serves more than just a primary need for image clarity. Using image-viewing software integrated into the website a user can zoom into any part of an image resulting in the same kind of magnification quality that a user experiences when zooming with a microscope. The resolution of images is such that the user can essentially view them at the gross anatomical level or zoom into them at a subneuronal level. Since the image slices across a plane are sectioned serially at close (micron-scale) intervals, virtual 3D navigation between images is also made possible. A remarkable level of functionality is achieved with the simple combination of high-resolution images, a web-accessible database, and appropriate image-viewing software. (URL: http://brainmaps.org/) Synapse Web: Synapse Web is a portal that describes a method and provides a software tool for digitally rendering 3D images of neuronal ultrastructure. Creating a realistic 3D graphical reconstruction of a neuronal segment such as a dendritic protrusion is a challenge. Multiple images of the structure (on a scale of microns) must first be captured, and the final 3D rendering must ideally convey the details observed at the microscopic level. Synapse Web describes a method that employs stained serial tissue slices, electron microscopy, and specialized software for image reconstruction. A high-magnification (electron microscope) view of each serial slice is photographed. Subcellular structures in the nanometer range are captured in these images. Digitized photographs are then sequenced to match the respective positions of slices of the original structure. Using a piece of specialized software called Reconstruct images can be appropriately cropped, scaled, and aligned so as to produce a 3D extraction of the structure of interest. The reconstruction can be viewed in static form or in a virtual reality format. NI tools developed to map out neural architecture and to render views of neural structure provide an obvious breakthrough in studying these complex systems. (URL: http://synapses.clm.utexas.edu/)

2.4 Tools and Methods for the Simulation of Neurons and Neural Circuits Just as the biophysical properties of neurons and neural conduction are of interest to cell biologists, the dynamics of neural functioning and neural circuitry are of interest to computational neuroscientists – those who deal with mathematical modeling of

14

2 Current Neuroinformatics Applications and Infrastructure

neural systems. Observing intercellular and intracellular neural events is extremely difficult. This is the case not only because of the microscopic scale of the environment but also because of the multitude of variables and permutations involved with neurons. Computer modeling of these neural processes therefore provides a means for understanding these events. Modeling may relate to events tied to a neuron (e.g., ion transfer across a membrane) or the collective workings of circuits of neurons (neural networks). Two examples of neural simulation tools are described below. Both of the tools relate primarily to events tied to neurons as opposed to properties of a large neural circuit. The latter case is primarily the focus of the field of neural modeling – the modeling of neural circuitry in order to mimic neural information processing. Neural modeling is a specialized subdiscipline that is not necessarily related to bioinformatics. However, bioinformatics applications or methods commonly apply modeling techniques. A brief recap is given to neural modeling further in this subsection because it can apply so broadly to NI (and a few of the NI applications discussed in subsequent sections do incorporate modeling methods).

2.4.1 Examples of Tools for the Simulation of Neurons General Neural Simulation System (GENESIS) is a computer platform for generalpurpose simulations of a range of neural systems from subcellular components to larger neural networks (see Bower and Beeman 1998; Bower et al. 2003). The development of GENESIS began at California Institute of Technology but has since been spread across a few US institutions and internationally. GENESIS is focused on structurally realistic computer simulations of neural functioning. Consider an example in which a section of an axon needs to be simulated: One component of the GENESIS platform, the Object Library, can provide a building block object (cylindrical shape). To this, the experimenter can add membrane channels and a synapse at the terminal end. Various neuronal properties such as the makeup of intracellular ions or the number of dendritic channels can be entered into a simulation or generated as a result. Another component of the platform, the Kinetic Library, allows for high level of molecular modeling using parameters as specific as a molecular reaction. The various “libraries” of the platform enable the user to obtain pre-made components that have been programmed to realistically simulate a neural component. The Channel Library, for example, contains many types of potassium, sodium, and calcium channels; a library of single neurons contains small section of varied neurons. To fully set up and run the simulation the user has to write some commands (using a simulation language). As with many advanced computer programs, commonly used programming sequences are packaged as scripts which the user can simply apply. Because of this object-oriented approach to simulation, a user with limited programming skills can successfully carry out some kinds of simulations. More advanced users can program more complex sets of variables and parameters into a simulation. Not uncommon in NI, the open platform environment of GENESIS confers enormous power on this simulation system. It provides its users with a

2.4

Tools and Methods for the Simulation of Neurons and Neural Circuits

15

well-provisioned modeling workspace. Bearing testament to the story of GENESIS are many hundreds of users and more than 60 institutions across numerous countries where GENESIS is used for educational purposes (Bower et al. 2003). GENESIS is a good example of dynamic, continually expanding NI platform, shaped by its users. (URL: http://genesis-sim.org/) Neuron: Neuron is another simulation environment for empirically based modeling of neurons and neural networks (see Hines 1994; 1998; Hines and Carnevale 2001; 2003). It originated at the Department of Neurobiology at Duke University and has since become a collaborative project with various departments at Yale University. Neuron places more emphasis on simulating the biophysical events to do with signal transmission within a neuron. Its simulation schemes are well attuned to variables of a neuron’s membrane properties, ion channels, cytoplasm, etc., but as a general simulation platform, Neuron is much more flexible. Neuron is also centered on simulating these physical and biochemical neural processes without indulging in a heavily mathematical/computational level of analysis. This makes Neuron ideal for those interested in the essential neuroscience of a functioning neuron. Neuron also uses an object-oriented system to set up a simulation model. Among its many features is CellBuilder, a utility with which a user can simply build a model by adding pre-built components. For example, a section of membrane containing ion channels that operate along Hodgkin–Huxley principles can simply be added to a neuron model. A user can also modify a component without having to directly edit the program code (application language). User groups, tutorials, and meetings to do with Neuron help expand this powerful simulation platform. Tied to the Neuron simulation environment are some specialized databases and modeling tools concerning neural structure and function (accessible via the Neuron URL). NeuronDB is a database of neurotransmitter substances, neural receptors, and types of voltage-gated conductance. SenseLab is a collection of neuronal and neural system models that apply to the olfactory system. ModelDB provides a platform to store and utilize published computational models of neurons, especially models that have arisen using the data from NeuronDB. Some Basics of Neural Modeling2 (See McClelland et al. 1986; Churchland and Sejnowski 1994; Arbib 1995; Rowe 1995; O’Reilly et al. 2000; Dayan and Abbot 2005). The terms computational modeling, neural network modeling, or neural modeling all describe the mathematical modeling of biological neural networks. An artificial neural network, or simply a neural network, is the resulting computerbased model that is to some degree modeled on neural networks in the brain. Modeling is especially useful in understanding parallel processing that is typical in large, widely distributed neural networks, hence the term parallel distributed 2 Computational

modeling is the dominant method of simulating neuronal circuits and as such is briefly covered in this subsection. The outline of main concepts given here will also aid in better understanding a few of the NI examples that the reader will encounter in Section 3. A comprehensive description of neural modeling is well outside the scope of this book. The interested reader is referred to the original sources cited.

16

2 Current Neuroinformatics Applications and Infrastructure

processing. In such a system, the net function is the result of the entire network, while each neuron is defined by connectional and functional properties that give it a distinct role in the collective function. Modeling can be viewed as a way of drawing abstractions of workings of a complex system such as the brain and thereby approximate neural functioning. There are a few areas of concentration in the field. There is a lot of focus on understanding actual neural systems in terms of their functional neurobiology and their role in cognition and then attempting to model these systems. On the application end, artificial neural networks are often the basis for artificial intelligence; hence neural modeling is closely tied to practical applications of robotics and engineering initiatives to develop computers that closely mimic human information processing.3 Though artificial neural networks (hereon referred to as “neural networks”) are generally much more simplified than actual networks in the brain, they help to guide research. Structure of Neural Networks: An artificial neuron (or processing unit) is the building block of a network. These units are interconnected such that they can influence each other and in this way modulate the larger outcome of the system. The term node may describe a single neuronal unit or a cluster of them that serve a common function in the network. Neurons can be arranged in various ways, for example, a network can have 100 neurons divided into 10 clusters and each cluster can be linked to each other, or a network may be built of multiple layers of neurons, each layer carrying out a distinct function and feeding information to the next layer. A certain pattern of input to the first layer of the network results in a differential pattern of activation across its neurons. Different “afferent” signals are then propagated and are received by the next layer. Successive stages of processing are carried across the network until an output layer generates a final result. There are many variables and functional parameters involved in neural networks and many of these are specified when designing neural networks. They include a host of neuronal properties such as pattern of conduction (influenced by ion channel properties), types of dendritic formations, numbers and strengths of synapses, how a neuron summates its signals, the rate of firing, and the action of neurotransmitters and receptors. In general, the closer the neurons of a network simulate integration of

3 Some

readers may have encountered the term “Cognitive Informatics,” a term coined by Wang (2002). The claim has been made that cognitive informatics is a new, innovative, emerging discipline concerned with (a) mathematical and computational approaches to understanding neural and cognitive systems, especially in terms of how they represent abstract knowledge, and (b) how this understanding can potentially lead to new types of computer architecture (see Wang 2003a,b; 2006; 2007a,b). These two goals, however, have been at the core of cybernetics and neural modeling for more than five decades (see sources cited earlier on neural modeling). On closer inspection, “cognitive informatics” is largely an approach to modeling influenced by a bit of philosophy about knowledge representation. Specifically, it is an approach predicated on a set of descriptive mathematical (algebraic) methods around which an elaborate set of propositions, corollaries, and theorems have been built (see Wang 2003b; 2006; 2007b). This clarification is being made here as a caution to the reader not to confuse “cognitive informatics” with neuroinformatics for neuropsychology. “Cognitive informatics” is an approach to modeling, and the relevance of modeling to NI in neuropsychology is articulated in this subsection.

2.4

Tools and Methods for the Simulation of Neurons and Neural Circuits

17

information from multiple synapses and at finer levels of neuronal detail, the more the network approximates spatial and temporal dynamics of an actual neural system. Neural networks are developed with computer programs, and a large variety of software packages are available. The software enables the user to select types of network topologies, types of neurons and various neuronal variables, different methods to train the network, and ways of testing the network. How Networks Learn: An important aspect of complex network is the bidirectional flow of information – when information from later nodes can be fed back to earlier nodes so as to adjust their properties. A network is trained by learning rules (algorithms) that serve to adjust parameters (e.g., synaptic strength) of earlier layers. Back-propagation is a commonly used training method by which the values of data output from the output layer of a network are compared to expected values. Discrepancy values are propagated backwards in the network as feedback data that the network uses to adjust its synaptic strengths. With continued training of the network in this manner the network can achieve an output that is consistent with an expectation. An adaptive network is one where connections and other network properties are automatically adjusted based on the input to the network and on how the information flows through the network during the training/learning phase. In neural modeling, neural networks are often configured as adaptive networks. They have a powerful capacity to adjust to new data and to outcomes. Core Application Functions of Neural Networks: Computer and mathematical models based on neural networks have wide ranging applications beyond theoretical models of brain function. They may be applied, for example, in everything from camera surveillance software to weather forecasting applications. In most cases, the role of the neural network model is just one or more of a few common, sometimes overlapping, functions. These have to do with algorithms that reduce or compress a large mass of data to a manageable form and algorithms that mine the data for similar temporal, spatial, or physical patterns or clusters; algorithms that match input patterns (physical, spatial, temporal, numerical, etc.) and match them to stored data so as to recognize the stimulus pattern or classify and store the pattern; algorithms that extrapolate or approximate existing data to derive an approximate value for an unknown value; and algorithms that do the same but on a temporal scale so as to predict data sequences or event occurrences. Relevance of Neural Modeling in Neuroinformatics: Neural networks are fundamentally methods of modeling data that have complex, non-linear relationships. Neural modeling is by and large mathematical and statistical and can be described as mathematical or statistical methods for the capture, classification, and matching of data and predictions based on these data. Bioinformatics and NI have the same goals relating to data but are not necessarily tied to statistical methods. Neural network methods can be applied to NI when data volume or data complexity warrants the application. They are hugely applicable in data mining – to help filter and condense the data, find relationships between variables, etc. Essentially, modeling is a tool that can directly tackle the larger NI goals of working out patterns and relationships by clarifying the complex layers between input and output data patterns.

18

2 Current Neuroinformatics Applications and Infrastructure

Neural networks are particularly relevant to NI in neuropsychology in areas of visual processing including face recognition/object/text recognition, speech recognition and speech synthesis, gestural recognition, and decision making (its relevance to NI in neuropsychology is illustrated a little further in Chapter 3). (URLs: http://neuron.duke.edu/ or http://www.neuron.yale.edu/neuron)/

2.5 Database and Knowledge Discovery Systems for Clinical and Academic Research Some of the NI examples surveyed above make evident that a database is often the nucleus of the NI environment. More than a mere repository, a database allows for selective querying and combining of data. When databases have to be designed for clinical and research environments, they face some unique design challenges brought about by the multi-tiered, multi-faceted nature of these environments. Large clinical and research settings involve many data sources and data types and call for numerous uses and applications of the data. A clinical setting typically also includes multiple users with varying levels of data access privileges. In the academic neuroscience research environment, massive volumes of data are churned out at rates faster than any single scientist can reasonably integrate. The volume of published literature is on a scale so enormous that it becomes impractical to grasp. Spread throughout the strata of clinical and research environments are valuable, often untapped data. Masses of data in databases cannot be manually examined for relationships among the data. Special procedures are required to mine the data for trends and patterns that may exist therein. The potential of the data is realized through data mining and related methods of knowledge discovery. Data Mining and Knowledge Discovery: Data mining refers to the process of applying statistical, modeling, and visualization methods to large databases in order to bring forth patterns and relationships that may lie obscured in the vastness of the data (see Piatetsky-Shapiro and Frawley, 1991; Lavraˇc 1999; Prather et al. 1997; Zupan et al. 1999; Zhang et al. 2004). It serves to extract useful information from the data by sifting out regularities or irregularities among the data. Since data mining applies to databases and serves knowledge discovery, the process has come to take on the name “Knowledge Discovery in Databases” (KDD). Data mining is a crucial step in the overall process. Typically, before data mining procedures can be applied, the data are cleaned and may be arranged in ways that facilitate specific data mining techniques. Data mining is ultimately concerned with drawing comparisons and viewing interactions among the data (Kötter 2001) and using the data to discover hidden correlations which may then be used to draw hypotheses (Wong and Huang 1996; Wong et al. 2004). Data mining has therefore come to constitute an actual method of research (Shepard et al. 1998). An entire branch of neuroscience research can be created just by mining the data that lies within the multiple strata of its subdisciplines. Ontology and Knowledge Discovery: In informatics, the term ontology refers to a model that lays out the relationships and properties of the features that constitute

2.5

Database and Knowledge Discovery Systems for Clinical and Academic Research

19

a domain of data (see Gruber 1995; Maedche 2002; Staab and Studer 2004). The model has a format that is readable by informatics tools and attempts to represent concepts and their relationships in a manner that is approximate to how they occur in reality. Ontologies attempt to conceptualize a data/knowledge domain by specifying possible data sets, data classes, data concepts, and features, rules, and relationships that possibly describe the data. By defining a common vocabulary and relational structure, ontologies aid researchers in organizing the data of a knowledge domain, which in turn makes for efficient ways of knowledge sharing. Ontologies are used to guide data mining but they are also modified and adjusted over time as more data are accumulated. Ontologies are crucial for knowledge discovery in informatics because (a) they guide the choice and structure of the data mining methods and (b) they critically influence results obtained through data searches and data mining. In other words, relationships in the data as brought forth by data mining are influenced by the ontology that guides the data mining procedure. Ontology that more accurately represents properties and relationships of a domain as they occur in the real world will generate more accurate results through KDD. This in turn can be used to refine the original ontology. Ontology development in NI is a highly specialized area. Ontology building is essential to semantic tools in knowledge discovery that enable the mapping of semantic relations among large data sets. Semantic web technologies and a suite of “onto-tools” help provide semantic frameworks for organizing the data and “engineering knowledge.” There has been a clear trend in neuroscience toward centralized databases in which the data are laid out along semantic categories (see Gorin et al.). Complex programs using semantic criteria work to archive, index, or retrieve the data. The NeuroScholar project (www.neuroscholar.org) is an example of database system for the mining of neuroscience data for research purposes (see Burns 2001a, b; Burns et al. 2003). NeuroScholar is designed to index multiple sources of neuroscience data such as published literature, websites, and laboratory-based data files. It utilizes techniques from software engineering to construct its knowledge model. Using NeuroScholar a user can, for example, extract the parts of published papers that are most relevant to a question the user may seek to understand. NeuroScholar terms these information parts “fragments,” which may be textual or graphical in form. In a workspace provided by the system, the user can then decide on the extent of linkage between the generated fragments. The user may also create fragments. Once the user establishes a linkage between a set of fragments, the relational pattern may then be stored on the system. This relational pattern may now constitute an interpretation of data fragments, providing a model for knowledge representation. NeuroScholar offers the facility to build complex neuroscientific relationships using isolated fragments of neuroscience data. A host of supporting tools also enables the NeuroScholar user to carry out other operations such as computational analysis of the knowledge representations. The Neuroinformatics Database System (Wong et al. 2002; Wong et al. 2004) is a database system for disease-oriented neuroimaging research, implemented at the University of California, San Francisco. It is an example of a system tailored to data integration within a clinical setting. In its current rendering, it contains data

20

2 Current Neuroinformatics Applications and Infrastructure

relating to patients with intractable temporal lobe epilepsy (TLE) – it serves and is served by clinicians and scientists involved with TLE treatment or research. In a large clinical setting, some of the departments that would typically be involved with intractable TLE patients are neurology, radiology, and surgery. In a typical hospital’s information system, each department inputs its data on a patient into the system. All departments may have access to the patient file and can therefore view the patient’s results from each department (presuming the hospital’s privacy policy makes the allowance) and this is usually the extent of data interconnectedness. Hospital information systems are designed to serve the essential functions of data input, storage, and retrieval. The Neuroinformatics Database System (NIDS) is an integrated system that builds on an existing hospital computer infrastructure. It is designed to draw multiple data types from multiple sources (departments) and offers specialized analytic and statistical components for data meta-analysis. NIDS can extract and replicate information from the hospital’s primary information system without interfering with patient records. “Warehousing” the data as NIDS does and then applying a range of analytic algorithms to the data enables a clinician or researcher to search for correlations and patterns that otherwise could not be gleaned from the standard database. The emerging role of NI-oriented databases and data mining systems for neuroscience cannot be underestimated. It is only through the NI-enabled integration of diverse data types that many complex relationships in neuroscience can be viewed. This idea has been succinctly expressed by Wong and Huang (1996): “[T]he motive is to realize the greatest possible benefit from the data that already exist. The new megachanges of the next 20 years will center around gathering, managing and using multimedia clinical information.” (p. 285).

2.6 Neuroinformatics Management and Infrastructure The neuroinformatics applications and projects sampled above often require a great deal of management, funding, and coordination. Proper management infrastructure is particularly important for an emerging multifaceted discipline like NI. The support structure can (a) provide an effective forum for exchange and communication of ideas, (b) help formally identify areas of research and development, (c) coordinate projects among research teams at national and international levels thereby maximizing research potential, (d) provide centralized databases and tool repositories, and (e) provide funding sources for NI projects. The scale of a support structure for NI can vary depending on the extent to which it serves one or more of the above considerations. A less obvious but equally crucial function that may be served by an organizing body for NI has to do with formative decisions regarding tool and database designs, modeling schema, and data-sharing protocols. These have the potential to influence the entire NI landscape over the long term. This consideration addresses the following questions: What data entry fields and querying procedures should be created when designing a database? What kinds of coding systems should be implemented

2.6

Neuroinformatics Management and Infrastructure

21

in developing a modeling platform for neural circuits, or how should a multicentered NI collaboration be structured? These are just sample questions that are better addressed at the onset if any part of the project is going to be open to multiple individuals or research teams. As is the case with any kind of informational management project, an early determination of the optimum ways to break down and organize the data can avoid many potential future complications. At the same time, the NI system’s design has to be flexible enough to accommodate later needs without major disruption to the overall function. Bearing on such factors are plain organizational, logistical, and systems-engineering considerations that are by no means unique to the NI environment. An organizing body can integrate diverse needs of NI researchers and implement designs and protocols to greatly facilitate sharing of the NI platform. Strong sources of potential funding are especially vital to promote and sustain NI. Funding can range from startup funds for a single researcher’s software-related initiative to major long-term support for a multi-centered collaboration. Ideal funding agencies for NI are those that fully appreciate the interdisciplinary nature of NI and the potential that NI brings to neuroscience. As described below, there are now in place organizations that aim specifically to advance the field of NI by funding NI projects. Some of these initiatives are so important to NI that they are simply infused with the field.

2.6.1 Examples of NI Organizations, Infrastructure, and Management The Human Brain Project and the Neuroinformatics Program of the National Institutes of Health: As alluded to earlier, the Human Brain Project (HBP) has been a major NI initiative aimed broadly at all disciplines involved with brain-behavioral sciences. Launched in April 1993, the explicit purpose of the Human Brain Project (HBP) was to foster the development of NI. This initiative had been sponsored by four US federal agencies, the National Institutes of Health (NIH), the National Aeronautics and Space Administration, the National Science Foundation, and the U.S. Department of Energy. At least 12 organizations within the NIH were involved with the HBP. The HBP was the product of a coordinated effort among these federal agencies to support the field of NI. The HBP was created in response to the emergence of large volumes of neuroscience data. It recognized the need for integration of data from multiple avenues of neuroscience in order to achieve cohesive and broader understanding of the brain. It also recognized the potential that computers and information systems offered in dealing with such data (Huerta et al. 1993). The formation of the HBP is often traced back to a few key historical events (see HBP website; Huerta et al. 1993; Huerta and Koslow 1996; Shepard et al. 1998). In the early 1980 s, neuroscientists had begun formal discussions on the potential of general computer technology in furthering neuroscience research. The actual technology available at the time was, however, not on par with the ideas about their potential facilitating role. By the late 1980 s, computer technology had made few

22

2 Current Neuroinformatics Applications and Infrastructure

leaps and the promise of applied information technology was already being seen in molecular biology projects such as the Human Genome Project and the C. elegans research database (a database for mapping the genome of the roundworm, Caenorhabditis elegans). Encouraged by these promising developments, the NIH and NSF, in 1989, asked the National Academy of Sciences’ Institute of Medicine (IOM) to create a working group of scientists that would explore the feasibility of developing a national database resource for neural circuitry. The group engaged in extensive discussion and consultations, and in 1991, published a report in which the idea of integrating computational technology with neuroscience was firmly endorsed (see Pechura and Martin 1991). Evolving from this initiative, the HBP was launched in 1993, formalizing a structure for neuroinformatics. Incorporating many of the recommendations made by the IOM’s working group in 1991, the HBP’s coordinating committee shaped a guiding strategy for the project. The HBP would aim to support research in all areas of neuroscience at both federal and public institutions. It would call for the research to show a clear involvement of both neuroscientists and technology specialists (computer/informational, mathematical, engineering, etc.). Very importantly, the HBP would require a certain degree of communication between the projects it would sponsor and it would require that NI tools and technologies developed under its sponsorship be compatible across research teams, and hence be easily shareable. The idea of the neuroscience community richly interlinked by a system of shared computer networks databases was articulated in the IOM’s working group. It is one of the main ideas that have shaped the HBP’s outlook. The HBP has been conducted along three phases (see Huerta and Koslow 1996; Huerta et al.; Gorin et al. 2001). In phase 1, it sought development of prototype NI tools for capturing, analyzing, and manipulating information digitally. Much of the emphasis in this phase was on tools and databases relating to (a) functional neuroimaging and (b) neuronal structure and circuitry (see Shepard et al.). In phase 2, it encouraged the refinement and integration of the tools and systems developed in phase 1. Attention was given to computerized methods of integrating graphical data (of the type generated with phase 1 tools) and non-graphical (e.g., theoretically described) neuroscience data. Phase 3 took the developments from phases 1 and 2 to shared platforms (e.g., Internet-accessible databases), making them available to the broader neuroscience community. High priority has been given to projects relating to data storage, data access, and data mining; digital brain atlases and related tools; simulation and visualization software; and Internet-based platforms for tool and data sharing and for collaboration between research teams and sharing. After more than a decade of successful initiatives, the HBP has turned its focus to how these initiatives can be more broadly applied in advancing research (HBP website). It is not, at the present time, soliciting new funding applications but many other divisions of the NIH continue to call for informatics-related projects. The HBP has greatly propelled and shaped the field of NI. It started with a visionary idea, which was translated into an effective plan of action. In addition to providing management and funding, the HBP has placed emphasis on the

2.6

Neuroinformatics Management and Infrastructure

23

cross fertilization between the neurosciences and information sciences; it has placed a premium on data sharing and the tools to bring about this sharing. The results have been many powerful NI tools and databases and many national and international collaborations that have assailed traditional research obstacles. (URL: http://neuroscienceblueprint.nih.gov/neuroscience_resources/neuroinformatics.htm) The Neuroinformatics Working Group of the Organization for Economic Cooperation and Development: The Paris-based Organization for Economic Cooperation and Development (OECD) is a forum that brings together the governments of some 30 member countries to tackle a range of social, economic, and trade issues. Economic development especially via advances in science and technology is a significant area of emphasis of the OECD. The OECD Global Science Forum (GSF) seeks to promote international cooperation in basic science research. As part of its science forum, the OECD established a Biological Informatics Working Group in 1996. In 2000, this group established the Neuroinformatics Working Group (NWG). The impetus for this group was, again, the recognition of the utility of NI in dealing with complex problems, great data volume, and multiple research teams in neuroscience and the potential it holds in leading to solutions for nervous system disorders. The NWG was given a 2-year mandate to develop guidelines for NI at the international level, to create resources for NI, and to propose measures for sustaining NI globally. It was chaired by Stephen Koslow (formerly of the NIH) and co-chaired by Shun-ichi Amari (RIKEN Brain Science Institute, Japan) and Sten Grillner (Karolinska Institute, Sweden). The activities, accomplishments, and proposals of the NWG have been detailed in two major reports compiled by the members of the NWG: Report on Neuroinformatics (OECD-GSF, 2002) and Neuroinformatics: The Integration of Shared Databases and Tools Towards Integrative Neurosceince (Amari et al. 2002). A few points from these reports are summarized here. The NWG stressed the need for international collaboration in data acquisition, research tools, and theoretical models in neuroscience. It also placed emphasis on unifying neuroscience research databases. Standardization in data dissection combined with a global network of databases has the potential to overcome common obstacles to data integration, such as socio-political and cultural boundaries, and intellectual property issues. To this end, it drafted a set of guidelines on database standardization. To sustain multileveled NI initiatives, a two-pronged strategy was suggested: local NI initiatives should be supported and ideally networked to create a larger global NI network (a bottom-up NI development framework). At the same time, international NI coordination should help synchronize local initiatives and facilitate interoperability between these local NI systems (a top-down NI development framework). To promote both these trajectories, the NWG proposed (a) the creation of an International Neuroinformatics Coordinating Council (INCC), and (b) the establishment of an international NI funding mechanism, a Programme in International Neuroinformatics (PIN). The INCC would serve the functions of international NI coordination. It would allocate tasks to the various national nodes and would optimize the use of international resources. The above recommendations made by the NWG to the OECD in 2002 were endorsed by OECD science ministers in 2004. In 2006, the OECD’s GSF carried

24

2 Current Neuroinformatics Applications and Infrastructure

forth the recommendations by launching the International Neuroinformatics Coordinating Facility (INCF). This body evolved after 16 OECD member countries as well as the European Commission elaborated the initial proposals of the NWG (www.incf.org). The INCF currently has 10 member countries and the memberships of several other countries are in process. Its secretariat is based at the Karolinska Institute. The funding mechanism, PIN, is well on the way of being a reality. The INCF has already created a framework for this mechanism (see www.incf.org). PIN will seek to fund NI projects for which national sources of funding may be lacking. The NWG also initiated two NI portals for international data sharing and resource exchange. The Neuroinformatics Site (www.neuroinf.org) largely serves the function of communicating information about NI meetings and workshops around the world. The German Neuroinformatics Node (www.neuroinf.de) provides an online access point through which users can share NI software and databases. It is currently in its prototype form and is funded through a grant from the German Ministry of Science Education. Both these portals offer links to numerous other NI resources. (URL: http://www.neuroinformatics.nl/OECD-GSF-NIWG.html)

Conclusion The rapid success of NI over the past years is closely linked to organizations like those discussed above that have made NI imperative. An open but centralized system for NI coordination, protocols, and standards for NI data and key funding have been essential ingredients to the rise of NI. Stable, long-term funding, especially when integrated with national science/research agendas are critical to large-scale NI projects. It is still, however, necessary that independent or local NI initiatives continually sprout new ideas. Whether in the form of exploratory committees, interest groups, or a small database on an academic department’s computer server it takes these seeds to grow into larger NI projects which may then become the nodes of a wider NI network. Small NI initiatives and the larger management structures that may assist in coordinating these initiatives often co-evolve. The various areas of NI sampled in this section highlight the definitive role of databases, computational tools, analytic and modeling software, and frameworks for data access in NI. Great emphasis is placed on the gathering, organization, integration, and sharing of data. In some instances, approaches to a problem in neuroscience are built around informatics methods that are actively brought to bear on a research problem. NI influences research questions as much as they influence it. NI shares great credit in shaping the current era of neuroscience. Almost every avenue in neuroscience has now incorporated NI.

Chapter 3

Neuroinformatics for Neuropsychology

There is clearly an enormous difference between NI applications and the general use of computers in the neurosciences. NI is much more than a simple extension of general computing. NI applications by definition constitute digital systems for active information-seeking, problem-solving, and modeling, generally based on strong theoretical frameworks. Neuropsychology has always been quick to embrace general computing. Even prior to the first generation of commercially available personal computers of the early 1980 s, simple neuropsychological tests began to appear in computerized form (see Adams and Heaton 1987; Chute 2002). Over the past two decades, arrays of computerized neuropsychological tests have emerged and these tests have shown increasing levels of complexity. However, even with the advances carried by these applications, they still remain far short of incorporating NI functionality. Neuropsychology features minimally in the landscape of NI. Similarly, NI has only a small presence in neuropsychology. Yet this small presence is marked by some powerful NI-oriented neuropsychological applications/models that are relatively new and which have been developed in highly specialized clinical or research environments. The two key factors marking these examples are that they are concerned with neuropsychological topics and they are designed from an informatics perspective. This section begins by spelling out the differences between NI applications for neuropsychology and the general use of computers in the field. Exactly what it means to have neuropsychology-specific NI systems is then articulated – an attempt is made to define core principles of NI approaches in neuropsychology. Following this, nine NI-based applications or models for neuropsychology are discussed.

3.1 Differentiating Between the General Computer Applications in Neuropsychology and Neuroinformatics Systems for Neuropsychology Consider the everyday clinical or research focus of neuropsychologists. Then consider the availability of tools and applications that confer NI capabilities to those tasks specific to neuropsychologists. Are there tools or databases for V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9_3,

25

26

3 Neuroinformatics for Neuropsychology

neuropsychologists analogous to the tools and databases such as CARAT and CoCoMac that allow neurobiologists to create maps of cortical regions and circuits? Do neuropsychologists have information sharing systems along the lines of the ICBM Subject Database that gives users the facility to upload imaging data to shared platforms and then reap the benefits of the collective data? Why have neuropsychologists not developed a simulation environment for their work while other neuroscientists have developed simulation environments like GENESIS and Neuron for modeling neural systems? The interface between NI and a definitive neuropsychological praxis is largely an uncharted area. This, of course, does not apply to some subspecialties allied to neuropsychology such as neuroimaging and clinical electrophysiology. Neuroimaging (see Sections 2.1, 2.2) has certainly become the preferred research mode of many experimental neuropsychologists but the contexts in which it is used and the medical/research questions it addresses are far from unique to neuropsychology. Clinical neurophysiology, especially to do with electroencephalography and evoked potentials, is now conducted with fully computerized, informatics-heavy tools. Like functional imaging, computerized electrophysiology is a specialized discipline that happens to serve neuropsychology but is also applied to studying a range of functions far outside the scope of neuropsychology. The NI seen within neuroimaging and electrophysiology do not constitute NI applications that can be considered unique to neuropsychology. In neuropsychology, the areas of (a) head injury research and (b) computerized neuropsychological assessment show levels of sophistication in computerized methods that are almost on the threshold of transitioning to NI methods. Informaticsreliant functions are steadily filtering into computer applications in these areas though they remain, for the large part, short of constituting NI functionality (automated assessment, discussed in Section 3.3, is a clear exception). Nonetheless, they signal the value of databases and computational methods for neuropsychology. The purpose of this subsection is to clarify aspects of computerized assessment that border on NI and aspects that do not constitute NI. A review of computerized neuropsychological assessment is not a goal of this subsection. Many such excellent reviews have been conducted and are cited. Head Injury Research: It has been suggested that a national traumatic brain injury (TBI) database will help with uniformity in TBI assessment, better communication, and data sharing among clinicians and improved prediction of clinical outcomes (Hall 1997; Johnston 1997). These reports have centered more on the general utility of such TBI databases. Hobbs (1999) mentioned “neuroinformatics” and the Human Brain Project and then described some implications for TBI rehabilitation, for example, NI-based brain atlases and 3-D reference systems for brain lesions can be used to model TBI and help to better match treatment with type of injury. In 2002, the Journal of Head Trauma Rehabilitation published a special edition on neuropsychological technologies. While none of the articles concerned NI, a few touched on issues that are also encountered in the broader discourse on NI. Schatz and Browndyke (2002) described scenarios where neuropsychological assessment can be beneficially carried out on portable computers or workstations and the data then

3.1

Differentiating Between the General Computer Applications

27

uploaded to databases on remote server. Schultheis et al. (2002) described various applications and benefits of virtual reality technology to neuropsychological assessment. Chute (2002), in a broad survey of neuropsychological technologies, also acknowledged “neuroinformatics” in describing future scenarios where informatics systems would be integrated with neuropsychology. General Computerized Neuropsychological Assessment: Computerized tests for neuropsychological assessment and for psychometry have been around for more than three decades. Major neuropsychological assessment batteries as well as standalone tests either offer some parts in computerized form or are available as complete computerized packages (see reviews by Kane and Kay 1992; 1997; Kane and Reeves 1997; Luciana 2003; Wilson and McMillan 1992). In the latter case, administration, scoring, and statistical analysis are all carried out using software packages. Computerized assessment applications add some distinct advantages over paper – and versions of the tests. They can bring greater uniformity to test administration, data handling and scoring, and they may be easier and less time consuming to administer (Gur et al. 2001). Consider a few examples of widely used neuropsychological tests and batteries: R R , the Wechsler Memory Test III , and The Wechsler Adult Intelligent Scale III R the Wechsler Individual Achievement Test II can be accompanied by a common software package that aids in scoring and interpreting results. A computer scoring option is also available for the Luria-Nebraska Neuropsychological Battery. The R is available in a form that can be administered onWisconsin Card Sorting Test screen. The software can score the patient s responses, generate reports, and enable comparison of the patient s performance across trials or sessions. The patient s basic R II demographic data can also be entered. The California Verbal Learning Test comes with a computerized scoring assistant that gives the examiner a range of scoring options. Scoring complexity can be increased mainly by selecting more scoring variables tied to measures such as learning characteristics, learning interference, delayed recognition, etc., or by choosing to see more statistical and graphical data. In stand-alone neuropsychological tests, for example, the Rey Complex Figure Test, computerized scoring has also been implemented. Using a sensor placed above the drawing space, a patient s drawing movements (tip of the pen) can be tracked and the drawing is automatically converted into a digital image, which can then be analyzed with a particular quantified approach. In these examples, a clear trend can be seen towards (a) a greater amount of data capture or conversion of data to digital form, (b) more accurate computer-driven scoring systems and (c) some facility for data comparison though almost entirely intra-subject. All of these functions depend on data files, which in the case of these computerized assessment applications, are integrated with the computer program as opposed to being shareable or accessible for manipulation. On the one hand, these applications are stylistically, informatics-geared. On the other hand, they remain essentially simple computer applications for scoring tests or applying a variety of statistical treatments. The application of computers here is limited to the goals of efficient, standardized test administration and achieving greater statistical analyses of data. The tests can be described as “computerized” because they are administered

28

3 Neuroinformatics for Neuropsychology

and scored with the aid of computers. The integration of statistical tools with the assessment programs also accounts for a large part of the computerized functionality conferred by these applications. Successive computerized versions of these tests are more likely to add small increments of change rather than functionality that confers a potential for research breakthroughs. A hard NI functionality remains absent. In general, neuropsychology has been using a very simple rubric when structuring computer applications. In these scenarios, the computer platform has not been used to dynamically modify the tests. In contrast, NI offers methods and tools that can be dynamically tuned to the very complexity of neurocognition. The sophistication of its methods is commensurate with the complexity of cognitive-behavioral patterns that neuropsychological assessment seeks to describe.

3.2 Defining Neuropsychology-Specific Neuroinformatics Systems What then does it mean to have neuropsychology-specific NI systems? How might NI for neuropsychology be described? What might be achieved by NI-driven neuropsychology that cannot be achieved by conventional methods? The essentials of NI obviously remain the same no matter the avenue of neuroscience in which it finds itself being applied. It becomes tailored to a neuroscience subdiscipline when its methods are brought to bear on research problems unique to the subdiscipline. Obviously the NI application would need to address clinical or research topics that are typically neuropsychological. This can only be offered as a general guideline since there is no distinct border separating neuropsychology from other subdisciplines in behavioral neuroscience. It should be expected that neuropsychological NI applications would address such things as (a) cognitive profiles and neural models relating to the functional domains of language, memory, spatial processing, etc.; (b) syndromes such as aphasia and neglect that are uniquely neuropsychological in that they have distinct neural and cognitive components; and (c) core assessment methods such as the test batteries used in clinical neuropsychology. Some particular guidelines can be proposed for creating NI applications that are specific to neuropsychology: • The application must render observations and findings that cannot be feasibly achieved with conventional neuropsychological methods – there has to be a very clear advance brought by the NI tool over and above the data rendered by conventional tools. • Where applicable, the structure of the NI system should be commensurate with the complexity of neurocognitive mechanisms as described by current theory in neuroscience, that is, the complexity of an NI system should, as far as possible, be tuned to the complexity of the neuropsychological phenomenon at which it is aimed.

3.3

Neuroinformatics Applications and Models for Neuropsychology

29

• Using theoretical and NI frameworks, attempts should be made where possible to reformulate neuropsychological research issues so as to generate more detailed or quantified renderings of models, problems and phenomena. This should be done with the perspective of examining research problems with NI tools. With an understanding that NI can enable certain data-matching and pattern-seeking functions, neuropsychologists can seek, for example, a finer grain of linkage between behavioral and neuroanatomic data – relationships that could be overlooked without the enabling tool. In devising NI systems for neuropsychology, consideration should be given to ways by which the data generated by the system can be potentially interfaced, using NI methods, with data from other research domains (e.g., phenomics). This will influence both the fineness of the neuropsychological constructs tapped by the NI system as well as the format of the data sets. Consideration should also be given to the possibility of the data being part of a larger knowledge-building network, in which case, data formatting considerations are again critical (this is discussed in later sections). The challenge for neuropsychology is to take one or more of the key data seeking and data handling methods of NI (see Table 3.1) and apply them to neuropsychological data, research problems and phenomena. To reiterate, these methods have to do with (a) Seeking a finer “grain” or increment of data, seeking to capture the data in digital form or converting the data to digital form, and optimizing it for database operations. (b) Aiming to database the data with a breadth of data fields (the fine increments of data), structuring the databases so that they are optimized for future data manipulation, and where feasible, using web-accessible and shared-platform databases. (c) Applying data mining/pattern-seeking algorithms and data modeling methods to elicit or test relationships between data sets. Clinical or research initiatives in neuropsychology can apply these NI methods in so many ways. These are best illustrated by examining examples of NI applications in neuropsychology.

3.3 Neuroinformatics Applications and Models for Neuropsychology Nine examples of NI applications or models for neuropsychology are described below. The examples relate to both clinical and experimental neuropsychology. The examples are grouped into four broad areas: (a) general neuropsychological assessment; (b) spatial processing, spatial neglect, and visual search/visual attention; (c) speech/language production, aphasia, and sign language, and (d) phenomics

30

3 Neuroinformatics for Neuropsychology Table 3.1 Summary of neuroinformatics applications and models for neuropsychology

Areas of neuropsychology to which the examples relate Neuropsychological assessment and test/battery development; clinical assessment and clinical research especially applicable to large sets of assessment data

Name of example

Functions, innovations, and features

1. Automated Neuropsychological Assessment Metrics (ANAM)

• Digitization of numerous tests of specific cognitive operations • Generation of subtest-specific data files • Databasing of test results and other subject profile data • Queries to analyze data • Internet-enabled assessment and databases

2. Internet-Enabled Neuropsychological Assessment of Army Aviators (INAAA)

Research on visuospatial processing, and assessment of spatial neglect and visual search/visual attention

• Centralized database for recording test data (especially suited to remote assessment) • Feasibility test of NI assessment system connecting remote sites to a central server

3. Computational Visual • Coordinate-based mapping and Field for Neglect digitization of the visual (spatial) field (CVFN) • Recording/databasing of field coordinates and coordinates of stimulus position • Computation of neglect gradient in precise coordinate terms • Dynamic animated effects to experimentally manipulate neglect patterns 4. Computer-Assisted Cancellation Test System (CACTS)

Research speech, 5. Directions Into language, and aphasia Velocities of Articulators (DIVA)

6. The Aphasia Database

• Tool for graphical analysis of visual search patterns and visual attention in cancellation tasks • Database recording of results, and computations of search patterns • Pattern identification algorithms for visuospatial analysis • Theoretical and computational model of speech production and acquisition • Modeling of neural and network dynamics in speech and language • Model that facilitates NI approaches to speech and language research in neuropsychology • Database of diagnostic and symptomological data in aphasia • Research tool to help clarify different aphasic syndromes that overlap in symptoms • A model for databases and data mining in aphasia research

3.3

Neuroinformatics Applications and Models for Neuropsychology

31

Table 3.1 (continued) Areas of neuropsychology to which the examples relate

The Interface of neuropsychology and other biomedical disciplines

Name of example

Functions, innovations, and features

7. SignStream

• Multimedia tool for annotation and analysis of visual and gestural language • Digital video database of signs and gestures • Model for capturing/analyzing gestural communication

8. The FaceValue Database Project

• Database of (a) quantitative measurements of craniofacial dysmorphology in various disorders and (b) neuropsychological features • Exploration of etiological subtypes of autism and schizophrenia

9. The Consortium for Neuropsychiatric Phenomics

• Transdisciplinary, multileveled investigative model • Standardized methods for describing cognitive and neuropsychiatric phenotypes • Novel NI tools for knowledge building in neuropsychology • Comprehensive NI platform for collaboration that signals future trends in neuropsychology

and neuropsychology. For each of these areas, considerable background is first laid out. Certain conceptual problems and methodological shortcomings in general neuropsychological assessment, the visuospatial domain, and in the neuropsychology of language, are described so as to highlight the rationale for NI. The examples pertaining to cognitive phenomics are contextualized by an overview of phenomics research that is pertinent to neuropsychology. Table 3.1 summarizes all of the examples. Many of the examples describe actual informatics applications and databases, and a few describe theoretical models or integrative research models that are oriented to NI approaches in neuropsychology. Earlier sections of this book illustrated that the broader landscape of NI is characterized by a variety of NI applications and systems. NI in neuropsychology is barely in its infancy and shows only a limited number of developments but these developments are very significant and already show some variety. Information on the examples described was gathered using a number of sources: published literature in neuropsychology and bioinformatics/software engineering, various websites, and very importantly, from material obtained directly from the few clinicians and researchers who happen to be working at the intersection of

32

3 Neuroinformatics for Neuropsychology

neuropsychology and NI. Since this field is so new, personal communication with these individuals proved to be a very fruitful way of gathering pertinent data.1 A lot of unpublished data, project proposals, and data on projects-in-progress, etc., were provided directed to the author in the course of 2006–2008 (see acknowledgements). This material also gave broader perspective to the various researchers’ published data.

3.3.1 General Neuropsychological Assessment “Further advances in neuropsychological assessment will come about only to the degree that they are linked to evolving concepts of brain-behavior relationships. Neuropsychologists now require a deeper understanding of basic neuroscience and cognitive psychology than was true in the past.” (Benton 1992, p. 407).

In many regards, conventional methods of neuropsychological assessment have a proven record. They have been very effective, for example, in outlining the existence of brain damage; inferring the likely site of damage (generally in gross neuroanatomic terms); identifying cognitive profiles, ranging from subtle to dramatic, that may signify types of head injury, degenerative diseases, and infectious diseases; and pointing to spared cognitive functions with the potential to aid in rehabilitation (Lezak 1995; Benton 1994; Goldstein et al. 1998). Neuropsychological assessment is viewed as having a unique role in formally describing cognitive behavior. It is also often described as a means to draw relationships between behavior and the underlying neuroanatomic system or region, by virtue of its sensitivity to dysfunction or cognitive processes (Goodglass and Kaplan 1979; Benton 1994; Goldstein 1998). Yet many tests and some commonly used batteries used in assessment are derivatives of tests of general intelligence that originated in the first half of the last century (see Lezak et al. 2004) and bear no direct tie to neuroanatomic systems. In adapting tests of general intelligence for the neuropsychological context and in formulating new assessment tools, there has been a tendency to design the tools based primarily on observed deficits. This stands out as a significant problem with conventional neuropsychological assessment tools – their construction has been based not on sophisticated, theoretical principles of cognition and brain function but instead on clinically observed cognitive features (Kolb and Whishaw 2003). To some extent, this is understandable because the understanding of functional neuroanatomy in the past was nowhere close to what it is in the present time. On the other hand though, clinical neuropsychological assessment seems to have become so comfortably grounded in cognitive features expressed by assessment tools that it has forgotten about the need to constantly sharpen and update the neuroanatomic reference frames for these features.

1 The

author’s descriptions of all examples in this section have been reviewed by the primary researcher/s associated with the examples. Appreciation is expressed in the Acknowledgements

3.3

Neuroinformatics Applications and Models for Neuropsychology

33

Another problem with many neuropsychological tests is that multiple cognitive and neural operations may be called into play by a test. The multifactorial nature of many tests impacts the validity of the tests – it can blur inference of the causal factor behind the performance or the score (Walsh 1987). The problem could perhaps give added reason for Kaplan’s (1988) emphasis on attention to the patient’s process and types of errors in neuropsychological assessment. Tests that involve multiple cognitive processes may have utility in tapping large or subtle sets of features, but they are not ideal to the task of bringing a specific cognitive process to bear on a precisely defined neuroanatomic system. In addition to factors of construct validity, factors such as reliability and practicality always concern assessment tools and there is ample room, if not a need, for improvement on these fronts. In the book Neuropsychological Assessment (Lezak et al. 2004), in a chapter subsection titled “What can we Expect of Neuropsychological Assessment at the Beginning of this New Century,” the authors stated While real progress has been made over the last few decades in understanding cognitive and other neuropsychological processes and how to assess them, further knowledge is needed for tests and testing procedures to be sufficiently organized and standardized that assessments may be reliably reproducible, practically valid, and readily comprehensible. . .. One means for achieving such a goal while retaining the flexibility appropriate for the great variety of persons and problems dealt with in neuropsychological assessment could be a series of relatively short fixed batteries designed for use with particular disorders and specific deficit clusters (e.g., visuomotor dysfunction, short-term memory disorders). Neuropsychologists in the future would then have at their disposal a set of test modules and perhaps structured interviews (each containing several tests) that can be upgraded as knowledge increases and than be applied in various combinations to answer particular questions and meet specific patients’ needs. ( Chapter 1, pp. 13–14)

The statement succinctly describes key aspects of assessment in need of improvement. It essentially calls for some redefinition of assessment tools. It is remarkable that at the beginning of the 21st century, following many decades of research and development of assessment tools, such shortcomings in assessment are prevalent and described here by an authoritative veteran of assessment, Muriel Lezak (and her colleagues). The point that even the general notion of NI has not been on the radar of clinical neuropsychology, has been conveyed earlier in this section. The point is only reinforced here when one considers that the needs and shortcomings of assessment as articulated in the above quoted extract are needs that can be comprehensively addressed with NI. Neither the major compendia of assessment tests (e.g., Spreen and Strauss 1998) nor the major reference texts on assessment (e.g., Lezak et al. 2004) reflect an awareness of NI and what it means for neuropsychological assessment. The Role of NI in General Neuropsychological Assessment: The benefits of NI when applied to assessment have to do precisely with bringing organization, standardization, and modularity to the tools. They have to do with bringing technologies that enable rapid changes to tools so that they can tie in closely with discoveries of cognition and functional neuroanatomy as these discoveries unfold. All of

34

3 Neuroinformatics for Neuropsychology

these benefits in turn translate to flexibility and practicality in the usage of tools, improved organization of assessment protocols, and greater reproducibility of the results generated. NI also introduces into clinical assessment one particularly radical idea, an idea that has been absent in the field of neuropsychology, if not rather alien to the field, and almost never raised as a possibility: that is, the potential for large-scale sharing of everyday assessment data. Conventional modes of assessment are such that the results of tests are stored in office filing cabinets or on the hard drive of a clinician’s computer. Valuable data go to waste and this has been the case for decades, as forgotten boxes of paper would go through the process of degeneration in moldy basements. One of the greatest impediments that the field of assessment suffers (somewhat unknowingly to the field) is the lack of large, national or international databases of assessment data. Server–client computer platforms and online databases make this sharing possible and the potential benefits are unparalleled. Certainly, the mere allusion to the possibility of large-scale sharing of assessment data is bound to raise instinctual doubt and disbelief among neuropsychologists. Sharing of patient data is often a violation of patient confidentiality – a code that is set deep in clinical practice. The issue of data sharing is, of course, fundamental to any discussion on shared NI systems for assessment. Not surprisingly, this is exactly the kind of issue that has been at the forefront in the broader discussion of medical informatics, especially in relation to genetic and imaging data, and many possible solutions to the problem have been proposed. In view of the centrality of this issue, it is given special attention within a different section of this book (Section 4.1) and the current subsection will remain focused on NI systems for assessment. In a similar vein, issues such as standardization in assessment are not washed over by an enthusiastic argument for NI-based assessment. Discussion is best given to such issues in a separate section where they can be laid out systematically. There have been some impressive NI developments in neuropsychological assessment. They relate to such things as (a) computerized tests that tap discrete cognitive operations; (b) the digitization of neuropsychological batteries so as to produce sets of data files that can be automatically uploaded to databases; (c) the structuring of databases for many types of neuropsychological data; and (d) the use of a, b, and c in conjunction with the internet and highly portable computer devices for large-scale assessment, data collection and analysis. The dimensions that NI brings to assessment simply cannot be achieved with conventional assessment methods. Two informatics-based neuropsychological assessment initiatives are described in this subsection. One describes an automated battery of tests interfaced with a database and with portable assessment devices. The other describes a project that was designed to test a possible structure for internet-based, database-linked assessment. It will be seen through the examples that great emphasis is placed on test modularity and on data capture. Both projects have served to establish the feasibility of NI-based assessment. They demonstrate numerous crucial components that make up a large-scale NI system for assessment.

3.3

Neuroinformatics Applications and Models for Neuropsychology

35

3.3.1.1 Automated Neuropsychological Assessment Metrics (ANAM) (see Reeves et al. 2007a; Friedl et al. 2007; Bleiberg et al. 2004; Dennis Reeves, personal communication (July 21; August 8, 10; September 26, 27, 2008; March 7, April 23, 2009); Joseph Bleiberg, personal communication (July 21, September, 23; 2007; February 10, March 7; 2008; April 14, 2009) URLs: http://www.armymedicine.army.mil/prr/anam.html http://www.nrhrehab.org/Research/Projects/ATRC+2000/ ANAM/default.aspx Description and Background: Automated Neuropsychological Assessment R (ANAM) is a library of numerous computerized tests, structured as a comMetrics prehensive set of test modules. It has been developed over the past 30 years, largely as an initiative of the US military and sponsored by the US Department of Defence. The military saw a great need for standardization in the various neuropsychological tests and assessment software that it was using. Earlier neuropsychological batteries developed within the military such as the Walter Reed Performance Assessment Battery (WRPAB) would provide a foundation for the further development of automated neuropsychological tests. Visionary leadership initiatives to consolidate an optimal neuropsychological assessment battery for the military then provided critical thrust that drove the evolution of ANAM (see Friedl et al. and Reeves et al. for a detailed historical review of ANAM). R constitutes a test system in that tests or modules can be selectively ANAM grouped as test batteries to assess a target behavior or aspect of cognition. In its current form, ANAM includes 31 test modules that run on Microsoft WindowsTM platforms. The tests assess specific aspects of cognitive-behavioral-affective domains such as attention, memory, spatial processing, psychomotor coordination, mood, etc., as well as many other aspects of performance, such as concentration and fatigue, cognitive set switching, and efficiency of mental processing. Examples of ANAM test modules are: Two-Choice Reaction Time; Digit Set Comparison; Code Substitution Learning; Matching Grids and Matrix Rotation; Stroop Color Word Test; Tapping Test; Pursuit Tracking; Automated Aphasia and Memory Scale; Malingering Task; and Mood Affect Scale. Some of the tests are adaptations of earlier developed tests, for example, the Memory Search test is an adaptation of Sternberg’s (1966) memory search/reaction time task, and the Tower Puzzle is a considerable variation of the classic Tower of Hanoi task. Parameters within each test, for example, rate of stimulus presentation, can be R is therefore a flexible, modular system of targeted tests. ANAM modified. ANAM has been shaped this way over the course of its evolution because this configuration has been recognized as being ideally suited to cognitive and performance evaluation in the military context: there needs to be a practical method for baseline neuropsychological assessment of soldiers; repeated testing of subjects are often required and this has to be carried out efficiently and uniformly with standardized tools. A test battery has to be suited to a range of applications in performance training, treatmentrelated clinical assessment, cognitive evaluations when neurotoxicological exposure

36

3 Neuroinformatics for Neuropsychology

is suspected; cognitive status monitoring when soldiers are in prolonged, stressful conditions, often in remote locations, etc. In the clinical realm, the battery must also be suitable for use with various clinical populations. These considerations can be summed by an essentially dual nature required for neuropsychological tests for military use (Friedl et al.): (a) The tests must be tuned to assessing changes in cognitive performance in healthy individuals as induced by various stressful situations. This is in contrast to conventional tests, which are structured around losses in functional domains and specific neurocognitive syndromes. (b) In a clinical context, the tests must also, in the classical sense, be sensitive to cortical dysfunction following brain injury or other forms of neurological damage and must also provide clear baseline data. The modular structure of ANAM enables easy configuration of batteries for different clinical or research scenarios or the use of preconfigured batteries. Incorporation of Informatics in ANAM: The development of ANAM has always been closely tied to developments in computerized test systems. In addition, the context of the military in which ANAM has evolved has been one of innovative technological forces aimed at harnessing the utility of computerized-automated assessment. As a result, ANAM has come to incorporate various computerized functions and design features that comprise core constituents of informatics-structured assessment tools (see Bleiberg et al. 2004, Reeves et al. 2007a; Cernich et al. 2007). Multiple Test Parameters and Parameter Modification: As indicated, the clinician has the option of configuring ANAM test modules in various ways. Parameters such as the number, size, color of stimuli, and inter- and intra-stimulus intervals can be changed through simple software commands (command line switches). Besides enabling customized batteries, this function gives the clinician the power to finely probe a behavioral response by making incremental changes to one or more stimulus variables. Such functions are only feasible with computerized systems (and are further illustrated by the example discussed in Section 3.3.2.1). ANAM scores also include a measure of the rate of correct responses per minute. This measure of cognitive efficiency, called “throughput”, was first introduced by the WRPAB (see Thorne et al. 1985; Thorne 2006) and has been described by Reeves et al. (2007a) as being one of the most sensitive measures of changes in performance. Uniformity of Data Output and Separated Data Files: The format in which scores are generated across all the ANAM tests is uniform, i.e., all tests produce a similar series of scores. Each subtest also produces a data file with a unique file extension, e.g., “.cds” for the Code Substitution Test and “.pro” for the Procedural Memory Test. Included in a test data file are the numbers of correct and incorrect responses, the result for each trial, and statistical and response time measures such as mean and median response times of various sets of responses. Uniform sets of raw scores promote easy import of the data into a database. The separation of data files of each subtest is necessary for various database/query driven inter-test comparisons. The coding of the raw data in a database compatible format and a diversification of data types, all contribute to a more complex set of data fields in a database. A subject’s demographic data are contained in a unique data file.

3.3

Neuroinformatics Applications and Models for Neuropsychology

37

Data Capture and the ANAM Access Database (AADB): The AADB is a Microsoft Access database designed to organize raw data files from ANAM tests, create summaries of the data, and examine or combine the data in relation to one or more variables of interest (see Bleiberg et al. 2004). ANAM test data files containing raw data and summary data can be automatically imported into the AADB. Data import is made very simple, involving a few steps. Data from ANAM subtests are fed into pre-existing tables in the database. Data from a particular test are automatically fed into a table for that test (tied to the test’s unique file extension). The database contains three main types of tables: Header tables include data such as the subject ID, date and time of testing, and variables selected for a test and the testing session (Fig. 3.1a). An entry (row) in the header table corresponds to a single run of a test. If the test is administered twice in a session, another row of data will be produced in the header table.

Fig. 3.1a Example of a header table for the Code Delayed subtest (showing one row without data). From the ANAM Access Database User’s Manual (2004); printed with Permission, National Rehabilitation Hospital.

Item tables contain data about the trials/stimuli of each subtest and the subject’s responses – correct/incorrect, reaction time, and accuracy (Fig. 3.1b). In this table, a row of data corresponds to a single trial or item of a subtest. A 15-item subtest will therefore produce 15 rows of data in its corresponding table in the AADB.

Fig. 3.1b Example of an Item table for the Matching Grids subtest (showing one row without data). From the ANAM Access Database User’s Manual (2004); printed with Permission, National Rehabilitation Hospital.

Summary tables essentially summarize ANAM test data files to present information such as mean and median scores on a test, number of items correct, number of lapses, and throughput scores (Fig. 3.1c). A row in a summary table corresponds to single run of a subtest. If a subtest was administered twice in a session, two rows of summary data will be produced in the summary table.

Fig. 3.1c Example of a summary table for the Running Memory (continuous performance) subtest (showing one row without data). From the ANAM Access Database User’s Manual (2004); printed with Permission, National Rehabilitation Hospital.

38

3 Neuroinformatics for Neuropsychology

The AADB also contains two demographic tables. These tables draw data from the demographics data file. They include information such as the file location (path) of the data files, date and time of testing, the subject’s name, age, data of birth, etc. Queries: Since the AADB is a Microsoft Access database, it is benefited by the enormous power of querying functions. Queries allow the user to view the data in various permutations. A query can be used to sift out or analyze data based on some criteria and combine data from two or more tables. A user can, for example, utilize a query to select out all subjects in the 22–23 year age range whose scores in one subtest are lower than scores in another subtest and whose mean ANAM scores are within a certain range. A database need not necessarily allow the user to utilize or create queries. Often, when a database is created for a specific purpose, the user access is limited to data entry and a limited number of data views. The AADB does not impose such limitations on the user. At the same time, its query process is designed to cater to a range of users – from those with little database skills to those who can write complex structured query statements. The AADB provides the user with a number of automated (prewritten) summary queries. The use of summary queries does not require any training in writing Microsoft Access queries. The user is taken through a simple wizard that presents choices on tables or variables the user would like to select, information on various aspects of the query, and instructions on how to complete the query (see Fig. 3.1d). Traditional queries or manual queries usually opted for by users familiar with Microsoft Access queries may be written by using the applications default new query interface window. Here various types of queries may be written and they may range widely in scope and complexity. Knowledge of structured query language obviously facilitates this process. Internet Enabled Assessment, Remote Assessment Devices, and Shared Software: ANAM is currently Internet enabled using a “store-and-forward” method (see Cernich et al.). Test modules can be downloaded from an ANAM server to a resident computer. The tests can be run off the local computer without need for continued link to the server (applications designed with platform independent computer languages such as Java make this possible without too many download compatibility obstacles). The data can then be uploaded to the server for storage and analysis. Internet-enabled function is a vital feature of ANAM’s avenues of application in remote cognitive assessment and monitoring. An Internet-accessible Oracle database is currently being developed to store data files from individual ANAM tests. It will be programmed with pattern-seeking algorithms, i.e., normative patterns across tests, populations, and test conditions. The AADB and ClinicView© , a software package for tabular and graphical displays of ANAM test results, are ANAM software utilities that have been developed through the National Rehabilitation Hospital (Washington D.C.). These third party software utilities are available at no cost to the user – see ANAM links above. They are crucial to the digital capture and display of test data. Portable Assessment Systems: Another significant ANAM related development that is aligned with an informatics perspective is that of portable, handheld cognitive screening systems (see Elsmore and Reeves 2004; Elsmore et al. 2007;

Click “Next” to move to the next screen.

Place checks in boxes next to the ANAM subtests that you would like included in the final summary table. Use the arrow keys to the right of this window to view all available subtests.

Place a check here to open the table when the query is finished.

Fig. 3.1d One of the windows of the Query Wizard for automated summary queries in the AADB (the instructions in boxes also appear). From the ANAM Access Database User’s Manual (2004); printed with Permission, National Rehabilitation Hospital

A series of temporary queries and tables are created in the automated query process. It is recommended that you select the option to delete these temporary tables and queries, as shown here.

To automatically export the created table as an Excel file, click here. Add a name for this excel file and “Browse” to choose location to save file.

Enter a name for the table. This table will be placed in the Table view when the query is completed.

3.3 Neuroinformatics Applications and Models for Neuropsychology 39

40

3 Neuroinformatics for Neuropsychology

Fig. 3.1e The BrainCheckers handheld device (PDA) displaying a trial of the ANAM Code Substitution Test. From, BrainCheckers User’s Manual (2007); printed with permission, Dennis Reeves, Behavioral Neuroscience Systems, LLC

Bleiberg et al. 2007; Reeves et al. 2007; Behavioral Neuroscience Systems 2008). BrainCheckers is a handheld computerized neuropsychological assessment system, and it is an advanced version of an earlier system called the ANAM Readiness EvalR PDA. A re-engineered uation System (ARES). BrainCheckers runs on the Palm version of ANAM adapted to the Palm operating system can be self administered with the system (Fig. 3.1e). This library of ANAM tests can be configured in many ways as BrainCheckers is used for different screening, diagnostic, and assessment purposes. The portability of the BrainCheckers system means that it can be utilized in emergency rooms, battlefield environments, and other such settings where desktop computer-based assessment is impractical. It is also cost effective. Both of these were major factors that drove its development. The informatics functionality of this portable system is significant: The system records subject information and test data in databases – a database record for each subject. When the data are uploaded to a personal computer, Dataman, a data manager and PC-PDA communication program, archives the data directly into a Microsoft Access database. From the Access database, the data can be easily pasted to other Windows applications for other kinds of viewing and analysis. A back-end data viewer enables the examiner to view test results and summary data on the PDA so that downloads to a personal computer are not required. As described in Part II, Internet-enabled tools, databases, shared software and devices that help digitize data are fundamental to the building of an NI environment. From an informatics perspective, ANAM has been a milestone development in computerized neuropsychological assessment. ANAM has taken a set of cognitive tests and has adapted them to a framework that incorporates informatics functionality. ANAM digitizes a wealth of raw data and creates data files that are compatible with a powerful database application. ANAM’s database is the only database of its kind available in automated assessment and is the first informatics-enabled

3.3

Neuroinformatics Applications and Models for Neuropsychology

41

database relating to neuropsychological assessment. The mere option of having neuropsychological test data in a database where the user has the facility to structure queries is a hugely innovative step in computerized neuropsychological assessment. With this kind of power given to the user, data can be viewed from various angles and with this facility comes the potential for new observations and discoveries. As a test battery, ANAM has both modified older tests and structured some new tests. In doing so, it has also made the critical addition of an informatics component, i.e., how the tests are set up so as to gather digitized raw data in the form of database-compatible data files. ANAM’s informatics has been geared heavily towards greater data capture from tests that have been used in conventional cognitive/neuropsychological assessment, with emphasis on clearly defined cognitive operations. Nevertheless, it is recognized that there is some functional overlap in the subtests of the current version of ANAM and that other NI methods are needed for ANAM tests to be interpreted with greater precision (see Friedl et al. 2007). As is the case with many NI projects, ANAM is a work-progress. There are numerous issues and new features that its developers are addressing (see Reeves et al. 2007a). An Internet Oracle database, for example, is being developed. This large-scale database will be capable of holding ANAM data from widespread clinical and situational conditions, thus facilitating the development of norms and the study of trend patterns. It is not unfeasible that as ANAM further develops, it will have a specialized team that focuses on algorithms that seek cognitive factors and patterns among the data. Since NI is so new to neuropsychology, neuropsychologists may not immediately realize what an advance ANAM represents in terms of NI for neuropsychology. The AADB in particular marks not a small, incremental step in computerized assessment but a strident leap into the realm in NI. 3.3.1.2 Internet-Enabled Neuropsychological Assessment of Army Aviators (IENAAA) (Walter Reed Army Medical Center (Personal Communication, Mark R. Baggett and Mark R. Kelly, September 27, 2007; Daniel Christensen, Oct. 5, 2007) URLs: http://www.stormingmedia.us/59/5911/A591104.html http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix= html&identifier=ADA401195 Background and Description: Aviation is one of the specialized areas within the US Army. Specially structured cognitive tests have traditionally been applied to assess baseline cognitive screening of army aviators. The critical need to assess specific cognitive functions in potential aviators or injured aviators is obvious in terms of fitness to fly or carry out complex support functions. Candidates for flight training need to have a certain cognitive baseline, especially in functions relating to the flight environment. A pilot with an injury that brings potential neuropsychological compromise needs to be assessed in relation to a functional baseline. In addition to requiring a critical screening tool, the context of army aviation also calls for standardization in the cognitive assessment tools it uses.

42

3 Neuroinformatics for Neuropsychology

The IENAAA project sought to create an Internet-enabled and databaselinked neuropsychological assessment system for cognitive assessment of aviators. The project was carried out in 2000–2001 and while it is no longer active, it led the way for informatics-based neuropsychological assessment. The project was led by MAJ. Mark Baggett, Ph.D. (Deputy Chief of Psychology, WRAMC), CPT. Daniel Christensen, Ph.D. (Neuropsychologist, WRAMC), Dr. Mark Kelly, Ph.D. (Chief, Neuropsychology Service, WRAMC), and LTC. Gregory Gahm, Ph.D. (Chief, Behavioral Health Clinic, MAMC). The project would show that automated computer-based assessment could feasibly be deployed to handle large amounts of data generated through large numbers of service personnel. The major objective of the project was to examine how computerized neuropsychological assessment could be carried out at remote locations via portable computers and how the data could be uploaded to a central server. To this end, a database (Microsoft SQL 7) was developed for data uploads. Two other objectives were tied to this main objective: (a) the development of an online neuropsychological inventory for demographic and medical background data and (b) the examination of relationships between sets of similar cognitive measures (subtests) from three batteries, the US Army-Aeromedical Cognitive Assessment Tool (USA-ACAT), which is a test battery derived from ANAM 2001, SynWin (Synthetic Work for Windows), and CogScreen. A secure server was set up at Walter Reed Army Medical Center. The server enabled downloads of the ANAM, Cogscreen and SynWin tests. The tests were downloaded at Fort Campbell, Kentucky, the evaluation site, where they were administered to 100 army pilots via desktop or portable computers. The resulting data were then uploaded to the server and the investigators were successful in accessing the data. Subsequent data collection via the website was also carried out at Fort Rucker, Alabama, in a related study. The IENAAA was simply a feasibility study about applying a set of standardized information-technology protocols to large-scale neuropsychological assessment, but the project pushed forward the concept of Internet-enabled assessment. It laid significant groundwork. From a plain NI-neuropsychological perspective, its initiatives are examples of critical foundational steps in any NI undertaking of Internetenabled neuropsychological assessment: A neuropsychological questionnaire for demographic data was modified to function as an on-line instrument. The USAACAT was configured as an application that could be downloaded and then run on a remote computer. The application would package the test results as a data file that could be uploaded to the database. The SQL server platform lay at the heart of the project – a cohesive center for application dissemination and data collection and analysis. ANAM and its Internet-enabled components now provide the functionality called for by the IENAAA. Currently, at the Defence and Veteran’s Brain Injury Center (DVBIC) at Fort Bragg, North Carolina, ANAM systems are used on the soldiers of the 82nd Airborne Division to obtain neuropsychological baselines prior to deployment and for examination following traumatic brain injury. The setup of the automated assessment system has been modeled on the IENAAA.

3.3

Neuroinformatics Applications and Models for Neuropsychology

43

Conclusion for Sections 3.3.1.1 and 3.3.2.1 The examples of ANAM and the IENAAA demonstrate the unparalleled gains of NI-based neuropsychological assessment. The examples represent responses to the need for streamlining and efficiency in the assessment process, in the context of the army. They demonstrate the process through which these needs can be met and how in the same process greater gains in assessment data can be made. Many concepts are successfully demonstrated by the examples, including the digitizing, databasing, and mining of assessment data. On a broader level, the examples described in this subsection illustrate the building of NI-based assessment systems at multiple levels – from the redesigning of test batteries to servers at remote locations to which data can be uploaded. The NI approach to assessment is optimized with attempts to spell out target cognitive operations with precision. Computerized tests of these operations are then carefully designed – not simply to be computerized versions of pencil-and-paper tests, but as tests that can be interfaced with databases. A key objective is to derive rich data that can be stored, analyzed, and re-rendered. Assessment can then be automated, Internet-enabled, and thereby be carried out in variety of contexts. NI methods when applied to assessment add dimensions that cannot be achieved with conventional or simple computer-assisted assessment tools. The framework of a shared platform seen in ANAM and the IEAAA provides the discipline of clinical neuropsychological assessment with a view to how a large-scale NI system can reinvent the discipline.

3.3.2 Visuospatial Processing, Visual Attention, and Spatial Neglect Visuospatial function is a major functional domain in cognition and is an area of significant focus in neuropsychology. It is an immensely broad domain, covering some aspects of primary perception but largely concerned with high-level vision (visual and spatial cognition) and visuomotor integration. As a descriptive term, “visuospatial” embraces a range of spatial processes relating to visual perception of figures, forms, objects, and body parts; mental/cognitive maps and representations; and various dimensions of space in which one acts. Defining Visuospatial Function: The complexity of this domain has made for an ongoing challenge to neuropsychology. There has been a clear absence of a breakdown of taxonomy of visuospatial processes for neuropsychologists to reference in describing the nature of a spatial deficit. (Contrast this with breakdown of language into units such as phonemes, morphemes, words, syntax, semantics, discourse, etc.) As a result, the constructs underlying visuospatial function have been vague and the terminology used to reference this domain has reflected great confusion: “Attempts to select and organize findings from neuropsychological studies of visuospatial disorders into a coherent and comprehensive schema are fraught with

44

3 Neuroinformatics for Neuropsychology

problems facing any reviewer in any field, only perhaps to a greater degree. The nature of the data, the absence of a clearly defined conceptual framework, and a history of confusing and inconsistent terminology all result in bias and lack of clarity” (Newcombe and Ratcliff 1990, p. 333). A fundamental problem has been the lack of agreement regarding the definition of visuospatial function. Further, descriptions of what is visuospatial in neuropsychology have been crafted largely with reference to operational definitions as carried by visuospatial tests: “A common practice is to define the visuospatial dysfunction on the basis of a particular experimental task rather than on conceptual or anatomic criteria . . . the term visuospatial must be explicitly defined . . . it is important to precisely specify what is being studied and how it conceptually relates to the broader theoretical framework of spatial cognition . . . ” (Levin 1990, p. 161).

In response to such problems, Jagaroo (1999) suggested that when referring to “visuospatial” function, neuropsychologists should attempt to define the particular spatial operation/s involved, the spatial frame (e.g. egocentric, topographic) that may be implicated, and the neuroanatomic systems at play. Visuospatial tests in neuropsychology have often been “designed to probe the capacity for visual analysis and synthesis” (Benton 1994, p. 6). Tests such as the Rey Complex Figure Test, the Hooper Visual Organization Test, and the various spatial subtests of the WAIS are tests that index amalgams of spatial processes. They paint a rather general picture of visuospatial integrity. These tests have not been structured to parse out precise spatial operations in cognitive or neural terms, and they also happen to be tests that have seen little or no evolution over the past many decades. The simplicity with which clinical neuropsychology has historically conceived of the visuospatial domain inevitably elevated the role of simple assessment methods: “The simplest and most direct procedures for revealing deficits in visuospatial functions involve paper-and-pencil drawings of familiar objects and geometric forms. Commonly used objects include clock, daisy, house, elephant, bicycle” (Goodglass and Kaplan 1979, p. 17). As alluded in the preceding subsection, among the variables influencing performance in these tests are, perception (involving multiple levels of processing); high-level spatial representations, and the generation of a motor plan based on the representations when motor actions are involved; and feedback between the neural centers involved with all these processes (see Jagaroo 1998a,b). Visual and Spatial Processing Described in Cognitive Neurobiology and Cognitive Neuroscience: The problems laid out above constitute a serious and unfortunate lag in neuropsychology because the study of visual and spatial processing in neurophysiology and cognitive neuroscience has systematically progressed. It has described greater and greater levels of complexity in visual and spatial processing in cognitive terms for which it has identified corresponding and complex neuroanatomic systems (see Van Essen et al. 1990; Chalupa and Werner 2004, for reviews). The pioneering work by Ungerleider and Mishkin (1982) described neural pathways that project from the visual cortex to inferior temporal and posterior parietal cortices in the macaque brain, carrying visual and spatial information about an

3.3

Neuroinformatics Applications and Models for Neuropsychology

45

object’s identity and location, respectively. The work provided the first systematic account of visual-spatial processing beyond the visual cortex, the functional properties of which had been mapped earlier in cats and monkeys (Hubel and Wiesel 1962; 1978). Through the 1980 s and 1990 s, neurobiological and neurophysiological studies on visual and spatial processing (largely in non-human primate brains) saw an explosion in numbers. They aimed to draw clear associations between visual/spatial processes and cortical systems. Noteworthy for neuropsychology is the precision with which they defined visuospatial, perceptual, and cognitive operations. These studies were concerned with such processes as the mnemonic coding of coordinates involved with mapping spatial plans for eye movements in the dorsolateral prefrontal cortex (Funahashi et al. 1989); spatial frameworks for movement kinematics and movement control coded by the area 5 of the parietal cortex (Kalaska 1991); the representation of head-facial profiles in neurons around the superior temporal sulcus (Perrett et al. 1992); and multimodal spatial coordinate representations and coordinate transformations in the posterior parietal cortex (Andersen et al. 1993). Synthesizing such data and the vast amount of data on cortical connections in neurobiology, Felleman and Van Essen (1991) compiled a map that indicated some 32 cortical areas involved with visual processing, including more than 300 connections among these areas and a distinct neural processing hierarchy among these areas. In visual neuroscience, this often cited map has served as a major reference for the functional anatomy of visual and spatial processing. Cognitive science has been equally focused on the “dissection” of visuospatial processes: Earlier pioneering work laid out, for example, the variables of features in visual attention and object recognition (Treisman, and Gelade 1980; Treisman 1982); graded differential hemispheric processing based on spatial frequency (Sergent 1982; 1991) and categorical versus coordinate relations (Kosslyn 1987; Kosslyn et al. 1992) in the visual scene; and the role of representational coordinate systems in mental manipulation (Just and Carpenter 1985). Functional imaging then signaled the possibility of localizing discrete spatial operations (see Roland and Friberg 1985; Haxby et al. 1991) and soon the landscape of visual cognition was transformed. Visuospatial processes would now be gleaned through a range of cortical regional activation patterns, for example: The posterior parietal area in relation to 3-dimensional mental rotation/manipulation (Cohen et al. 1996; Richter et al. 1991); prefrontal, premotor, and occipito-temporal regions in relation to spatial working memory tasks (Jonides et al. 1993); responses properties of a fusiform gyral area (the fusiform face area) in relation to faces (Kanwisher et al. 1997; Kanwisher 2000) and its potential to be tuned to sets of objects that share similar looks and spatial form (Tarr and Gauthier 2000); a parahippocampal gyral area (the “parahippocampal place area”) in relation to visual scenes that convey relative layout of objects (Epstein and Kanwisher 1998); and a region of the lateral occipitotemporal cortex (the “extrastriate body area”) with regard to pictures of bodies and non-facial parts of bodies (Downing et al. 2001). Numerous cortical modules for visuospatial processing could then feasibly be laid out as functional-anatomic maps built from functional imaging studies alone (see Grill-Spector and Malach 2004).

46

3 Neuroinformatics for Neuropsychology

Implications for Visuospatial Assessment in Neuropsychology: Against the vast backdrop of visuospatial research in neurobiology and cognitive neuroscience, the modes of visuospatial assessment in clinical neuropsychology are baffling. They are at odds with the type of constructs and the level of cognitive and neuroanatomic detail with which visuospatial processing is described by these disciplines. Visuospatial assessment simply cannot afford not to look to the data from these disciplines. The data essentially amount to the finest available grain of spatial processes and their neuroanatomic systems. This, however, does not mean that visuospatial assessment should defer to cognitive neuroscience or entirely to the modality functional imaging for a fine assessment of these functions. Clinical visuospatial assessment can draw on all the types of data laid out above and it can factor key principles in designing a new arsenal of visuospatial assessment tools. It can, at the same time, by adopting NI methods, remain fundamentally in the realm of clinical neuropsychological assessment. To be shaped by the data of visual neuroscience means that the clinical visuospatial assessment will gain greater construct validity, greater alignment with neuroanatomic systems, and that its methods will be conferred with immense research potential. To accomplish this, NI, once again, is a critical means and there are many potential applications. The discussion below will focus on one major avenue of application. A Role for NI – Coding the Medium of Spatial Tests: One of the reasons for a general imprecision when addressing visual and spatial operations in neuropsychology is the lack of sophisticated tools to code a common medium for visuospatial assessment – the area of space (a page or a computer screen) on which visuospatial tests are presented and the stimulus presented within this space. Points in the spatial field of assessment would each need to be coded in some way. There have been some descriptions of (a) computer interfaced methods and algorithms to enhance the sensitivity of cancellation tests (Donnelly et al. 1999) and (b) computer recording of coordinate points in line bisection tests (Potter et al. 2000). The studies are notable for devising simple computer-enhanced methods to code the spatial field. A comprehensive informatics system for spatial assessment, on the other hand, would require a spatial coding system where each point or region in the assessment space is given an address, and it would also require an array of software modules to handle the data. With this system, the addresses of all the coordinates that make up this space could easily be programmed into a database as can the address of an occupying stimulus. Information on varying stimulus patterns and the patient’s responses to each condition can also be databased. With this data, a wealth of insights can be gained. Two NI projects are described below, both to do with NI systems for visuospatial assessment. One places more of an emphasis on decoding patterns of spatial neglect while the other focuses on visual search and visual attentional patterns. Both represent significant departures from conventional tools. Their functions enable (a) databasing and coordinate-based analysis of spatial information and (b) the capture and rendering of kinematic and temporal spatial data.

3.3

Neuroinformatics Applications and Models for Neuropsychology

47

3.3.2.1 Computational Visual Field for Neglect (CVFN) URL: www.cvfn.net The CVFN is a computational tool designed mainly for assessment and research of spatial neglect. It is being developed by the author at Boston University and Emerson College, based on NI architecture described in the past (Jagaroo 2002). A beta version of the tool is currently being tested in a pilot study. Background: The phenomenon of spatial neglect has an immense bearing in neuropsychology. The complexity of the phenomenon draws upon the processes of attention, motor planning, spatiotopic encoding, and representational (“mental”) space. A few neuropsychological models have attempted to explain the condition, especially the predominance of left field neglect following damage to the right hemisphere (see Farah 2000; Heilman et al. 1993). One of these models, the “representational” theory of neglect, suggests that one half of the internal representation of space is either cut or “extinguished” (Bisiach and Luzzatti 1978; Bisiach et al. 1996) or that this mental representation of left field hemispace is compressed or distorted in a manner that can be described with mathematical precision (Burnett-Stuart et al. 1991; Halligan and Marshall 1994; 1998). In hemispatial neglect, it is primarily the representation of length along the horizontal axis that is distorted (Burnet-Stuart et al.) though neglect can occur along any axis in the spatial field. The degree of perceptual distortion and the degree of attentional bias differs for each axis of orientation (imagine the horizontal axis being rotated about its midpoint, traversing successive axes as it completes a 360◦ rotation). In the line bisection task, a test that is very sensitive to neglect, patients transect the line significantly to the right of the true midpoint (see Bisiach et al. 1983) and the degree of transection displacement is linearly related to the length of the line (see Nichelli et al. 1989). Halligan and Marshall have explained such effects with the notion of distortions in representational space that are governed by fixed geometric constraints. They illustrated through a series of experiments a central idea that right hemisphere damage can distort the representation of space: “These distortions are fully consistent with a model whereby points in ‘left space’ are compressed rightwards; the compression function is linearly proportional to the coordinates of Euclidean space.” (1991, p. 628) This idea is consistent with animal models of the posterior parietal area (area 7) that suggest it to be a region specialized for coordinate mapping of space (Andersen et al. 1987; 1992; Stein 1991; Duhamel et al. 1991) and with other neuropsychological models of the area that cast it as a center for spatiotopic coordinate transformations (Jagaroo 2004). Altogether, these models suggest that representation of the entire visual field is integrated in the right PPC along spatiotopic principles – a particular coordinate in external space in mapped to a precise coordinate in the right PPC representational matrix. They also suggest intricate spatial dynamics tied to neglect and how these dynamics might be experimentally manipulated in studying neglect. They point to the degree to which neglect and the parametric space being neglected can be deconstructed given their discrete geometric and neural properties. A proper deconstruction of the hemispatial neglect must therefore take into account the context of the stimulus within a coordinate-based visual field.

48

3 Neuroinformatics for Neuropsychology

Degree of neglect calculated by counting coordinate cells

Cells of the Grid

Major coordinates recorded in an Excel Sheet or Access Database

Grid (invisible to patient)

Stimulus Item (visible to patient)

Fig. 3.2a Schematic illustration of the grid matrix

The Limits of Conventional Tools in Assessing Neglect: "Paper-and-pencil" tests have been the primary method of assessing the visuospatial aspects of neglect. Some examples are the Boston Visuospatial Quantitative Battery, the Rey Complex Figure Test, and letter cancellation and line bisection tasks. In these forms of assessment, a sheet of paper or a computer screen containing the target illustration is presented to the patient. The patient is required to look at the visual stimulus and copy it, reproduce it from memory, answer questions about it, or carry out bisection or selective cancellation tasks. The patient s reproduced drawing or answers about the visual stimulus are then qualitatively assessed. Simple quantitative assessment methods are sometimes manually applied to the patient s drawings. There is a huge discrepancy between the neuropsychological complexity of neglect and the neuropsychological tools used to assess neglect. Paper-and-pencil tests are fine in terms of the gross assessment of neglect but they provide no information about the intricate mechanics of the condition. As indicated, simple illustrated drawings cannot produce information about precise spatial coordinates of the visual stimulus or the visual field that contains it. Equally limiting, because these tests are static in form, they are not useful in examining temporal and dynamic effects – how moment-by-moment incremental changes to a stimulus, for example, can change the neglect pattern. Many of the more interesting and hidden dynamics of neglect remain obscured in conventional assessment. In order to capture precise patterns of visual field gradients in neglect far more complex methods are needed. NI Approach to Assessing Neglect: In view of the limitations described, the NI approach to assessing neglect must involve a digitized visual field and computational system to map the coordinates of the visual field. This system should be able to capture the subjective coordinates of a visual stimulus (the perceiver’s response or production) in two-dimensional Euclidean space and measure this against an objective reference frame. In clinical assessment, the visual field is typically a parametric section of space defined by the area of a page or computer screen. The CVFN takes a large computer screen and turns it into a computational visual field matrix. In this system, a computer-generated matrix divides the computer screen into many cells

3.3

Neuroinformatics Applications and Models for Neuropsychology

49

(squares) to produce a grid. The fineness of the grid can also be varied from a few large cells to thousands of small cells. The cell at the midpoint of the grid is given the coordinates (0; 0). In one configuration of the grid, cells on the X-axis to the left of the center are assigned negative coordinates, e.g., from –1 to –40. Cells to the right are assigned positive coordinates, +1 to +40. Cells along the Y-axis, above cell (0; 0) are assigned positive coordinates, +1 to +30. Cells below are assigned negative coordinates –1 to –30. The resulting matrix amounts to a grid of uniform cells, each with a unique address – a set of unique coordinates (see Fig. 3.2b). The default position of the grid is such that coordinate (0; 0) defines the midpoint of the presented visual field: The Y-axis (–30 to +30) defines the midline separating each hemifield and the X-axis (–40 to +40) defines the horizontal separation between equal-sized upper and lower fields. The grid acts as a reference system, i.e., an objective Euclidean map. When used in assessment, two computer screens are involved. The examiner views one screen and the subject views another. Both screens are tied to the same control system but the grid is invisible on the subject s screen. On the subject’s screen, the flower will appear on a plain background. The flower, however, occupies certain cells on the invisible grid on which it is imposed. Each cell in the grid is mapped to a database called the Coordinate Recording Database (CRD). The position of a stimulus placed within the grid is also mapped to the database. For each cell in the grid, the CRD has other data fields in addition to the fields the code the cell’s position. The other fields can record attributes such as stimulus color, percentage of the field being occupied, etc. There are a number of other important programmed components that make up the system: The grid controller is used to stipulate how many cells should make up the matrix, that is, the cellular density or complexity of the matrix. It can also elongate cells and cell contents vertically or horizontally to warp regions of space as may be required experimentally. The option of varying the matrix configuration is an important one. A large-cell grid configuration, for example, would be better suited to a patient exhibiting very gross neglect. The coordinate computational kernel (CCK) is a computational engine that is composed of algorithms and queries. This component essentially converts grid data to numerical arrays and then compares the data. Computational criteria can be programmed as needed. A graphics database provides a selection of basic shapes and patterns and can also accommodate other figures and pictures that may be inputted for use. A graphics kernel provides a simple graphics editor to alter the shape, form, size, and color of a stimulus. A patient profile database is comprised of a set of tables that store the patient’s performance data. It can be programmed with analytic queries as needed. An analytic module is a set of strings, macros, and queries that run on the CRD and Patient Profile Database to look for relationships between stimulus/grid conditions and the patient’s performance and to carry out inter-subject comparisons. All the databases included in the system are designed using Microsoft Access; other components are for the most part programmed with C++. The beta version of the system employs only some of the components described: The CRD, a simple Grid Controller (offering 3 three-grid configurations), and the Patient Profile

50

3 Neuroinformatics for Neuropsychology

Fig. 3.2b Schematic illustration of the CVFN’s Coordinate Addressing System. From, Jagaroo (2002). Dynamic computational visual field matrices: A computerized mapping system for the analysis of visual perception, spatial processing and featural recognition. In C.H. Dagli, A.L. Buczak, J. Ghosh, M.J. Embrechts, O. Ersoy, & S.W. Kercel (Eds.) Smart Engineering System Design, Neural Networks and Fuzzy Logic, vol. 12. Printed with permission, American Society of Mechanical Engineers Press

Database. Other components are being worked on as the greater project progresses. Figure 3.2c provides a schematic of the architecture of the CVFN system. Functionality Gained Through a Computational Grid Matrix: An infinite number of stimuli and visual field variables and permutations are enabled by this NI system

3.3

Neuroinformatics Applications and Models for Neuropsychology GRID CONTROLLER VAR (1:8) Seq 1 Σ((+3–5), (–5+6)) Seq 2 Σ ((+17–9), (–30+29)) If CRD(X,X) = open Enter (X,X) table[gridpoint] power =3

Controls grid density and enables warping of grid sections

GRAPHICS DATABASE Offers a selection of graphical objects (stimuli). Objects can also be added to this database.

Increment = (4) Mark (–1+4) Begin level {23} Position if ƒ(+2–5) bit (0, α –1, 1, α –5) bit (3, δ +1, 4, δ –7) Stop Level {45} End

51 COORDINATE COMPUTATIONAL KERNEL Converts grid data into numerical arrays for data comparison

(+4+3) (+5+7) (–3+5) (+4–5) (–23+24) (–30+29) (+13–14) (+27–18) (–10+12) (–1–3)

COORDINATE RECORDING DATABASE

GRAPHICS KERNEL Simple graphics editor for creation and manipulation of objects: size, color, etc. Select Fig 8 Bound M:shape = rec Border = 6 Size = DEF Deg vLap (4:7) circ Warp inc=1 down

THE GRID MATRIX Composed of numerous cells each with a unique coordinate address

ANALYTIC MODULE Analyzes performance (intra–and inter– patient) Assoc String 5 P1[C3] com P2[C3] Dim = tpointt Write data [Table 10]

Records the coordinates of each cell in the matrix as objects places in the matrix

Subject/Patient 78 Run Query C3 Adjust with Macro P78 Subject/Patient 34 Run Query C3 Adjust with Macro P34 Subject/Patient 10 Run Query C3 Adjust with Macro P10

PROFILE DATABASE Stores data on patient’s responses/performance

Fig. 3.2c Schematic illustration of the CVFN’s Architecture. From, Jagaroo (2002). Dynamic computational visual field matrices: A computerized mapping system for the analysis of visual perception, spatial processing and featural recognition. In C.H. Dagli, A.L. Buczak, J. Ghosh, M.J. Embrechts, O. Ersoy, & S.W. Kercel (Eds.) Smart Engineering System Design, Neural Networks and Fuzzy Logic, vol. 12. Printed with permission, American Society of Mechanical Engineers Press

of assessing neglect. Analysis need not be limited to stimuli presented on-screen. A subject’s drawings of stimuli can be scanned and fed into the system for analysis. The key advances of the computational grid system over conventional methods may be summarized as follows: Coordinate Referencing and Tracking: In line bisection tasks, performance accuracy can be systematically improved when the line’s true center is incrementally

52

3 Neuroinformatics for Neuropsychology

moved to the right of the visual field midline (Halligan and Marshall 1995). Such discoveries have been made when both the line length and the point of transection have been carefully measured (see Bisiach et al. 1990; Halligan and Marshall 1991). Such data have been obtained manually, often requiring hundreds of paper-illustrated variations of a stimulus. The tedium of such manipulations can discourage further such analyses. With a computerized matrix system, all the following data can be preset and/or referenced in seconds: (a) the coordinates occupied by the line – indexing line length and displacement; (b) the coordinate corresponding to the point of transection; (c) the relative degree (in numerical units) to which transection is displaced towards one hemifield; (d) the pattern of the perceiver’s regression or improvement over a number of trials or over a period of time; (e) the aggregate critical “sector” of the line where transection occurs (within and across patients); (f) the average coordinate position of the patient’s (biased) midline based on his/her line transections; and (g) transection points across varying line lengths. Stimulus Displacement and Field Gradients: In neglect, a patient’s omissions are not always limited to the left hemifield (see Small et al. 1994). Omissions may occur along the midline and in some cases, in the right hemifield. This neglect “gradient” often defines a patient’s unique neglect profile. Attempts have been made to map this gradient along a simple 3-point scale (see Small et al.), a system that crudely neglects large sectors in the hemifields. With a computerized matrix, the gradient can be generated instantaneously and can be accurate to the point of a matrix cell. All averaging functions and tracking functions (over time) can be applied to gradient assessment. The stimulus pattern can also be displaced by very fine increments towards either hemifield in order to establish the X-axis point at which the gradient is zero, that is, by displacing the stimulus by precise increments, one cell displacement at a time, it will be possible to compute the patient’s perceived (displaced) midline. Masked Stimuli, Global-Local Effects, and Figural Modulation: The system can be programmed to implement masking effects, obscuring effects, and global-local effects on visual stimuli, for example, (a) stimuli in one hemifield can be masked at varying degrees along a fine gradient; (b) featural or outline cues can be added to one hemifield, again to assess how perception is modulated under these conditions; (c) framing effects, shape modulation, and global cueing effects can also be incorporated; and (d) the effects of masking or featural cueing on neglect can be tracked. Chimaeric Stimuli and Stimulus Compression: Chimaeric stimuli used in the study of visuospatial processing have produced dramatic results but have been limited largely to pictures of faces and simple objects (see Young et al. 1992). Again, practical constraints in manually generating and altering these figures prevent complex permutations. With a matrix-controlled system, almost any aspect of chimaeric stimuli can be easily adjusted, again, akin to the adjustments allowed by a simple computer graphics application: (a) the matrix of one hemifield governing the chimaeric piece in that field will be able to adjust the dimensions and color of the figure; (b) adjustments can be made to one half of the chimaeric figure so as to make it incrementally similar or dissimilar to the other half. In this way, the degree of

3.3

Neuroinformatics Applications and Models for Neuropsychology

Blinking arrows in neglected field while displaying stimulus item

Center-to-left Arrow Train in neglected field while displaying stimulus item

53

Center-to-left Color Wave in neglected field while displaying stimulus item

Fig. 3.2d Schematic illustration of dynamic stimuli with movement/animation field-enhancing effects in the CVFN

“chimaeric gradient” can be manipulated; and (c) differential chimaeric gradients can be computed for different categories of objects or for different sets of coordinates. Animated Priming Effects: Another major innovation of the system has to do with its capacity to introduce movement or animated effects. In neurocognitive research, the implementation of animated/movement effects made possible through animation software offers huge potential. It provides a means of studying motion perception systems as well as spatial processes that can be modulated by motion dynamics (see Jagaroo and Wilkinson 2008, for a discussion on this topic). In the CVFN, movement effects can add salience and highlights to the neglected field. Dynamic stimuli such as blinking arrows, moving trains of arrows, and color waves can be applied to the background of the neglected field. The aim is to highlight the field and in theory, to do so using synchronous movement patterns that trigger the dysfunctional neural centers that may produce some forms of neglect. Figure 3.2d provides a schematic illustration of a few examples of dynamic stimuli. In summary, the power of the matrix lies in its computerized (digitized) coordinate structure and function. The system brings computational, graphical, and database power to neuropsychological assessment of neglect. This NI approach to studying neglect marks a radical departure from conventional pencil-and-paper methods. The complexity of the system is attuned to the complex spatial dynamics of neglect. The system can potentially be adapted to studying a broader range of visuospatial functions where testing utilizes a parametric visual field. 3.3.2.2 Computer-Assisted Cancellation Test System (CACTS) (See Huang and Wang 2005; Wang et al. 2006; Huang and Wang 2008; personal communication, Ho-Chuan Huang, October 19 and 22, 2007; July 21, 27, and 28; 2008) CACTS is a computerized system for the graphical analysis and assessment of visual search and visual attention. It is geared primarily for cancellation tasks but can be adapted to other visual search/attention tasks. CACTS has been developed through a collaborative effort between information technology and occupational therapy specialists at the National Kaohsiung University of Applied Sciences and

54

3 Neuroinformatics for Neuropsychology

the National Cheng Kung University, respectively, in Taiwan. Its development has been supported in part by the National Science Council of Taiwan. Background: The development of CACTS was motivated by some key factors. It was recognized that while visual search, scanning, and cancellation tasks have been widely used in visuospatial assessment, inherent limitations are contained in the conventional paper-and-pencil nature of these tasks (of the type described in the preceding subsection). Again, the static nature of these conventional tasks makes for a poor rendering of the overall visual search and attention process. Information about onset time, search paths, search strategies, stops and pauses, etc., can be captured only crudely. A computerized interface was sought to address these limitations. A second consideration was the need to capture visual search and scanning trajectories without necessitating the use of eye-tracking devices that usually require more elaborate hardware and a specialized testing environment. A highly portable system that would be capable of registering attentional search would be much more preferable in, for example, a school setting where a clinician may need to assess visual attention in a large population of children in an efficient manner. CACTS was therefore developed to utilize only a personal computer and a monitor. The aim was to create a computer-interfaced system to capture dynamics processes in a subject’s visual search and attention during cancellation tasks and then apply algorithms to analyze the attentional pattern. Design and Architecture of CACTS: The system runs on a personal computer using Microsoft’s Windows XP platform. The software is programmed in C++. In the latest setup of CACTS, the hardware consists simply of a tablet computer with a touch-sensitive screen and a cordless screen pen. When the user applies the pen to the screen, its pressure-sensitive tip transmits signals to the tablet computer at about the point of contact. Euclidean (x, y) coordinates together with corresponding time of the event are automatically recorded in a database. CACTS involves four software modules (see Fig. 3.3a). A profile module serves to hold a subject’s general and demographic data such as age, sex, and education level. A test module is designed to contain various forms of attention/cancellation tests. An outcome analysis module records and analyses a subject’s attentional pattern and provides the examiner with graphical tools for the analysis. A print module allows the examiner to be selective in terms of data to be printed. A Microsoft Access database is used to store the subject’s demographic and test data. Kinematic and Metric Functionality: One component of the CACTS database, the process table, stores data on kinematic aspects of the subject’s visual scan path. General attentional dynamics can subsequently be pictured using this data. Kinematic data include response coordinates, distances between consecutive response coordinates, and pause and completion times between consecutive response coordinates. Total times for movement, pauses, and completion can also be recorded. The system’s analysis module can use this data together with a pattern identification algorithm to construct a scaled figural representation of the subject’s search sequence and overall scanning strategy (see Fig. 3.3b). Sequential nodes along the visual scan path can be highlighted. The coordinates marking the subject’s attentional center can also be highlighted. This convenient function can effectively

3.3

Neuroinformatics Applications and Models for Neuropsychology

Examiner

Participant

PROFILE MODULE User Profiles

OUTCOME ANALYSIS MODULE

Test Information

TEST MODULE Test Stimuli 1, 2, 3 (Numbers)

•

, (Symbols)

55

Type of Presentation

A, B, C (Alphabets)

Structured

" 日"" and "工 (Radicals)

Random

Raw Data Analysis

Updating of Data

Analysis of Search Patterns

Analysis of Scanning Trend

PRINTING MODULE Data Printing

Graphic Printing

Databases

Fig. 3.3a CACTS System Architecture. Adapted from Wang et al. (2006) and printed with permission, Elsevier

summarize an individual’s search strategy; it can also sort out different search patterns among a group of individuals. In the current version of CACTS, the test module contains four forms of tests stimuli – numerical forms (0–9); alphabetical forms (A–Z) in upper case; 50 two symbol forms (example as “ ” and “•”); and 26 basic components of Chinese alphabetical characters, called radicals (example, “ ” and “ ”). The system also allows the option of adding new stimulus items to test module. The stimuli may be presented in random or structured arrays. As is the case with the CVFN, the center point in the CACTS screen is the 0;0 coordinate point. All other points are referenced in terms of x and y coordinates relative to the x and y axes (this is a standard method in many image analysis applications). To aid the examiner with analysis of the subject’s performance, the coordinates in the CACTS screen can also be grouped into nine regions – a central circle surrounded by eight circular demarcations. Dividing the screen in this way allows for a simpler delineation of the response pattern, such as “center,” “left,” “far left,” etc. The examiner can view test results with a screen interface of the Outcome Analysis Module (Fig. 3.3b,c). Different outcome screens offer different data sets or different levels of detail for a data set. Figure 3.3b is a snapshot of one outcome screen. It includes some of the subject’s personal data such as name, education

Test modes: star, symbol, etc.

A pie chart showing the accuracy (blue and yellow for right and left screen, respectively; 0% forthe inaccuracy).

Click for the test results in different mode of cancellation tasks

Time, accuracy, omissions and commissions on the lateral screen

Fig. 3.3b Snapshot of CACTS outcome analysis screen 1. From, Wang et al. (2006); printed with permission, Elsevier

Structured or random arrays

Records of each cancellation by orders

Personal data

Total task time, accuracy, omissions and commissions on the screen

Scan path from the first (blue circle) to the last target cancelled (red square)

56 3 Neuroinformatics for Neuropsychology

3.3

Neuroinformatics Applications and Models for Neuropsychology

Scan path from the first (blue circle) to the last target cancelled (red square)

Time, accuracy, omissions and commissions on the lateral screen (left column) and the whole screen (right column)

57

Duration or distance amplitudes of consecutive cancellations

Asymmetry pattern, Attention and neglect center with vector

Fig. 3.3c Snapshot of CACTS Outcome Analysis Screen 2. From, Huang and Wang (2005). Toward a graphical analysis tool for computer-assisted assessment of visual search patterns. Paper presented at the Fifth IEEE International Conference on Advanced Learning Technologies; printed with permission, IEEE (©2005 IEEE)

level, and date of testing. It also shows the number of cancellations made by the individual in the left and right half-fields; the number of correct responses in each half-field; total time spent in each half-field, and a graphical depiction of the subject’s scan path. Another outcome screen, shown in Fig. 3.3c, includes some of the above data and offers more detailed temporal data tied to the subject’s scan path. Cancellation tasks constitute a special category of tests in the assessment of attention and visuospatial function. Their simplicity, ease of administration, and often clear results have given them a proven utility in neuropsychology. These factors have also resulted in the tool remaining essentially unchanged in more than three decades. So much more data can come from cancellation tests, data that are not feasibly realized by the conventional paper-and-pencil versions of these tests. CACTS is a system that changes the landscape of cancellation tasks. It adds a powerful informatics dimension to these tasks. In so doing, valuable hidden information about attentional strategies and visual field biases is brought to the fore.

58

3 Neuroinformatics for Neuropsychology

Conclusion for Section 3.3.2.1 and 3.3.2.2 Much of visuospatial assessment uses the medium of a spatial field defined by a sheet of paper or a computer screen. The CVFN and the CACTS systems demonstrate ways of digitizing this spatial field so that points in the field can be linked to a database. With this kind of functionality, a subject’s scanning/response patterns can be tracked with mathematical precision and without the need for eye-tracking hardware. These systems can practically be used in the clinical environment. The dynamic operations enabled by these systems are invaluable is tying spatial assessment to rich theoretical models of spatial processing. Conventional tools cannot capture precise spatial and temporal properties of hemifield bias and coordinate-based processing, neither can they dynamically alter stimuli and record the effects. Visuospatial processing involves a host of complex operations and none but informatics tools match up to this complexity.

3.3.3 Speech, Language, and Aphasia The broad functional domains of speech and language are ripe and ready for neuroinformatics. Production and perception of speech and language involve complex neurocognitive mechanisms. These mechanisms continue to be investigated at an exciting pace only to unveil an even more multifaceted puzzle. In formulating models or dealing with assessment of speech and language, neuropsychology is called to (a) parse out contributing cognitive variables such as phonology, syntax, semantics, discourse, working memory, auditory and orthographic perception, and sequencing of cognitive and motor programs, (b) match these functions to their corresponding neural systems, and (c) determine the nature of disruption to these functions following damage to a neural system. The difficulty of such tasks need not be emphasized. Neuropsychological Models of Language: The evolution of neuropsychological models of language, including neuropsychological speech mechanisms, while marked by numerous epochs, can be viewed in terms of the degree of emphasis placed on functional-anatomic systems, sensory and motor processes, cognitive processes, or neural networks involved with speech and language (Arbib and Caplan 1979; Benson 1993). Neuroanatomic localization of psycholinguistic and speech sensorimotor processes has been an underlying theme. The Broca-Wernicke model of language has been a canon in neuropsychology. Key events around this model are common historical knowledge in the field (and even though this model has been described endless number of times in neuropsychology, a few highlights are called for here to serve the discussion building up to the need for NI). Broca’s (1861/1865) reports associated motor speech production with the frontal opercular region. Wernicke’s (1874) extensive monograph served to delineate processes of sensory (auditory) language comprehension tied to the posterior aspect of the superior temporal gyrus and to highlight the fiber tract linking this area to Broca’s speech motor area. Lichteim’s (1884) elaboration on Wernicke’s model (the Wernicke-Lichteim model) then consolidated

3.3

Neuroinformatics Applications and Models for Neuropsychology

59

what would become an anatomic-connectionist perspective of aphasia. This classical view was famously reformulated by Geschwind (1965): Wernicke’s area codes complex auditory patterns with meaning, a process that is required in generating words to be spoken and in recognizing spoken words. Through the longitudinal white matter pathway of the arcuate fasciculus, the coded information from Wernicke’s area is serially transferred to Broca’s area. Here it is rendered into neural code for speech motor sequences required for speech production. This reformulation was “disarmingly logical and compelling” (Goodglass 1993, p. 40). It had practical simplicity in that cardinal features in Broca’s and Wernicke’s aphasia could neatly be addressed to neuroanatomic sites.2 “But it also comes into conflict with other clinical observations and with more sophisticated views of possible information processing mechanisms used by the brain. Progress toward understanding the relationship between anatomy and language functions requires a more fine-grained analysis of anatomical detail in the one hand, and language processes on the other” (Goodglass 1993, p. 42). While a century of research on brain and language is not being discounted, it is the emergence of neural network models of cognition that brought major redirection in the neuropsychological models of language. “Neurolinguistics must be computational,” asserted Arbib and Caplan (1979). By the late 1980 s, computational models of neurocognition offered a plausible framework for neuroanatomic network models of language and other cognitive functions (see Petersen et al. 1988; Mesulam 1990; Damasio 1990). In these models, a range of sensory, motor, and cognitive processes involved with speech and language were tied to many sensory, motor, or association areas (nodes), altogether constituting a larger, distributed network. A wider range of symptoms in aphasia could now be explained on the basis of nodes involved with a symptom or an aspect of the symptom. There was a shift of emphasis to parallel processing across nodes of the network in describing the neuropsychology of language. In the current era of cognitive neuroscience, the general framework of a distributed network for speech and language has been greatly expounded upon through data from functional imaging studies (see Binder 1997; Binder 2008). Carefully designed fMRI experiments can aid not only in identifying discrete language areas but also in distinguishing, for example, how activation for specific cognitive operations such as working memory is patterned in a linguistic task versus a non-linguistic task (Smith and Jonides 1998). Activation patterns can offer clues to the dynamics of neural networks involved with language. Working models for the neuroanatomic basis of language can be synthesized from fMRI data and a range of findings across cognitive neuroscience (see Binder 2008). They describe multiple loci, each addressing an aspect of language such as syntax, phonology, and semantics and altogether constituting a large network. Any component of the model can be expanded. 2

The model has also been advocated (Pinker 1994) as the neural basis of Chomsky’s (1957) Generative-Transformational model of language. In this context, its shortcomings have been described from the standpoint of evolutionary neurobiology of language (Deacon 1997; Lieberman 2002, and see Aboitiz and Garcia 1997).

60

3 Neuroinformatics for Neuropsychology

It can flexibly incorporate new research data and allow for greater precision in associating symptoms of aphasia with neuroanatomic systems. Limitations of Box-and-Arrow models: A neuropsychological working model of language that describes wide spread neuroanatomic loci may represent an achievement over the classical Broca-Wernicke model in that it draws a larger network. It may recognize that interaction and feedback can occur between the loci. Yet, however elaborate this kind of “box-and-arrow” model may be, it will always have some unfortunate limitations: It has neither a way of systematically exploring the relationships between the boxes (nodes) nor a way of dynamically testing the model. The dynamics of the network cannot easily be studied or rendered with this kind of model. The model becomes proliferated with more and more boxes-and-arrows (see Marshall 1986) Assessment of Language: The assessment of language in clinical neuropsychology has hardly been deterred by the difficulty of formulating comprehensive models of this functional domain. Certainly with regard to aphasia, assessment tools such as the Boston Aphasia Diagnostic Exam, the Western Aphasia Battery Tap are well-delineated behavioral features. They strive for well-defined symptomological categories. As neurocognitive models of speech-language evolve, some challenges for neuropsychology will be to further clarify symptoms and to match them more closely with neural systems. Role of NI: Arbib and Bota (2003), writing about neurolinguistics and neuroinformatics, stated that “ . . . we need neuroinformatics methods to develop in a fashion which more tightly integrates modeling with databases . . . to allow the more effective integration of data from neuroanatomy, neurophysiology, brain imaging . . . ” (p. 1257). Neuroinformatics is integral to the process of clarifying speech and language mechanisms in neuropsychology. Coding, sorting, and connecting the myriad of cognitive and neural variables involved with speech and language is dependent on NI methods. NI can structure databases of symptoms such that syndromes of language are indentified. NI can help model discrete neural operations in speech, which in turn can contribute to complex neural models of speech-language production. To further improve models of speech and language, neuropsychology must expand its scope. It has to borrow concepts from computational neuroscience that relate to neural coding and representations of data, neural transformations of the coded data, and feedforward and feedback mechanisms. Models are needed that take into account the dynamics of speech-language circuitry and NI methods can help to parse out perceptual and cognitive speech-language processes that result from these interactive dynamics. A related model is described in this Section (3.3.3.1) and its significance in fostering NI approaches in the neuropsychology of speech and language is emphasized. Sign and Gestural Language: The study of language in neuropsychology has not been limited to spoken language. Descriptions of the complexity of structure of sign language (Bellugi and Klima 1976; Klima and Bellugi 1979) prompted many neuropsychological reports on the neural basis of sign language and gestural communication (e.g., Hickok et al. 1996; McNeill and Pedelty 1995; Petitto and Bellugi 1988). Spatial relations and spatial processing involved with sign language syntax (Poizner et al. 1984; Bellugi et al. 1988), linguistic versus affective facial

3.3

Neuroinformatics Applications and Models for Neuropsychology

61

expressions in sign language (Corina et al. 1999), and their differential lateralization have been among the major topics in this area. More recently, neuroimaging studies have advanced finer distinctions in the neural mechanisms of sign language, e.g., for different types of facial expressions, aspects of phonological processing, and in dissociations between motor and linguistic processes (see Emmorey 2002; Emmorey et al. 2007; McCullough et al. 2005). In sign language research, the transcription and analysis of signs and gestures are fundamental steps, but this is complicated by the fact that signed languages are visual languages with no standard written form. Thus, transcription from video involves making annotations of some kind, frequently glossing signs by the nearest spoken-language equivalents, despite the fact that there is no one-to-one correspondence between a sign in, e.g., ASL, and a word in, e.g., English. Moreover, it is difficult to capture essential information about non-manual gestures – movements of the head (including periodic movements such as nods or shakes) and upper body, and facial expressions (e.g., raised or lowered eyebrows, eye aperture, mouth movements, etc.) that carry important grammatical information – which occur over phrases (frequently sequences of individual manually produced signs). Somewhat at odds with sophistication of topics in sign language research has been a long standing lack of consensus on how to annotate signs and gestures and a lack of appropriate tools to aid with the transcription (see Miller 2001; Pizzuto and Pietrandria 2001). One of the reasons for the lack of a standard method has to do with theoretical and analytical assumptions reflected in symbols and notation systems (Emmorey and Reilly 1995; Miller 2001). “One of the most difficult issues for sign language is how to represent space itself and the linguistic distinctions conveyed by space . . . ” (Emmorey and Reilly, pp. 14–15). Role of NI: It is precisely on issues such as the annotation and spatial frameworks for coding sign language that NI offers refreshingly new and efficient options. NI lends the methods of digital recording and computerized motion analysis in the capture and analysis of gestures. It also offers ways to database a range of gestures and signs and to mine this data. The third example in this Section (3.3.3.3) illustrates the case. The potential of NI in the neuropsychological study of speech and language in general is illustrated below through three examples. The first is a computer model of motor control in speech production, but one that relates to the cortical circuitry for language, and accounts for various phenomena in speech and language. The second is a web-based aphasia database designed to help clarify aphasic syndromes by using computational methods to cluster symptoms. The third is a computerized tool for digitally annotating and analyzing gestural language.

3.3.3.1 Directions Into Velocities of Articulators (DIVA) URL: http://speechlab.bu.edu/diva.php (See Guenther 1994, 1995; 2006; Guenther et al. 2006; Guenther et al. 1998; Frank Guenther, personal communication (September 25 and 28, October 5 and 6, 2008; April 17, 2009).

62

3 Neuroinformatics for Neuropsychology

DIVA is a computationally defined neural network model of speech production and acquisition (SPA). The model describes an adaptive network for acoustic, somatosensory, and motor components of speech. It takes into account numerous cortical and subcortical areas involved with speech as well as the connections between these areas. The neural network described by the model uses feedforward and feedback systems to (a) direct the production of speech, (b) gain input about acoustic properties of speech and the sensory-proprioceptive properties of oro-facial muscles, the vocal tract, and other parts of the speech apparatus, and (c) compare speech sounds produced to learned templates and make adjustments accordingly. A major aspect of these processes involves the neural coding of desired trajectories (“directions”) of the speech articulators and the subsequent signal transformation into calibrated motor commands (“velocities”). Hence the model’s name, Directions Into Velocities of Articulators. DIVA integrates many neural principles to describe how various aspects of speech emerge as dynamics and properties of neural networks. By describing a comprehensive neural network for SPA, including considerable workings of the nodes and connections of the network, DIVA is able to account for a range of phenomena relating to development of speech and speech production/perception. It is a highly unified model and therefore, not surprisingly, the model is at the forefront in contemporary cognitive neuroscience of speech and language. How does a model of SPA fit in with a discussion of NI in neuropsychology? What is the relevance of DIVA to NI in neuropsychology? The examples of NI applications for neuropsychology discussed so far (3.3.1.1–3.3.1.4) can be characterized as computer systems with informatics functionality. In this sense, the inclusion of DIVA may appear to be somewhat at odds with this NI-neuropsychology discussion – it is a theoretical model of SPA. On the other hand, the modeling routines used to test DIVA and the computational framework that is so integral to DIVA are part and parcel of NI. The introduction to Section 3.3.2.1 mentioned the need for NI modeling as a crucial method for understanding the dynamics of widely distributed neural system for language – a perfectly valid reason for including DIVA as one of the examples in this section (a brief introduction to computer modeling is covered in Section Section 2.4). A reason, perhaps, even more pertinent to this broader discussion is that DIVA is a type of model that enables and promotes NI approaches to speech and language in neuropsychology. Criteria for neuropsychology-specific NI systems suggested in Section 3.2 describe that “the complexity of an NI system should, as far as possible, be tuned to the complexity of the neuropsychological phenomenon at which it is aimed . . . [and] using NI frameworks, attempts should be made where possible to reformulate a neuropsychological research issue so as to generate a more detailed or quantified rendering of the issue.” While not purposefully cast as a NI system, DIVA fits these criteria very well. DIVA is mainly concerned with speech production but its sophisticated theoretical and computational framework (a) provides a powerful example of the kind of models that are needed for language in neuropsychology and (b) demonstrates how the interface for NI approaches in the study of SPA in neuropsychology can be carried

3.3

Neuroinformatics Applications and Models for Neuropsychology

63

out. It is in this context that attention is given here to DIVA (and this relevance elaborated later in this subsection). DIVA is both a neural and computational model of SPA. It is neural in that all its components are tied to centers of the cortex or cerebellum – it posits a broad neural network that is informed greatly by neurobiological and neurophysiological data. It is computational in that (a) many aspects of the model can be simulated in computer modeling, which includes the learnt control of an artificial voice tract (synthesizer) and (b) data from other avenues of SPA research, especially fMRI research, can be modeled in the DIVA framework for further interpretation. Detailed descriptions of DIVA can be found in the above references.3 A synopsis is provided below. Synopsis of DIVA: The DIVA model involves four major classes of information or reference frames: (a) an acoustic frame that deals with how sounds are coded as auditory maps; (b) a phonetic frame constituted by sets of speech sounds learnt through the neural network; (c) a somatosensory (orosensory) frame that receives sensory/proprioceptive information from the vocal tract to monitor the sound being produced, and (d) a motor (articulatory) frame that controls the speech musculature. DIVA also specifies the nature of the relationships or mappings between the four frames as well as feedforward and feedback systems. Mapping refers to the transformation of a neural representation that occurs between one node of the neural network and another. Figure 3.4 is a schematic of the DIVA model. The neural systems underlying the framework can be grouped as three interacting subsystems involved with the control of speech production. These are (1) a feedforward motor system and a feedback system that involves (2) a subsystem for auditory feedback and (3) a subsystem for somatosensory feedback. A region in the left ventral premotor cortex, within Broadmann’s areas 6 and 44, is hypothesized to be a starting point in speech sound production. This area contains speech sound maps, i.e., higher-level motor representations of speech sounds. A speech sound refers to any single speech sound unit such as a phoneme or syllable. When a speech sound map is activated, the three subsystems are involved in the relay of afferent signals to the motor cortex to drive and control the production of speech: The Feedforward Control Subsystem: The feedforward system is involved in the actual, primary production of a speech sound. Feedforward signals (A) from the premotor speech area have two components, one going directly to areas of the primary motor cortex and the other going first to the cerebellum and then to motor cortex speech areas. Target regions are all areas of the motor cortex involved with speech sounds (e.g., areas controlling tongue, lips, mandible). A region may code the position and velocity of a part of the speech apparatus in directing a motor movement.

3 DIVA

has been described with immense mathematical and neuroanatomical substantiation and is best appreciated when viewed against its mathematical and theoretical backing. For these details, see especially Guenther (1995), Guenther et al. (1998), and Guenther et al. (2006).

64

3 Neuroinformatics for Neuropsychology

Fig. 3.4 Schematic of neural processing stages in speech production and acquisition as described by the DIVA Model (From, http://speechlab.bu.edu/diva.php; adapted with permission, Frank Guenther)

The likely cerebellar regions are the anterior parvermal and superior lateral areas as target input areas and the medial subcortical cerebellum as the output area. The Feedback Control Subsystem: This subsystem involves auditory and somatosensory feedback systems. A set of premotor signals (B) generated by a speech sound map targets the auditory area/superior temporal cortex. The pathway of this signal encodes the spatiotemporal auditory properties of the speech sound. When the auditory area receives feedback (C) via subcortical nuclei (the sound of the produced speech), a comparison is made between this sound and the expected sound. Two auditory maps are involved, a state map and an error map. In neural terms, coordinates of activation in the auditory area (state map) are compared with the expected coordinates of activation based on previous learning. If a discrepancy occurs between expected and activated coordinates, an error map is generated. A corrective signal (D) is then sent to the motor cortex where required motor adjustments are made. The hypothesized loci of state map cells are the medial aspect of Heschl’s gyrus (primary auditory) and the anterior planum temporale (association auditory). Auditory error map cells are thought to lie at the temporoparietal juncture in the left Sylvian fissure or in the lateral aspect of the posterosuperior temporal gyrus.

3.3

Neuroinformatics Applications and Models for Neuropsychology

65

Another set of premotor signals (E) generated by a speech sound map targets the speech areas of the somatosensory cortex. This pathway encodes the spatiotemporal somatosensory properties of the sound. The somatosensory areas also receive tactile and proprioceptive feedback signals (F) from the speech articulators during production. As with the auditory feedback system, two maps are involved. The occurring somatosensory activation pattern is compared with the expected pattern (state map) coded through previous productions. Discrepancies are coded in an error map. It in turn sends corrective signals (G) to the motor cortex. Somatosensory state maps are hypothesized to lie in the inferior postcentral regions for the speech; lips, tongue, larynx, etc. Error maps are thought to lie along the supramarginal gyrus, in the inferoposterior parietal cortex. Feedback and Feedforword Process in Speech Acquisition: In the DIVA model, attempts at production of a speech sound can begin after the acoustic properties of the sound have been coded, i.e., auditory coding in neural terms has been established. The first attempt at producing a particular sound will typically be crude. The feedforward neural circuitry will be operating for the first time and hence will be greatly reliant on the auditory feedback system for feedback of the sound errors. With subsequent attempts, feedback is more smoothly integrated, the feedforward system becomes smoothly conditioned, now producing fewer errors and becoming less dependent on auditory feedback. The same principles would apply to the somatosensory aspects – reliance on proprioceptive input is lessened with practice as the sound’s proprioceptive “signature” gets well coded and the burden of the somatosensory error to state check is minimized. With the above described representational and sensorimotor processes, DIVA provides an account of how individual speech sounds such as phonemes are strung together in speech acquisition and how neural signals are transformed in producing fluently sequenced movements of the speech articulators. The framework also enables DIVA to account for a variety of speech-language phenomena or provide a unique investigative avenue into these phenomena. Three examples are very briefly conveyed here: Infant Babbling: According to the model, it is during the babbling phase that critical aspects of sensorimotor parameters of speech sounds are tuned. Feedforward and feedback signals involved with babbling allow the infant to mark the canonical range or signature of sensory-acoustic, motor, and sensorimotor states tied to particular speech sounds. This example very well conveys the self-organizing system described by DIVA where learning is based entirely on sensory and motor neural information. Phonetic Categories: The neural dynamics of acoustic representational (neural) space for phonemic categories are richly detailed in the DIVA model. Phonemic encoding occurs through a consolidation of neural representational matrices – as a sound is learnt and successfully produced, the neural cluster encoding the sound in the auditory/association auditory cortex is tuned to rely on fewer neurons with a tighter relationship between them. Prototypical sounds learnt in one’s native language are, for example, coded in this manner. Later learning of a non-prototypical

66

3 Neuroinformatics for Neuropsychology

sound (a different language) becomes difficult because of the neural re-organization required. Neural Decoding of Speech and Neural Prosthetics for Speech: Informed by the theoretical framework of DIVA, a neural prosthetic/speech-decoding device has been experimentally tested in a patient with “locked-in syndrome” (near complete paralysis following a trauma-induced stroke).4 In this case, speech paralysis was targeted by implanting an electrode in the patient’s premotor speech area (see DIVA website). The patient’s attempt to produce sequences of vowel sounds presented to him on a computer screen resulted in premotor neural activity which was captured by the electrodes and fed to a computer. The signals were analyzed using DIVA’s association of specific speech premotor signals with specific formants/characteristic vowel frequencies. These computer-enhanced signals were then fed to a synthesizer, which produced the corresponding vowel sounds. Many trials have been conducted and at present, the system has seventy-five percent accuracy in producing the patient’s intended vowels sounds. Needless to say, the results of this work have reverberated through the field of neuroscience. DIVA distills the neural processes of SPA, describing crucial neural computations and their spatiotemporal sequence, spread across a broad network. DIVA “speaks” of sensorimotor transformations, representational vectors (of sounds), and orosensory maps. All components of the model are linked to neuroanatomic loci. The model describes an interplay between speech perception and speech production and how a particular neural system can be differentially involved in speech perception and production. Numerous phenomena and numerous sets of data in speech-language research can be interpreted using the model. Why models like DIVA serve NI approaches to speech/language in neuropsychology: As indicated, DIVA’s relevance to NI in neuropsychology can well be viewed in terms of its modeling paradigm or the above described brain-computer interface to decode neural signals of speech. DIVA simulates many of its components and mechanisms with computer models. These include, for example, the neural representation of speech sounds, the production and combination of learnt sounds, and neural corrective measures following errors in production. The greater relevance of DIVA to NI, however, has to do with the conceptual framework it provides for the study of SPA. The critical mass of research data on neurocognitive aspects of SPA desperately calls for comprehensive, biologically plausible models against which the data can be interpreted. NI approaches bring another dimension of complexity. If NI methods were to be optimally developed or applied in regard to the data, an ideal condition would be the availability of large scale, comprehensive theoretical models. This will better contextualize topics for NI approaches. At the same time, it will promote convergence in the data emerging

4

The case of Erik Ramsey and the DIVA-inspired electrode signal decoding of his premotor speech neural signals has already received much attention in the popular media (for example, New Scientist, July 9, 2008; CNN, December 14, 2007). A major collaborator on the project has been Dr. Philip Kennedy at Neural Signals, Inc. in Atlanta, Georgia.

3.3

Neuroinformatics Applications and Models for Neuropsychology

67

if multiple NI initiatives tackle different problems in speech/language research. (These points are also conveyed in Section 3.3.2.1 with the example of the Computational Visual Field for Neglect – how the tool is tuned to properties of coordinate mapping and compression described by the posterior parietal/representational model of neglect.) As with many neuropsychological domains, more and more research in SPA is being done with functional imaging. With numerous fMRI investigations showing multiple brain regions of activation in speech production tasks, DIVA “provides a conceptual and computational framework for interpreting many of these datasets” (Guenther et al. 2006, p. 280). By laying out a detailed neuroanatomic network and the nature of activity and information flow within the network, DIVA is able predict and simulate fMRI activation patterns for a specific speech production task. It does this by utilizing the SPM2 software package to produce simulated fMRI activity based on a theoretically specified neural activity. This strategy is useful in generating hypotheses and guiding fMRI studies tied to SPA.

3.3.3.2 The Aphasia Database (see Axer et al. 2000a,b,c; Jantzen et al. 1999a,b; Hubertus Axer, personal communication, August 1, 11, 15 and 22, 2008; Jan Jantzen, personal communication, August 1, 11, 8 and 17, 2008) URL: http://fuzzy.iau.dtu.dk/aphasia.nsf/htmlmedia/index.html The Aphasia Database (AD) is a database of diagnostic and symptomological data drawn from aphasia patients. It serves primarily as a research tool to help clarify different aphasic syndromes that overlap in symptoms. It also serves as a model for databases and data mining in aphasia research. The AD is a collaborative project between the Department of Anatomy at Rheinisch-Westphalian Technical University (RWTH) in Aachen, Germany, and the Department of Automation at the Technical University of Denmark in Lyngby. The project has been sponsored by the European Network for Fuzzy Logic and Uncertainty Modeling in Information Technology (ERUDIT) and the European Network on Intelligent Technologies for Smart Adaptive Systems (EUNITE), organizations supported by the European Commission. ERUDIT and EUNITE ran from 1995 to 2003 with a mission of promoting computational technology development and infrastructure in education and industry. Rationale and Goals of the Aphasia Database: The AD was created in order to address problems arising from the classical Broca-Wernicke taxonomy of aphasia (see discussion earlier in this section). Syndromes of aphasia defined around this taxonomy overlap in terms of features. This often results in (a) unclear boundaries between aphasic syndromes, (b) lack of clarity about when and how symptoms are shared across aphasic syndromes, and (c) no definite way of relating neuropathology in aphasia to normal brain language processes. The AD sought to take a set of uniformly classified cognitive and neuroanatomic aphasic features and then place the data in a database. Computational methods and neural network models then

68

3 Neuroinformatics for Neuropsychology

applied to the data could potentially yield fine tuned relationships between the data, advancing significantly over conventional methods of diagnosis. In addition, any success with a database and computational techniques relating to classification of such a complex set of syndromes (aphasia) would provide an informatics model for classification of problems in general medicine. Structure and Contents of the AD: The AD system uses a server-client set up. Data for the AD were originally collected in the format of a Microsoft Excel file. The data were then exported to a Notes/Domino database residing on a Lotus Domino server5 (in Denmark). Pages of the server website are accessed in database views in any browser. Data collection for the AD began in 1986 in the Department of Neurology at the RWTH Aachen. The database currently has 265 records. Two main types of data are included in the AD, aphasia test profiles and neuroanatomic lesion data. Language profiles for the AD are generated by the Aachen Aphasia Test (AAT), a test battery commonly used in German speaking countries. The AAT contains six subtests: spontaneous speech, token test, repetition, written language, confrontation naming, and comprehension, each designed to tap a particular symptom. All subtests have multiple subcategorical measures, for example, the repetition subtest, involves tests of single phonemes, monosyllabic nouns, local and foreign words, compound words, and sentences. Database entries are the numerical scores of the subtests. The neuroanatomic data in the AD are lesion profiles obtained through CT scans. Neuroanatomic data are included for only a subset of 146 patients in which the lesion sizes were found to be relatively stable. The AD holds a lesion profile as a set of 3D coordinates of the border lesion. These coordinates are derived from the Aachen Voxel Model (see Section 2.2) that can be used to describe the locus of a lesion using standardized 3-dimensional model of a normal brain. For the 265 records in the database, many types of (aphasia) diagnosis are included but the most frequently occurring types are Broca’s, Wernicke’s, Global, and Anomic aphasia. Neural Modeling and Diagnoses with the AD: The AD and its related modeling system are geared to standardization and consistency in relating aphasic symptoms to aphasic syndromes. Tied to the AD is a neural network modeling program. The point of this artificial neural network is the association of symptoms with diagnoses, computationally. The neural network is programmed to examine the various symptoms, i.e., test scores in the AD. The multilayered network selects and classifies particular features of the symptoms and outputs diagnoses with a specified degree of certainty of Broca’s, Wernicke’s, Global, or Anomic aphasia. Figure 3.5 represents

5

The Notes database is a non-relational database. A data object or “note” may consist of any number or type of data fields, and the database can hold any number of these notes. Because the database header and note header contain certain kinds of information, a user can search multiple Notes databases at the same time. This structure is advantageous for the efficient storage of diverse data types in the same database. Lotus Domino is a very flexible server application that can be used with a variety of Lotus Notes applications.

3.3

Neuroinformatics Applications and Models for Neuropsychology

Examples of INPUT (test scores from subtests/items of the Aachen Aphasia Battery)

69 OUTPUT (diagnosis of an aphasia syndrome with a certain degree of certainty)

communicative behavior articulation automatized language semantic structure phonological structure syntactic structure

Artificial Neural Network (intelligently classifies data based on particular modeling parameters)

Anomic Aphasia Broca’s Aphasia Global Aphasia Wernicke’s Aphasia

repetition reading aloud

Fig. 3.5 Schematic of neural network diagnosis by classification of AD Test Scores. (Adapted from Axer et al. 2000a,c; and Jantzen 1999a; printed with permission, Elsevier)

this process. The key aspect of the neural network is the classifier. It is capable of classifying symptoms (score patterns) even if it has not previously encountered these symptoms. Using a back propagation learning method, the network was trained on a test set drawn from the database. For example, high scores in measures a, b, and c combined with low scores in measures x and y and an average score in measure z, may amount to the profile of a certain type of aphasia. When the network encounters this pattern consistently, it reinforces its association between the particular symptom profile and the diagnosis. The classifier can be adjusted in terms of its complexity and the number of input data. In experimental trials with AD data, one classifier produced correct diagnosis in 92.4% of test cases (Jantzen 1999). The AD’s modeling component can also use another approach in searching for similarities in the language assessment and neuroanatomic data. The nearestneighbor approach measures an abstract (standardized) distance between an object in the database and all other objects across all records or selected records. The objects most similar to a given object can thus be found. This method can, for example, be applied to the lesion data to gauge the average anterior border (coordinates) among all cases of Broca’s aphasia. The AD stands as an embodiment of the potential that neuroinformatics holds for research in language and aphasia. With a relatively small but carefully categorized data set, it has demonstrated the advantages of digitizing aphasia test data and the utility of neural network models in analyzing the data. The AD initiative relates to the need for standardizing aphasia test data, the need to objectively diagnose a syndrome using a certain data set, and the need to develop ways to accurately associate the data with neuroanatomic loci. In this regard, AD, a neuroinformatics system, is an outstanding development in aphasia research and provides a strong foundation for other NI initiatives in aphasia.

70

3 Neuroinformatics for Neuropsychology

3.3.3.3 SignStream: An Informatics Tool for the Digital Analysis of Visual/Gestural Language URL: http://www.bu.edu/asllrp/SignStream/ (See McLaughlin et al. 2000; Neidle 2001; 2002a,b; Neidle and McLaughlin 1998; Neidle et al. 2001; Dreuw et al. 2008; Carol Neidle (personal communication, October 1 and 2, November 21, 22, and 24, December 27 and 28; 2008) R SignStream is an informatics tool for the annotation and analysis of visual and gestural communication. It is best known as a tool for transcription and analysis of signed languages but it can be applied to any form of gestural communication. The tool offers a comprehensive multimedia platform tied to a database. Gestural communication is captured through digital video recordings, which can then be richly annotated. This format, combined with the use of algorithms to search, combine and manipulate the data, enables sophisticated analysis of communicative gestures. R has been led by Carol Neidle and a research team of the AmeriSignStream can Sign Language Linguistic Research Project at Boston University and various components have been developed in collaboration with researchers from Rutgers University and Dartmouth College. The project has been sponsored by the National R was introduced in 1997 and has since been Science Foundation. SignStream widely distributed. It is made available to the research community on a non-profit basis. As is the case with most NI tools, its development is ongoing, shaped by research needs and technological advances. Version 3.0, a Java reimplemenR with powerful new features, is expected to be released tation of SignStream in 2009. R as a Response to Limitations in Sign Language Research: A major SignStream limiting factor in the study of sign/gestural language has been the lack of availability of a large corpus of language gestures in annotated form. A large corpus would be crucial to research in that it would provide shared, reusable data from which relationships and analyses could be conveniently drawn. There were a few key obstacles to the development of such a corpus: the process of transcribing analogue video data on manual signing has always been a tedious one. Coding non-manual expressions (e.g., a frown) in these recordings has been even more problematic. Serial searches through videotapes to locate a specific gesture from a participant would inevitably be a time consuming and cumbersome process. Further, with written representations of the video data, came disagreement about the interpretations of the data. Altogether, these problems amounted to the lack of a shared objective and efficient coding system for the rapid capture, retrieval, and analysis of signs and gestures, as well as the lack of a corpus of gestural language on which to test and develop this system, which, in turn, would expand the corpus. The lack of such a system also meant that there would be no effective method of extracting complex data patterns contained in the kinematics and trajectories of signed gestures, nor would there be a means for easy comparison of these patterns within and across

3.3

Neuroinformatics Applications and Models for Neuropsychology

71

R individuals. SignStream , as a digital, computational, analytic tool, was developed in direct response to the problem.6 R The General Structure and Functionality of SignStream : At the National Center for Sign Language and Gesture Resources (NCSLGR) at Boston University, a subject’s gestural communication is captured through digital video recordings. Multiple synchronized digital cameras simultaneously record the subject’s gestures from different angles. Similar hardware arrangements can be set up at different research centers with relative ease. The digital video recordings are made publicly R requires formats compatiavailable in multiple movie file formats; SignStream TM ble with QuickTime . Segments are marked that correspond to different gestural utterances. These individual “units” of utterances are the data units that populate R database – which is therefore a collection of utterances. Since an the SignStream utterance is usually captured from various angles, more than one video file may be associated with the utterance. The user first annotates a segment of video by entering data about the utterance. This produces a corresponding transcription for that utterance. The SignStream program provides numerous data fields through which an utterance may be coded. The three main types of data fields are non-manual fields, gloss fields, and text fields. Types of non-manual gestures (e.g., raised eye eyebrows, rapid headshake, or squinted eyes) are coded through non-manual fields. For each event, annotation includes identification of start and end frame, in addition, potentially, to onsets, offsets, or holds of particular gestures. The ability for part of speech tagging is included and there is also an English translation field available. The new version of the program will provide additional tools for fine-grained phonological analysis (e.g., specification of hand shapes and movement types). The data fields that are provided (see above snapshot) also allow for distinctions in the intensity of specific gestures. Non-manual gestures are coded through multiple fields relating to form, position, and movement of the eyes, head, etc. Head behavior, for example, is coded through 8 fields that amount to a rich 3-dimensional coding frame. They address head position at the starting (base) point and then in relation to x, y, and z coordinates. R offers some intuitive ways for data entry such as manual typing SignStream into the fields or selections from drop-down menus. It also allows the user to create new fields that the user may wish to define – enabling the description of new types of data. Numerous other options are available for screen display parameters and for audio and video controls. R program’s screen, a video segment and its corresponding On the SignStream transcription can be viewed in separate adjacent windows at the same time. Up to four video files may be displayed for each utterance. As a video segment plays, a media alignment indicator (a vertical bar) moves from left to right in the transcription window. This provides for the temporal alignment of the linguistic features

6 There are now also other computer-based tools available for transcription of multimedia data (e.g., Elan; URL: http://www.lat-mpi.eu/tools/elan/).

72

3 Neuroinformatics for Neuropsychology

Fig. 3.6 Snapshot of SignStream video and gloss windows showing annotated ASL data. (Figure provided by Carol Neidle and printed with permission. © Boston University/National Center for Sign Language and Gesture Resources)

described in the transcription window with the visual images in the video window. Hence, the program offers dynamic linkage of the coded linguistic data with the visual data. R Database: The database links transcription data for an utterThe SignStream ance with its respective video files. A sample database accompanies the application. It contains 10 short American Sign Language (ASL) utterances. It provides the starting point for users to develop their databases. In addition, users may sign on to an online database ( http://ling.bu.edu/asllrpdata/queryPages /) from which data can be drawn for analysis. The online database therefore acts both as a distribution point R encoded data as well as a data repository serving the ongoing for SignStream effort of building large shared corpora of sign and gestural language. At present, the data in the online database is from native signers of ASL. The online database now contains over 1300 utterances gathered primarily from four subjects since data collection began in 1999. It is a constantly expanding database and the brand new web-based Database Access Interface (DAI) greatly facilitates searching through the data and downloading subsets of videos and annotations that may be of interest to particular researchers. The DAI provides an array of search options to the user, as R itself. Many types of queries rangdoes the search capability within SignStream ing from simple to complex may be formulated. Search criteria may be combined, to search, for example, a specific sign only when it co-occurs with a specified gesture

3.3

Neuroinformatics Applications and Models for Neuropsychology

73

such as a head tilt to the right. Temporal search criteria may be applied such that a gesture may be located only when it occurs with, before, or after another gesture. R allows the user to select a set of utterances A script window within SignStream and position them to be viewed in a particular order, a function that is particularly useful in teaching and research. Queries and scripts can be saved for future use. The NCSLGR shared corpus of data has achieved a size and scale such that subsets of data can be extracted to study a particular feature of gestural communication. These subsets of data can then be grouped and benchmarked to form smaller, specialized databases that relate to discrete features. Dreuw et al. (2008) have reported on three data subsets that relate to language recognition in ASL. These have been created at Rheinisch-Westphalian Technical University (RWTH) in Aachen, Germany, drawing from the central database at Boston University. One of the newly created databases at RWTH is the RWTH-Boston-400, a database of 843 sign language sentences. This is currently the largest publicly available benchmark corpus for video-based continuous sign language recognition. Computational Analysis of Gestures: A major aspect of the collaborative research involving linguists (esp. Carol Neidle) and computer scientists (esp. Stan Sclaroff at BU and Dimitris Metaxas at Rutgers) has to do with developing algorithms for the automatic analysis and recognition of gestures and motion in communication. Algorithms used in computer-based recognition/machine vision are applied in coding the forms, positions, orientations, and trajectories of the eyes, face, head, hands, upper body, etc. Because these body parts move simultaneously and often subtly in communication, it is an enormous challenge to classify their motions and their relationships with regard to each gesture. Computational algorithms enable this kind of analysis, and with such a fine-grained analysis comes the potential to sift out rich yet often-elusive phonological details contained in gestural communication. The project has given much attention to algorithms for coding head motion in view the linguistic significance of head/face movements. R is particularly remarkable. As In the context of neuroinformatics, SignStream a unified platform, it seamlessly integrates so many facets of NI – it offers a computer application geared to the analysis and databasing of gestural language. To aid the analysis, it develops computational methods for finely deconstructing this comR plex behavior. It opens itself to the research community worldwide. SignStream provides a well-founded example of neuroinformatics for neuropsychology in action.

Conclusion for Sections 3.3.3.1, 3.3.3.2 and 3.3.3.3 This subsection has described three major developments that serve NI approaches to speech-language research. Clearly, there are many ways of applying NI to neuropsychological research on speech and language. There are other NI initiatives in this area, either smaller projects or projects indirectly related to neuropsychology. The Aphasia Modeling Project (also known as

74

3 Neuroinformatics for Neuropsychology

Webfit)7 is an online tool for the analysis of aphasia picture naming and repetition data, using the two-step model of lexical access in language production (see Dell et al. 1997; Dell et al. 2007; Foygel and Dell 2000). A patient’s picture naming test data are entered into a program and are mapped to the model’s parameters that fit normal error patterns. The model can then be “lesioned” by adjusting phonological and semantic variables to produce various error patterns. There are also tools geared to linguistics and normal language. WordNet8 is an online, downloadable lexical database of English. It contains open class words (nouns, verbs, adjectives, and adverbs), which it organizes as synonym sets. Lexical, semantic, and conceptual relationships link these synonym sets such that a word search generates numerous class and contextual attributes. Such a database of the English lexicon and relationships of its elements makes for a useful tool in cognitive and computational linguistics. Latent Semantic Analysis (LSA) is a statistical method for extracting and representing semantic relationships between words in phrases that are contextually used in written documents across a body of knowledge (Landauer and Dumais 1997; Landauer et al. 1998).9 The LSA method mathematically decomposes text into a matrix, which allows for the analysis of each work or passage in relation to other words and passages. The representation of a particular body of text (e.g. text on social psychology, cardiac research, French literature) is defined LSA as a semantic space. Words within this space (i.e., corpus of texts) are mathematically coded in terms of particular vectorial dimensions and weightings based on their occurrence in relationship to other words. Using a technique akin to factor analysis, a particular usage of a word is weighted in terms of the average meaning derived from its usage throughout the semantic space. The meaning of a passage of text is coded in relation to the average value/meaning of all its constituent words. LSA therefore extracts relationships written discourse using a statistical technique as opposed to using a dictionary, knowledge base, or artificial intelligence program. Its frame of reference is generated by the semantic space in question and so offers a unique way of analyzing patterns in the discourse of the texts. LSA is of special relevance to neuropsychological assessment of language in that its methods can be adapted in analyzing word usage and discourse in aphasia. NI brings promising new avenues to the neuropsychology of speech and language. The sophistication in the methods and tools of NI are invaluable in matching up to the cognitive and neural complexity of speech and language.

7 Webfit is a project of the Language Production Laboratory at the University of Illinois UrbanaChampaign. URL: http://langprod.cogsci.uiuc.edu/cgi-bin/webfit.cgi 8 WordNet is based at the Cognitive Science Laboratory at Princeton University. URL: http://wordnet.princeton.edu WordNet: An Eletronic Lexical Database (Fellbaum 1998) is a written guide to the online tool. 9 The Latent Semantic Analysis project is based at the University of Colorado at Boulder URL: http://lsa.colorado.edu / See also the Handbook of Latent Semantic Analysis (Landauer et al. 2007)

3.3

Neuroinformatics Applications and Models for Neuropsychology

75

3.3.4 Phenomics and Neuropsychology Just as the term human genome describes the full compliment of human genes, the term human phenome refers to the full complement of human phenotypes (Mahner and Kary 1997). Phenotypes are the outward expression of genes and range from the physical (e.g., molecules, proteins, cells, organs, morphology) to the cognitivebehavioral (e.g., perception, impulses/drives, high-level cognition). Genomics and proteomics were briefly mentioned in Chapter 1 – as transdisciplines that are dependent on bioinformatics. Similarly, a new discipline, phenomics, has arisen. Phenomics refers to the systematic study of phenotypes and it is a transdiscipline in which informatics methods are integral. The mapping of the human genome, while hugely significant, is but one step in understanding the workings of genes, which involve pathways of gene expression, and the outward result of that expression, phenotypes. The discipline of phenomics is aimed at specifying phenotypes and it aims to draw the data in such ways that enable their relationships with genotypes to be studied (Bilder et al. b, in press). “Phenomics approaches require collecting phenotypic information in any given individual, at a series of different levels of resolution (molecules, cells, tissues, and whole organisms) and then determining how these features can profitably be studied together” (Freimer and Sabatti 2003, p. 16). This above description was laid out in the authors’ detailed proposal for a systematic databasing of phenotypic information. The need to specify phenotypes and reap maximum advantage of gene mapping studies, they argued, gives primary impetus for such a project. They named the proposed project The Human Phenome Project (HPP). As is typical in bioinformatics, the project would necessarily involve a diverse group of biologists, behavioral scientists, informatics, and modeling specialists. The expressed need for the HPP speaks of the importance of understanding phenotypic data in the larger context of biology and neurobehavior. Cognitive and neuropsychiatric phenomics define an area of the discipline concerned with cognitive and behavioral features in normal and clinical populations. It is the “systematic study of neural systems phenotypes on a genome-wide level, to bridge the gap between complex behavioral syndromes and their biological mechanisms” (Bilder 2007b). It focuses on neurobehavioral phenotypes that can more accurately reflect a disorder and that can be systematically linked to other levels of analysis in biology and medicine. The concept of endophenotypes: An important concept in phenomics is that of the “endophenotype.” Endophenotypes refer to intermediate traits lying between the gene level and the disease/syndrome level, that can be precisely defined, quantified, and which may be causally linked to genotypes (Bearden and Freimer 2006; Flint and Mufanò 2006). Because of their intermediate position between genes and syndromes, endophenotypes are viewed as more amenable to research and as offering a more direct path in linking genotypes to phenotypes. The endophenotype concept is central in phenomics because endophenotypes are considered to be features defined with relative precision and therefore more accurately indicative of neuropsychological profiles or complex neuropsychiatric syndromes (see Gottesman and Gould

76

3 Neuroinformatics for Neuropsychology

2003). Because they are associated with causes rather than effects of a condition, they are viewed as more reliable diagnostic indicators of syndromes (Zobel and Maier 2004). In contrast, the traditional methods of neuropsychiatric classification and diagnosis in psychiatry, as embodied by the DSM IV and its predecessors, are based on general consensus around observed clinical features and are theoretical. (Clinicians need only be reminded of the vigorous discussion and debates regarding classification of mental disorders that played out in medical journals over the last three decades.10 ) Using neurocognitive endophenotypes as markers of neuropsychiatric conditions may bring a more scientific, objective dimension to neuropsychiatric classification and diagnosis. The role of neuropsychology: Neuropsychology has a crucial role in phenomics research. Neuropsychological constructs and tools are relied upon to describe cognitive and neuropsychiatric endophenotypes such as response inhibition, attentional control, and verbal working memory. Consider how performance on the Wisconsin Card Sorting Test is viewed as an index of executive function/set switching and prefrontal functioning; neuropsychological phenotypes are about such processes but with a need for more precision in their definition (see Bearden and Freimer 2006; Kremen et al. 2007). In the context of phenomics research, neuropsychology also finds itself operating a sophisticated web of neuroinformatics. Neuropsychology is challenged to describe cognitive-behavioral phenotypes in highly refined ways and in a manner that can be interfaced with other levels in inquiry (see Bilder 2007b; Bilder et al. a, in press). To accomplish this, neuropsychology has to be informatics compatible. How neuropsychology is integrated in current phenomics projects provides a broad and compelling illustration of what it means to have the discipline of neuropsychology geared for NI, why this should be the case, and ways to go about the process (Section 3.2 stressed the importance of defining NI systems that can be considered unique to neuropsychology. It also indicated that NI systems for neuropsychology should be designed with the view of potentially integrating neuropsychological data with data from other disciplines. The examples of phenomics research described in this section, especially the second example, demonstrate particularly well the latter aspect.) Two examples of phenomics research are described below. The first concerns a body of work that seeks relationships between craniofacial dysmorphology, neuroanatomy, and neurosychological variables in neurodevelopmental and neuropsychiatric disorders, using informatics methods. The second describes a large-scale, NIH-funded research consortium for neuropsychiatric phenomics, through which numerous novel and powerful NI developments in neuropsychology are being pioneered. It is a unique example of the kind of informatics infrastructure that serves knowledge building in neuropsychology.

10

See for example, Spitzer et al. (1975); Spitzer et al. (1980); Williams & Spitzer (1982); Klerman et al. (1984); Sadler et al. (1994).

3.3

Neuroinformatics Applications and Models for Neuropsychology

77

3.3.4.1 Neuroinformatics and Neuropsychology in Studies of Craniofacial and Brain Dysmorphology (See Matthysse et al. 1992; Deutsch et al. 2000; Deutsch and Joseph 2003; Farkas and Deutsch 1996; Lainhart et al. 2006; Curtis Deutsch, personal communication, April 4 and 13, 2007; January 5, 6, and 7, 2009) URLs: http://www.umassmed.edu/shriver/research/psychological/projects/ cbrsds.aspx http://www.mclean.harvard.edu/research/mrc/psychlab.php There has been ongoing collaboration between researchers from the University of Massachusetts Medical School, the Kennedy Shriver Center (Massachusetts), the Harvard Center for Genetics and Genomics, McLean Hospital (Massachusetts), and Boston University Medical School, relating to neurodevelopmental abnormalities in autism, ADHD, schizophrenia, and bipolar disorder (see above references). One line of research stemming from the collaboration examines brain and craniofacial developmental abnormalities of form/structure and seeks possible relationships between these two forms of dymorphology (see for example, Deutsch et al. 2000). The work is funded by grants from the NIH. Specifically, the work is concerned with (a) how certain forms of craniofacial dysmorphology may signal abnormalities in circumscribed brain regions; (b) how the same pathogenic factors involved in these abnormalities may be common to different disorders (e.g., schizophrenia and bipolar disorders); and (c) how neuropsychological features may be linked to certain dysmorphologic forms. The work is grounded in a principle of developmental genetics/neurobiology that has long been established (see Sadler 2004): In embryogenesis and fetal development, the formation of different bodily regions and organs takes root from designated regions of the embryonic cell mass. Specific primordial embryonic cells are fated to proliferate and differentiate into specific parts of the body and such relationships are described by embryological fate maps. The brain and the face arise from a shared embryonic region, a prominence called the anterior neuropore. Hence, if there is a problem with the genetic program controlling the cells of this region or if the cell cluster is affected by a toxin during critical periods in embryogenesis, abnormalities can be manifested in both the brain and the head/face. Deutsch et al. (2000) applied a fate map in studying forms of brain and craniofacial dysmorphology in schizophrenia: They showed that dysmorphology/asymmetry of the frontonasal-maxillary juncture, common in patients with schizophrenia, could reliably predict the occurrence of midline (longitudinal cerebral fissure) deviation in the region of the mesencephalic-diencephalic juncture. Role of NI and Neuropsychology: Deutsch and colleagues have been developing quantitative methods to detect and measure craniofacial dysmorphology. Their approach is twofold. First, they utilize reliable, objective anthropometric methods to quantify facial morphology. Second, they study combination of anomalies derived from embryologic principles, for instance, combinations of morphological anomalies (by types or regions). To facilitate their research, the team has been building

78

3 Neuroinformatics for Neuropsychology

a large, normative database (Farkas and Deutsch). The database permits quantification of craniofacial dysmorphology, in, for example, autism and schizophrenia. Phase 1 of the database development is complete. It involved the creation of a simple PC-based tabled database. The database consists of numerous anthropometric measurements, each entered in separate column and subjects are entered by rows. Data continue to be added as studies progress. Phase 1 also involved the overall plan for the incorporation of a neuropsychological data component. Having data on both physical and neuropsychological features (a) will better enable the search for possible links between different classes of phenotypes in autism and (b) could possibly point to etiological subtypes in autism. Deutsch and Joseph (2003) demonstrated this kind of convergence in a study that linked uneven development of non-verbal/spatial skills in a subgroup of autism patients who also happened to manifest unusually large head circumferences (macrocephaly being a frequently seen feature in autism). In Phase 2, the researchers are now developing database functions that introduce descriptive statistics and data visualization functions. The aim is to work towards a larger normative database that will be referenced against a craniofacial atlas. The studies of craniofacial and brain dysmorphology described above help to map out complex etiologies in neurocognitive disorders by examining convergences of certain physical and behavioral traits. Matthysse et al. (1992), in describing a few ways to clarify the complex mix of genetic factors and phenotypes in neuropsychiatric conditions, stated that “[i]n the long run, the most successful method is likely to be direct refinement of non-Mendelian behavioral and physiological traits into more fundamental components” (p. 461). The behavioral component, to do with defining cognitive-behavioral phenotypes, is essentially neuropsychological. In such an intensely multivariate research context, NI offers the enabling framework for the interface between neuropsychological and physiological data.

3.3.4.2 A Consortium for Neuropsychiatric Phenomics (See Bilder 2007; 2008a; 2008b; 2008c; Bilder et al. in press a,b,c; Freimer and Sabatti 2003; Sabb et al. 2008; Krensky 2008; Robert Bilder, personal communication, January 6, 16; February 5, 6; April 15; June 8, 9, 2009) URL: http://phenomics.ucla.edu In the proposal for the HPP mentioned earlier, the authors (Freimer and Sabatti) also laid out a possible scientific and organizational structure for the project. They described that the project will need to be a large-scale coordinated effort (akin to the HGB) that would bring together experts from a variety of disciplines. It would have to involve numerous databases, holding all levels of phenomics data, and the data will need to be objectively defined. Quite crucially, it will need to develop and lay down methods for standardizing the description of phenotypic data. These methods will not only enable researchers to share a common vocabulary for knowledge development but will also serve the proposed need to relate

3.3

Neuroinformatics Applications and Models for Neuropsychology

79

different levels of phenotypic data (cellular, neuronal, anatomic, etc.). As was the case with the HGP, the proposed HPP will require new technologies to automate complex processes of data analysis, comparison, and modeling. The proposal also emphasized that novel collaborations and research methods will be essential to the project. The Consortium for Neuropsychiatric Phenomics (CNP) is a large, new consortium that is focused on neurobehavioral phenotypes. It can be viewed as the first large-scale phenomics project/component to be embarked upon that fits in with the proposed HPP. Thus all that is described in the above paragraph is consistent with the CNP. The CNP is based at the University of California at Los Angeles and involves 52 researchers and clinicians from across the institution. Earlier work at the UCLA Center for Cognitive Phenomics laid the platform for the CNP. The CNP is funded by 5-year, a 22 million dollar grant from the National Institutes of Health. Among its strategic initiatives laid down in its 2006 Reform Act (see Krensky 2008), the NIH established a fund to support transdisciplinary research. It set forth a series of “Roadmap Initiatives” to fund highly integrative, transformative biomedical and behavioral research. It stressed research opportunities in emerging interdisciplinary junctures in medical science and the potential therein to solve complex medical/health problems. It also stressed the role of bioinformatics as new pathways to discovery. Nine grants were awarded to research teams around the US. One was a grant to establish the CNP, which would lead numerous initiatives in behavioral phenomics. Of the projects funded by the NIH Roadmap Initiatives program, CNP is the only one that has a distinct neuropsychological component. Robert Bilder, a clinical neuropsychologist by training, is the primary investigator and director of CNP. It should be noteworthy to neuropsychologists, that this groundbreaking, informaticsinfused initiative is led by a neuropsychologist. (At the February 2008 meeting of the International Neuropsychological Society, in Waikoloa, Hawaii, a continuing education course titled “Phenomics for Neuropsychology: Transdisciplinary Strategies for Research on Brain-Behavior Relations in the Post-Genomic Era” was given by Robert Bilder.11 ) The Mission and Strategy of CNP: Simply stated, the CNP seeks to identity neuropsychiatric phenotypes tied to perception, cognition, and emotion. This is motivated by the complete lack of a knowledge base on cognitive/neuropsychiatric phenotypes – “there is no coherent knowledge base representing cognitive concepts and cognitive measurements” (Bilder 2008c). Building a knowledge base of behavioral phenotypes can potentially serve to unravel the mechanisms of neuropsychiatric syndromes and aid in more accurate classification of clinical disorders (see discussion on endophenotyopes above): It is often the case with neuropsychiatric disorders that a set of endophenotypes may manifest in more than one disorder, i.e., similar behavioral features mediated by common neural mechanisms may manifest

11 The

references cited, Bilder 2008b,c, refer to Robert Bilder’s 97 slides presented at the conference and the audio recording of the lecture, respectively.

80

3 Neuroinformatics for Neuropsychology

in two or more different disorders. This can blur both the scientific understanding of a disorder and its diagnosis. Of greater value are certain “signature” endophenotypes or combinations of endophenotypes that may be associated with a neuropsychiatric condition. Also of interest is how shared endophenotypes may vary in subtle ways across disorders. Understanding the behavioral/neural endophenotypes specific to a disorder leads to the possibility that disorders can be accurately diagnosed by sampling of target behavioral phenotypes. The idea comes from genome-wide associations studies (GWAS), where a rapid scanning of genetic markers as opposed to a scan of the entire genome can reliably signal risk or presence of a condition. If one attempts to link the rather precise genomics data to neuropsychiatric data, one is forced to use conventional psychiatric clusters (syndromes) as the neuropsychiatric data set. There is wide discordance between the two data types – symptom clusters are not compatible with a scientific structure that seeks to mechanistically link syndromes to genes. This seriously inhibits linkage between genotypic and phenotypic data in trying to understand neuropsychiatric mechanisms. Hence at one level (“horizontal”), the CNP seeks a mapping of neuropsychiatric phenotypes to better characterize neuropsychiatric disorders and link them to underlying genotypes. The CNP also seeks to describe data such that they can be related to levels of analysis above and below the level of cognitive phenotypes. Refer to Fig. 3.7a. The levels above are symptoms and syndromes (and various clinicians and clinical researchers are involved at these levels). The levels below cognitive phenotypes are those of neural systems, cellular systems, proteins, and genes (each involving different specialists in biology/neuroscience). These levels of analysis define the “vertical” structure of the CNP – researchers at each of these levels aim to generate data relating to the same set of neuropsychiatric disorders. The greater goal of the

Syndromes

Symptoms

Cognitive Cognitive Phenotypes Phenotypes

Fig. 3.7a Schematic of the multi-leveled (vertical structure) of the each level, and the position of cognitive informatics in CNP, the focus within relation to the other levels. From, www.phenomics.ucla.edu/index.asp; printed with permission, Robert Bilder

Neural Systems Cellular Systems/ Signalling Pathways Proteins

Genes Environment

3.3

Neuroinformatics Applications and Models for Neuropsychology

81

CNP is to draw mechanistic links between the gene level and the syndrome level, through all the other levels. With such a multileveled investigative paradigm and an emphasis on phenotypes, “we may start to see phenotypes quite different from what we are used to as the genotype data sets start to come in” (Bilder 2008c). Neuropsychology and Neuroinformatics in the CNP: Of the CNP’s five interrelated research projects, three are concerned with schizophrenia, bipolar disorder, and ADHD. They presently focus on neurocognitive phenotypes relating to memory mechanisms and response inhibition, which cut across all three disorders. (Since the phenome is so vast, a manageable study must limit the number of phenotypes on which it focuses. The proof-of-concept that is established paves the way for expansion.) A current study aims to examine the phenotypes (and their corresponding genotypes) in a sample of 2000 normal controls and a sample of 300 individuals who suffer one of the three disorders of interest. In both groups, genotypes and physical and functional neuroanatomy are examined and cognitive/neuropsychological functions (working memory and response inhibition/cognitive flexibility) are assessed. For levels of analysis (molecular and cellular) that cannot feasibly be carried out on human subjects, transgenic mouse models are developed to examine very specific effects. In the context of the CNP’s work, neuropsychology bears a very sophisticated role. This involves defining cognitive/behavioral endophenotypes, which in turn involves thoroughly examining various cognitive constructs. Existing neuropsychological tests that may tap an endophenotype has to be selected or new tasks have to be tailored to an endophenotype. All of this has to be done with the consideration that the data have to be meaningfully integrated with other levels of analysis and that comparable tasks have to be constructed when devising animal models. One of the goals of the CNP is to develop statistical and informatics tools for analyzing the data across the many research levels of the consortium. The tools relate to data visualization, modeling, and hypothesis generation (see Bilder et al. a, in press; Sabb et al. 2008). PubGraph, PubBrain, and Phenowiki are three novel tools that have been developed by CNP and available for public use: URLs: http://www.pubgraph.org/ http://www.pubbrain.org/ http://www.phenowiki.org/ PubGraph is a tool for literature mining and for the visualization of the associations between a set of results. It allows the user to enter one or more search terms, which it uses to search PubMed/MEDLINE. It then generates a graphical view of the results, representing resulting terms and concepts as nodes. Lines or “edges” linking the nodes convey coefficient and correlational strengths of associations between the terms, as statistically inferred through literature mining. Figure 3.7b shows the resulting graph for the search of the term “schizophrenia.” One way by which the tool can be utilized is in aiding neuropsychological refinement of a cognitive construct. Strengths of association between the nodes of the graph ultimately reflect factors and variables common to the nodes. The construct of “cognitive control” as

82

3 Neuroinformatics for Neuropsychology

Fig. 3.7b A PubGraph representation of the literature mining results for the term “schizophrenia” (results on cognitive/behavioral studies are clustered on the left, and results on genetic/cellular studies are clustered on the right). From, www.pubgraph.org/; printed with permission, Stott Parker

related in PubMed indexed literature, for example, is closely tied to the constructs (nodes) of response inhibition, response selection, task/set switching, and working memory (Bilder 2008b; Sabb et al. 2008). PubBrain works in a similar fashion – the user inputs search terms, which are queried in PubMed. The difference is that the results are displayed as a list of search

3.3

Neuroinformatics Applications and Models for Neuropsychology

83

terms that are extracted from the most frequently occurring search results. Simplified brain activation maps are generated based on the list of anatomical terms in the returned list. Phenowiki is a database-enabled wiki platform for the collaborative building of cognitive phenotypes. It enables users to explore a concept (e.g., working memory) by laying out definitions and citations, by describing related concepts, by annotating the existing content on the wiki, and by linking the descriptions to other resources such as PubMed, PubGraph, PubBrain, and Wikipedia. Phenowiki provides a space that promotes incubation, development, and modeling of neurocognitive concepts. Researchers are given a common medium in which their collective insights can be brought to bear on ideas and constructs and this is interfaced with other knowledge building resources. There are other related developments at CNP: A Cognitive Atlas has being built for illustrating maps/relationships between cognitive concepts. Cognitive Ontologies that reflect the sophistication of cognitive concepts and their complex associations are also being developed. Sophisticated search ontologies (see Section 2.5) are crucial for accurate data retrieval and for the building of detailed knowledge bases. These informatics-based tools developed by the CNP are more than just novel. They are quantitative tools geared for the association of complex data and they are a vital compliment to large and complex agenda of the CNP. In the context of the CNP, neuropsychology has to operate in an environment far different from that in which it evolved. In this kind of research and clinical environment, the fine granularity on genomic and cellular data pertaining to a disorder sets the bar that shapes the entire working environment. Neuropsychology, as a discipline, is required to apply the data in sharpening cognitive and behavioral constructs. It in turn is relied upon to precisely identify and investigate cognitive/behavioral phenotypes. It has to deliver neuropsychological data that are meaningful in the larger scheme of an explanatory framework that stretches from genomes to phenomes. The study of cognitive and neuropsychiatric phenotypes is a rapidly emerging, leading-edge discipline that involves neuropsychology. Once again, being part of such an enterprise requires that neuropsychology assimilate a rich variety of informatics methods and tools.

Conclusion for Section 3.3 The nine examples of NI for neuropsychology described in this section embody the ideas and goals of the NI approach to research and assessment. In relation to their respective neuropsychological domains, all the examples aspire to “capture, relate and analyze diverse types of data at multiple levels of abstraction” – the goals of NI as described by Wong and Koslow (2001, p. 104). All the examples can be described

84

3 Neuroinformatics for Neuropsychology

as responses to fundamental limitations in existing models and research/assessment methods and all can be viewed as radically new ways of addressing these limitations. Informed by theory, they deconstruct cognitive functions or neuroanatomic systems for a fine-grained level of analysis. They modify or devise tools for complex, multivariate data capture. They begin by testing the NI system on small groups or hypothetical data sets. They always develop databases to hold their wealth of data. They search and mine the data for complex relationships between variables. They look to related disciplines such as computational modeling for novel ways of coding and analyzing the data and they develop new ways of rendering the results. Like all areas of NI, the examples in neuropsychology are constantly evolving. Software is updated. Methods of analysis are improved. Databases grow in size, as does the user base. The NI system also incorporates new computer and hardware technologies as these technologies emerge. The initiative, whether at an early stage of development or one that has been through numerous iterations and has a larger user base, is always a work in progress. This format allows it to easily adapt, change, and improve, even while it is being actively used. The format is dynamic. All of the examples described happen to be projects that have been initiated by individuals or research groups. Some of the projects, for example, SignStream, have grown into larger projects involving multiple research teams. Irrespective of the size of the projects, they generally operate on an open platform and involve collaboration, while maintaining a center of coordination. Methods for deconstructing data and the designs for databases are often initially determined by the lead team, but standardization of these protocols is achieved through user input. A good deal of flexibility is built into the NI tool or database. This enables the user to manipulate the application with great ease and so tune it to his/her research or assessment needs. The contribution of data to a shared database is encouraged and the availability of the database gives the user a convenient reference tool. These NI applications in neuropsychology greatly empower researchers/ clinicians to clarify questions and clarify data. They generate new models of neuropsychological functions. Very importantly, they constitute a dynamic, userempowered, information-sharing and information-generating environment.

Chapter 4

Obstacles and Aids to Neuroinformatics in Neuropsychology

The threshold between neuropsychology and NI has been crossed. Neuropsychology is an established discipline with many long held conventions, and NI is a relatively new, highly innovative practice, the emergence of which is impossible to ignore. As these two fields begin to interact, there are bound to be those forces that are reticent about the prospect of neuropsychology being reinvented by informatics. There are also bound to be those forces that will eagerly endorse the growth of NI in neuropsychology, in recognizing enormous potential in NI. Each of these sets of forces will likely bring strong arguments based on the contexts in which they were shaped. Among the major goals of this book is to articulate the case for NI in neuropsychology and to do so in part by illustrating various limitations in neuropsychological models and assessment methods. This hardly implies a devaluation or dismissal of an entire tradition. Examining possible ways in which neuropsychology and NI will relate to each other becomes a complex matter. On the one hand, forces in neuropsychology likely to impede NI approaches need to be conveyed. These may relate to attitudes shaped by decades of professional consensus, tradition (or wisdom); practices imparted by disciplines to which neuropsychology has been closely tied; practical constraints; and special interests. On the other hand, bioinformatics has been so widely practiced over the past few decades that a hard-worn trail has been left: Many specialized disciplines have been through a reshaping by informatics and have taken the best of it, to emerge more powerful than ever before. Issues of data confidentiality and ownership have already been discussed as other clinical disciplines adopted informatics methods, and solutions have been proposed and implemented. Technical and logistical challenges have been plentiful and they have only made practitioners more determined to overcome them. The larger issue here cannot be cast as neuropsychology versus NI. In each field, there are such a variety of factors: some that better serve a productive meeting of the fields and some that do not. This section lays out three broad, related overarching considerations. Through each, some specific factors that could potentially slow the implementation of NI in neuropsychology are described, as are factors that can potentially mitigate the obstacles. The point of this section is to sample some of the major issues and some ways by which they can be resolved. V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9_4,

85

86

4

Obstacles and Aids to Neuroinformatics in Neuropsychology

4.1 Data Sharing and Issues of Privacy Issues of privacy that arise in the discussion of NI in neuropsychology pertain mainly to (a) confidentiality of patient data generated in clinical settings and how this should be treated and (b) data generated in academic/research contexts, the sharing of the data with other researchers, and the potential that the identity of research subjects can be gleaned from the data. As any clinician knows, rules of patient/data confidentiality apply to almost all types of clinical practice and clinical research. With its emphasis on fine granularity of data, NI applications may appear unsuited to clinical data since confidential patient information may be carried in the fine grain. This however is not the problem nor does it accurately reflect the NI approach. In general, NI approaches seek fine-grained data. To a great degree, this can be accomplished within the constraints of ethical and practical boundaries. NI approaches are shaped by greater clinical and research contexts. Just as a published clinical study may contain a lot of patient data without revealing patient identity, so can databases (and private and federal agencies do compile such databases for a variety of purposes). Neuropsychologists can develop databases for the data generated in their everyday practices. Even under the patient privacy rules of an institution and under ethical guidelines of a clinical discipline, there is still ample room to database clinical/assessment results that will otherwise be undocumented or filed away. The examples described in Section 3.3 suggest many ways by which NI can be applied to neuropsychology without requiring changes to privacy policies. At the same time, neuropsychologists should actively engage in a re-examination of privacy codes that have their roots in a bygone (“noninformational”) era, just as this re-examination has taken place in neuroscience (see Koslow 2000). When discussions have occurred in neuropsychology about privacy issues brought about by information technology in clinical practice (for example, report by the National Academy of Neuropsychology 2000; Bush et al. 2002), the discussions have been no more than a reiteration of established guidelines. What neuropsychology, especially clinical neuropsychology, needs to come to terms with is that the issue of privacy is recognized even at the level of National Research Council as a balancing act between individual privacy rights and public benefit from shared clinical data (see report by the National Academy Press/National Research Council 1999). Issues of data privacy prompted by developments in NI have played out intensely in the academic/research context. Nowhere has the debate been more vigorous than in discussions on whether to share functional imaging data. In mid-2000, when Michael Gazzaniga was director of the then newly created National fMRI Data Center (a brain image repository) at Dartmouth College, he made an announcement that pulsed the imaging world: Authors submitting imaging studies to the Journal of Cognitive Neuroscience (of which he was the editor) would also need to submit their primary data (fMRI scans) to the data center – an open access repository for research (see Editorial, Nature Neuroscience 2000; Aldous 2000). Many opposed the idea, citing issues of researcher rights to ownership; access by others to the data; and possible misinterpretation; entrustment of the data to another

4.1

Data Sharing and Issues of Privacy

87

institution, etc. Many supported the idea, citing the potential opportunity to review the work of others; facility to compare similar scans from different studies; and the building of a large image database – the debate continues. The controversy, said Gazzaniga, did a service to the field by engaging imaging researchers in an examination of the data sharing (see Aldhous 2000). “Sharing is justified because analysis of the same data sets by different scientists might yield a different interpretation and would not require someone to repeat the whole experiment to query the data with a different question. Sharing would also provide a rich data source for theoretical neuroscientists” (Koslow 2002). Clinical neuropsychologists may sooner or later be engaged in similar debates, as NI systems become part of their discipline. Some early lessons can be gained by looking at how issues around data sharing have been mediated in neuroscience. Van Horn and Gazzaniga (2002) described measures taken by the Dartmouth fMRI Data Center to ensure subject anonymity and protection: In compliance with US federal regulation 45 CFR 46 on human subject protection, we had to address the requirement that any and all information that can be used to identify a subject must be removed from the data, while maintaining its experimental integrity. This is accomplished in a two-pronged effort. First, contributing authors remove identifiers before submitting study information the Data Center. Then the Data Center checks for identifiers that might have been missed by the authors, while removing other potential identifying subject descriptors To remove [the possibility facial contour reconstruction from high resolution head images], the high resolution images are stripped of facial features.”

Many proposed methods for data sharing involve design elements in the NI system to do with controlled access to information (see Lee et al. 2000). Some examples are (a) A database that holds detailed raw data from multiple researchers but only gives a researcher access to an index of the data; the index contains the contact information for the contributor of each data set. With additional permission protocols, the data set can presumably be obtained from the contributor in a ready-to-use, standard format. (b) Data can be stored at progressive levels or stages, ranging, for example, from specific personal descriptors to the broad diagnostic categories and at each level, it can be divided in sets. Particular configurations of the data grid can be made available to a researcher based on the level of permission he/she holds (IT network and personal computer operating systems involve different levels of user rights). The greater point in this discussion concerns not the exact details of data access but the need for neuropsychology to recognize that engaging with NI also requires engaging the question of access to primary data. Stephen Koslow, the former director of the Office of Neuroinformatics at the NIH, has argued passionately on this cause (for data sharing) in articles such as “Should the neuroscience community make a paradigm shift to sharing of primary data?” (Koslow 2000) and “Sharing primary data: a threat or asset to discovery” (Koslow 2002). In the current time, these debates in NI have less to do with the question of whether data should be shared; that is largely accepted. The questions are about the rules and guidelines for data sharing that are needed. The need has come about suddenly with a new enabling technology showing enormous prospects for knowledge discovery through data sharing.

88

4

Obstacles and Aids to Neuroinformatics in Neuropsychology

Clinical neuropsychology especially needs to quickly get on board the discussion on data sharing that has been taking place in NI and neuroscience. Rather than filing away assessment results on pieces of paper and being unquestioning about privacy policies, it needs to examine its conventions against the prospect of collective scientific benefit brought on by NI.

4.2 Standardization, Taxonomies, and Ontologies for Neuropsychological Data At a purely practical level, if clinicians or researchers wish to share data in an expedient manner, the data need to be specified in a common format. There are various types of data formats depending on the context. For printed journal articles in neuropsychology and much of the medical field, the proverbial format involves an abstract, introduction, hypotheses, statement of methods, etc. With digitized data for NI applications, formats have to be viewed in the contexts of databases designed to hold the data and algorithms that will work on the data. A key variable relating to this format of data is the data field, i.e., a piece or unit of data of a certain type that is entered into the database. When researchers design a database for data sharing, they first try to establish exactly what kinds of data/data fields they wish to specify, and once decided, these data fields become a common standard for describing the data. The concept of throughput in informatics refers to the movement and processing of data through an informatics system so as to favor production of a desired output. It is measured by the average rate of successful output. Two main factors influence throughput, the design of the system and the structure of the initial form of the data. The latter, again, has to do with how the data are broken down on initial entry into the system. It also concerns how related pieces of data may be clustered at the data entry level and the validity of relatedness between elements of the cluster. If the NI system is to compare to data variables at a later level in its processing and if the data variables are inadequately specified at the data entry level, then throughput is affected as are potential comparisons that may require the fields of data. All of these point to the crucial elements of well-specified data input and uniform sets of data fields. A number of issues arise when determining the nature and extent of data fields for a database related to NI in neuropsychology, for example: Who decides the data fields? Whose research or clinical interests do they serve? On what theoretical grounds, if any, are the data fields derived – do they individually or as a cluster have construct validity? How many data fields should there be? (High levels of data granularity require many data fields and make for tedious data entry but hold more potential for complex analysis and vice versa). These are important considerations in database development. They involve anticipating many of the functions of the data and confronting potential problems sooner than later in the game. Irrespective of the complexities, the fact always remains that a particular set of data field will be common currency when databases are shared. Such standardization also enables interoperability between NI systems.

4.2

Standardization, Taxonomies, and Ontologies for Neuropsychological Data

89

Increasing throughput may involve among other things a great reliance on automated and online assessment tools: “Another possibility for increasing throughput is to adopt less labor-intensive procedures [f]or example, it is possible to collect information of certain behavioral features through assessment tools that can be self-administered over the Internet by individuals throughout the world.” (Freimer and Sabatti 2003). How open will the field clinical neuropsychology be to the introduction of large-scale automation in such a core component of its charge? How will these issues play out if neuropsychologists embark on developing an open-platform online database of results from assessment batteries? What kind of neuropsychological battery should be used? Would it be an adaptation of the widely used WAIS? Would it be more akin to ANAM? What if one or the other is considered inappropriate in terms of the precision with which a perceptual or cognitive feature is parsed out? If one embarks on a very ambitious NI project such as that conducted by the Consortium for Cognitive and Neuropsychiatric Phenomics, then it becomes imperative that the data fields in the database be very finely tuned to a cognitive phenotype. Usage of any neuropsychological test or battery presupposes a definition of the functions being tested (an issue raised many times in Chapter 3). Neuropsychology has always been using taxonomies, both implicit and explicit. The big problem here is that neuropsychology cannot optimally engage with NI without first confronting some of its taxonomies. Some may be simply inaccurate in the light in newer data and some may be so implicitly held in assessment tools that they have never been questioned (see Chapter 3). With refined taxonomies, the NI systems for neuropsychology can build ontologies that are more suited to the classification of complex data (see Bilder et al. c, in press). Taxonomy and ontology development influence each other. Data mining functions require some level of specification of how groups of data are related and if these relations are inaccurately specified, subsequent computed relations will be inaccurate (see Sections 2.5). Almost certainly, the talk of shared databases, open-ware neuropsychological batteries that will upload to these databases, and the radical reinvention of assessment tools to make them NI capable will raise the ire of certain commercial entities. These are the entities that have for many decades been pumping out spring-bound cardboard booklets and painted puzzles, nicely contained in immaculate black briefcases. Neuropsychologists, generally well intentioned, have aided in the process by authoring revisions of the batteries, and the field of neuropsychology has partaken in establishing assessment norms for various tests. Altogether, the arrangement has been that neuropsychologists have been dependent on these batteries (there have been no viable alternatives); this has legitimized the batteries even though they are based on questionable constructs and have designs that are antithetical to shared electronic data building; and in return, neuropsychologists have paid premium prices for copyrighted sheets of paper containing some rows and lines, on which to score results! In this era of NI, much superior alternatives are afforded. In fact, if these alternatives are developed under federal funding programs, they are likely to be freely available to neuropsychologists. How can neuropsychologists on a global scale achieve the development of assessment tools that are truly

90

4

Obstacles and Aids to Neuroinformatics in Neuropsychology

informatics-based and informed by contemporary cognitive neuroscience? Informatics developments for clinical and academic research funded by agencies like the NIH and the NSF provide the ideal model – bringing the best scientific teams to create wonderfully sophisticated tools that are then made freely available to an entire community. Consider some examples described in earlier sections: The Caret software for surface-based brain atlases is funded by the NIH, NSF, and other federal agencies. SignStream, the sign language database and analysis tool is funded by the NSF. PubBrain, PubGraph, and Phenowiki, are all developed at the Consortium for Neuropsychiatric Phenomics with funding from the NIH. These projects are funded by the public (tax payers); it follows logically that all scientists and clinicians have free access to the end results. In precisely the same way that SignStream has been developed, neuropsychologists need to team up and initiate a federally funded development of a large, modular, informatics-based assessment battery. To some degree, this has already been initiated with the development of ANAM in the context of the army. The civilian population deserves a similar level of protection! Federal funding agencies are the strongest force behind the idea that research data be digitally archived and available for public access. Tools/facilities like MEDLINE and PubMed enable the process. Articles published in the Proceedings of the National Academy of Sciences as well as in other journals published by federally funded agencies, are easily and freely available online. Authors submitting articles to such journals are required to use specific electronic formats that facilitate the process of online access. In the same way, researchers submitting data to the NIH’s National Center for Biotechnology Information (NCBI), an integrated onestop resource for molecular biologists and geneticists, use standardized methods. R the NIH database for gene sequences, is part of NCBI. It holds a large GenBank, collection of publicly available DNA sequences. This is made possible because data submission is through a common protocol. Will future neuropsychological research and practice involve these kinds of data sharing platforms and coordinating facilities? Will the discipline be able to establish uniformity in its subject descriptors, in methods of assessing them, and in methods of relating them? It is hard to imagine that neuropsychology will not reach this stage or that it will remain delayed. Neuropsychology, as always, is influenced by the greater contexts of medicine and psychology. Modern medicine, at least, is inseparable from the thrust of biomedical research, which is turn has informatics in its DNA. Neuropsychology will sooner or later be pressured or even mandated to be an equal player. In his presentation at the 2008 mid-year INS meeting, Robert Bilder poignantly summed up this complex issue: “. . . would you be willing to donate data if you had to transform everything so that it followed the rules made up by someone in Washington or someone at UCLA . . . and so that also is a problem; to have uniform ontologies for depositing data becomes a challenge . . . I think the problem is one of massive proportions almost to the point of being currently intractable with current methods because of the heterogeneity of collection methods and the way we go about specifying all of the phenotypic data that we collect . . . ontology development is already upon us; what is going to happen, the same way that the NIH is now mandating that genome wide association data be deposited under certain criteria for exactly how those data need to be deposited, in the future, all phenotypic data are going to

4.3

Attitudes, Re-organization, and Training

91

be required to be deposited following the rules of some federated ontology, so if you have an index of working memory, you will be able to choose from a menu of what that is . . . otherwise justify exactly how what you are doing is different and why you are not using existing techniques. I think that’s where it’s going.” (Bilder 2008c)

Indeed that is where it’s going! All the signs and signals on NI in neuropsychology, as laid out in Section 3.3 clearly say so. An urgent focus for neuropsychology is a broad-based examination of how to make this transition as smooth as possible.

4.3 Attitudes, Re-organization, and Training Major change of paradigms requires change of attitudes, and when paradigms change through technology, preparation and training are often needed. Strong resistance to NI in neuropsychology is unlikely. Resistance may occur to the extent that the issues described above (privacy issues and standardized NI methods) will be debated. Certain commercial interests may try various strategies to maintain their market share of neuropsychology but none are likely to prevent a major shift to shared NI systems for neuropsychology.1 Neuropsychological assessment methods have been developed over many decades. Norms for tests and batteries have been established through large masses of data gained over hundreds of thousands of clinical hours. The general operating system for clinical neuropsychology is one that is securely installed. A new technology that suddenly flies overhead on the trajectory of a sharp new paradigm will surely be unnerving as it transmits new rules. Change and the urging of the progress, however, are hardly new in neuropsychology. Pioneers in the field have always issued caution or made calls for revision. Writing in 1964 about the problem of small sample sizes in lesion studies, Ralph Reitan stated “We may be able to accumulate large enough groups within the next 20 years, but we would hope by that time the results might have lost their significance at least partially through obsoletion of the test battery!” (Reitan 1964, p. 305)

Benton (1992) reviewed the progress made in clinical neuropsychology over a period of three decades (1960–1990). He described that neuropsychology in the 1960 s was slow to tap development of the 1950 s in neurology and psychology: “However, in 1960, neuropsychologists had yet to apply the novel assessment techniques employed in these studies in their own clinical research and practice. Nor had they taken advantage of the many tests described in the literature of clinical neurology and the literatures of educational, vocational, industrial and applied psychology that might have been adapted for neuropsychological use. For the most part, clinical practice in the 1950 s relied

1A

commercial entity that produces a NI tool that facilitates information sharing in neuropsychology and does not impose numerous limits that serve primarily a profit making interest, cannot be viewed as an entity that impedes NI in neuropsychology.

92

4

Obstacles and Aids to Neuroinformatics in Neuropsychology

on analysis of performance patterns on the Wechsler-Bellevue, which was hardly the optimal instrument for the purpose, or on the first version of the Halstead-Reitan battery.” (Benton 1992, p. 410)

Benton went on to suggest that assessment in the current era needs to be informed by contemporary theory just as assessment decades ago needed to be informed by theory of the time: “Given our present knowledge of brain-behavior relationships, of cognitive processes . . . are we doing as well as we can? . . . In short, we should take a hard look at the neuropsychological examination. It deserves its own critical examination.” (p. 415) As the discipline faced such cautions and calls for changes, it did apparently respond – neuropsychology today is not what it was in its infancy and is marked with many successful efforts towards formalization. What then is different about the present call for neuropsychology to adopt NI? Will the field just adapt as it faces the evolutionary pressures on NI? As already conveyed, NI is not just a technology that can be tacked on nor does it amount to a retrofitting project. Unlike other adjustments that neuropsychology may have made in the course of its development (a tweak here and a recalibration there), NI implies deeply systemic changes. Nevertheless, as the benefits of NI in neuropsychology begin to emerge, the system can be expected to reorganize. To aid the process, many simple steps can be taken within the field: Professional bodies can appoint task forces to examine possible NI approaches and make recommendations. Conference organizers can aim to build NI themes into conference agendas. Clinicians and researchers can seek alliances with informatics specialists to help design NI systems. Perhaps the most effective (conventional) way to spread NI consciousness is through neuropsychology journals – by adding a component/section devoted to NI or at least encouraging manuscripts on the topic. An old problem in neuropsychology is that when relevant new technologies emerge they are often outside the view of the neuropsychologists. Kane and Kay (1992) noted this problem and Kane (2007) summed it up (referring here to computerized assessment): “Information about these efforts was available; however, it was found mostly in technical reports or in journals not typically monitored by neuropsychologists. Kane and Kay (1992) noted that the amount, quality and importance of the work being done in the field of computerized assessment was not fully appreciated and was in fact occurring under the radar of most neuropsychologists. (Kane 2007, p. S3).

Something similar can be said of NI developments that relate to neuropsychology. These developments, arguably, have been occurring in full range of the radar of neuropsychology, but field has just not been tuned to the unique signals of NI. This is all the more reason that NI should be given coverage in neuropsychology journals. While the above measures are needed, the logistical and technical challenges in reshaping how a discipline is practiced should not be understated. NI can be very technical, both in theory and application. Programmers, database developers, systems engineers, IT specialists, computational modeling specialists, etc., are those

4.3

Attitudes, Re-organization, and Training

93

who typically handle the more technical aspects of NI. It is imperative though that the neuropsychologists familiarize themselves with the field of NI in general. Neuropsychologists should have a good sense of NI potential – its functions and capabilities – so that they can at least conceptualize different NI methods when thinking of clinical and research problems. Of course, keeping abreast of cognitive neuroscience will also aid the formulation of these ideas. Writing in the context of automated assessment (see Section 3.1), Schlegel and Gilliland (2007) stated, “Given the rapidly expanding frontiers of neuroscience, test developers will increasingly have to keep one eye on test programming and one eye on neuroscience research to remain abreast of construct validity advances.” (p. S54). Conferring NI perspectives on neuropsychology trainees should be a high priority. This implies changes to the syllabi and requirements of clinical, doctoral, and postdoctoral programs. Just as training in psychology generally requires courses in statistics, budding neuropsychologists should be empowered by courses in NI for neuropsychology. In making such curricular adjustments, a good resource may again be federal funding agencies. The NIH, for example, has a funding program for clinical research curriculum development; it supports the development of formal courses in many biomedical areas including medical technology. The more neuropsychology is informed about NI, the smoother its course of engagement will be. Exposure to existing projects (Section 3.3), understanding their rationale and their gains, and the emergence of a few more could help catalyze change with the field. Bringing NI to neuropsychology will undoubtedly be challenging but all that has occurred in NI thus far should be inspiring and motivating.

Chapter 5

An International Society for Neuroinformatics in Neuropsychology

URL: www.scnn.org Neuroinformatics interest groups have steadily emerged within research organizations and academic societies. The NIH has one, as does the Society for Neuroscience. These interest groups can vary in scope, from offering platforms for discussion to acting as central coordinating facilities for global research in NI. The neuroinformatics interest group of the OECD (see Section 2.6) is an example of the latter. The importance of NI interest groups has been conveyed throughout this book – they foster collaboration and promote NI initiatives. In the course of writing the book, the author developed a collaborative network with many clinicians and researchers whose projects are described in Section 3.3. Much of the collaboration involved exchange of information, updates on NI projects, reviews of data, etc. As noted in the introduction to Section 3.3, essential material for the book was obtained through this collaboration. By the end of writing the book, the author had networked a group of researchers with common interests. A logical step was to form an organization to promote these interests. An international organization devoted to neuroinformatics for neuropsychology was hence recently formed. It was named the Society for Neuroinformatics in Neuropsychology (SCNN). A website for the organization has been launched (see above URL). The website may be of benefit to researchers interested in NI for neuropsychology. The SCNN will serve three broad functions: (a) foster discussion on the topic, (b) compile and communicate resources, and (c) assist in coordination of projects. These functions will to some degree overlap with each other. Emphasis on each function will vary at different times depending on the needs expressed by members and the specific priorities of the organization at a given point. However, the organization will remain structured around the three main functions. These functions are summarized below (see Fig. 5.1). Discussion Forum: A discussion forum will offer researchers the space to explore ideas for NI in neuropsychology. It will also help to focus discussion on a number of relevant topics such as • Database platforms and distributed architecture for large-scale NI systems for neuropsychological assessment data • NI systems for the study of specific cognitive functions • Synchronization of databases, computer platforms, and protocols between multiples teams V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9_5,

95

96

5

An International Society for Neuroinformatics in Neuropsychology

Examples of Discussion Topics -General design of NI systems

Examples of Working Group Projects

-Database/network architecture

-Ontology for data mining

-Synchronization of systems

-NI in neuropsychology curricula

-Theoretical frameworks

-SCNN conferences

-Data privacy

PROJECT COORDINATION & WORKING GROUPS

DISCUSSION FORUM

SCNN

-Neuropsychological NI in National Health Information Networks -Coordination of multiple small scale projects

Examples -Relevant NI projects -Books, websites, etc

COMMUNICATION & RESOURCES

-Funding programs -Meetings, conferences, etc. -Resources for software/hardware development

Fig. 5.1 Summary illustration of the major functions of the society for neuroinformatics in neuropsychology

• Theoretical frameworks for neurocognitive functions – to guide NI initiatives • Ontologies for knowledge discovery in neuropsychology • Data privacy issues Communication and Resources: The SCNN aims to communicate developments in the field. The organization’s website lists a range of resources. Researchers are encouraged to submit links to other resources. Resource categories are expected to expand based on user needs and feedback. Resources currently listed include • Current NI-neuropsychology projects • Articles, books and websites • Meetings, conferences, and symposia (in neuropsychology, computational neuroscience, informatics, etc.) • Funding programs that support NI initiatives in neuropsychology • Academic programs in neuropsychology at universities with bioinformatics programs • Various software and hardware resources and vendors • Software policies and support for open-source developers

5

An International Society for Neuroinformatics in Neuropsychology

97

Project Coordination and Working Groups: As needs and projects arise, the organization will be able to draw on its network to form working groups and committees. These may be set up to address, for example, some of the topics in the discussion forum that may be actively discussed and thereby call for special focus committees. Working groups could be set up to explore for example, ways of introducing NI in neuropsychology training curricula, and the place of neuropsychological informatics in the context of national health information networks. Another possible area for a coordinating committee is the tracking and coordination of many small-scale NIneuropsychological efforts around the world so that research groups can consider ways of making their data sets and NI systems interoperable. The SCNN is a very new organization and is currently led by the author in collaboration with some of the researchers whose work has been described in earlier. It is hoped that as the group gains a broader membership, more individuals can contribute to its structure and mission. The SCNN website will provide updates periodically.

Chapter 6

Concluding Remarks

Neuropsychology has a long and proud history. It grew out of behavioral neurology and psychology and developed into a full-fledged discipline. So much of cognition and behavior is meaningful only when studied in relation to the brain – and this is where neuropsychology has successfully demonstrated its unique focus. Clinical neuropsychology and experimental neuropsychology have been the tracks that have formerly defined the discipline for more than half a century. Since the emergence of “cognitive neuroscience” about three decades ago, there has been a noticeable “shrinking” of the field of experimental neuropsychology. Many topics and researchers that once belonged to experimental neuropsychology have sought a more fitting (and perhaps more prestigious) home in cognitive neuroscience. One can debate endlessly the degree of overlap or difference between experimental neuropsychology and cognitive neuroscience or even whether there is any difference at all. What is less arguable is that the playing field of the discipline of neuropsychology changed drastically in the 1980 s and 1990 s as brain-behavioral research rapidly accelerated in disciplines like neurobiology and as new imaging methods came onto the scene. The traditional neuropsychological methods of deconstructing cognitive behavior and seeking relationships between cognition and neural systems had by this point reached a plateau. Assessment tests and lesion models in neuropsychology would remain valuable as always but they had reached fundamental limits. Much clearer, discrete brain-behavioral associations were being generated through behavioral-neurophysiologic investigations in primates and through cognitive functional imaging in humans. What was once a clear transitional zone between neuropsychology and neuroscience started to see some complex demarcations – making for the emergence of “cognitive neuroscience.” Clinical neuropsychology, though influenced by this change in the landscape, remained largely as it was. Since it was a thriving discipline, established in its own right, there was no pressure for fundamental shifts from its well-codified tools and procedures. It continued along with its assessment batteries that had their roots in psychometry and intelligence testing. It continued with its armament of individual tests that were tuned to gross cognitive/perceptual complexes. A strong neuraltheoretical basis for many of these tools would remain unclear though the need was acknowledged. It had revised and formed these tools over many generations V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9_6,

99

100

6 Concluding Remarks

and they influenced the definition of the functional domains on which clinical neuropsychology traditionally focused. For the most part, the data generated by these tools were in physical form, viewed by the clinician and then stored away. As was the case in almost all other disciplines, knowledge building in neuropsychology was a slow and manual process. Data would be published in journal articles, and over the years, patterns would be observed and models could be built. A Major Change in the Neuropsychological Landscape: The landscape on which neuropsychology operates is once again changing. The change began towards the end of the 20th century. It continues into the beginning of the 21st century and all indications are that it will transform the way many disciplines are practiced. This change is all about data. It has to do with how data are extracted and dealt with in this greater digital informational era. Once upon a time, clinicians and researchers were relatively disconnected and only got to share their ideas and data in publications and at conferences. Now, through the Internet, through databases, and a host of informatics technologies, new and previously unimaginable methods for collaboration have been minted. These methods are rapidly becoming the currency for discovery, effectiveness, and innovation in clinical, research, and academic worlds. The vast amount of data on the Internet is only one of two key factors in this revolution. The other factor centers on a powerful set of next-generation Internet tools. These tools facilitate the knowledge building process by enabling individual sets of data across the web to be integrated and by enabling researchers to dynamically participate in the process. These changes in how knowledge is built are influencing all disciplines including the likes of literature and philosophy. Neuropsychology is at a critical threshold. It faces entry into an altogether new way of transacting its business. Yet, ironically, the identity that neuropsychology has so carefully built over the decades seems to have locked in a consciousness that prevents it from recognizing this change. How it crosses this threshold will have far flung ramifications. It is imperative that neuropsychology perceives very clearly the new informational platform that has grown all around it.

6.1 Collaborative Knowledge Building in the Next Wave of the Internet: Web 2.0, Neuropsychology 2.0, and Global Network Innovations The Internet and the abundance of information it has generated have been obvious definitive features of the past two decades. This larger phenomenon has had many societal implications, among them, a displacement of the traditional model of knowledge dissemination (see Jensen 2007). The Internet greatly advanced the democratization of knowledge. Even specialized knowledge, once held by few as a scare entity, became commonly available. Expertise became less and less the domain of a few specialists and more and more spread across a large set of Internet users. The first generation of the Internet (from the early 1990 s to approximately 2004) while revolutionizing the spread of information bore some elements of traditional

6.1

Collaborative Knowledge Building in the Next Wave of the Internet

101

knowledge models: It was characterized by a large number of servers that held information; users went to these servers and got the information. The sheer number of servers and users and the resulting global flow of information made for an information revolution. Yet the server (vendor) and client (receiver) model was in some ways reminiscent of information dissemination during the pre-digital age. In 2004, formal recognition was made of a unique set of practices that had begun to evolve over the Internet. These practices altogether established ways of collective knowledge building in which large groups of users became the contributors and co-developers. “Web 2.0” was the term coined to describe these practices, the principles behind the practices, the specialized web tools involved, and the radical shift to a model of networked communities.1 Web 2.0 captures a trend in open platform Internet use that is geared to user-driven online networks and knowledge bases that constantly change and grow. It is a flexible, user-driven approach to building a product. “. . . [A] single monolithic approach, controlled by a single vendor, is no longer a solution, it’s a problem” (O’Reilly 2005, p. 2). Wiki platforms are one of many tools that enable collaborative knowledge building. The online encyclopedia wikipedia.org is among the best known examples of a product derived with Web 2.0 principles. Web 2.0 in essence is about (1) harnessing interactive user networks that stem from the Internet, to (b) collectively develop and customize ideas and products and (c) refine these ideas or products as more users contribute to them. The result is a knowledge-oriented participatory environment in which great amounts of data are integrated and novel products are molded. Dynamic forms of collaboration and engagement are hallmarks of Web 2.0, and the power of the phenomenon has been referenced by numerous terms over the past few years, such as digital democracy, collective intelligence, wisdom of groups, architecture of participation, and dynamic content creation. At the February 2009 meeting of the International Neuropsychological Society in Atlanta, Georgia, Robert Bilder presented a paper titled “Neuropsychology 2.0: Leveraging Genomics, Phenomics, and the World Wide Web”. Bilder described how existing technologies (of the Web 2.0 variety) can be used to re-energize the field in view of the limitations brought about by its outmoded neurobehavioral models and “instrument inertia.” Bilder proposed an action plan involving web collaborations to build cognitive ontologies, knowledge bases, and automated assessment instruments. It is inevitable that such developments will transform neuropsychology, making for “Neuropsychology 2.0”. The field is destined to reach a critical point of departure from its now maladapted tools and traditions. Informatics tools will enable new modes of data gathering and knowledge building. With NI, neuropsychology 1 An

authoritative commentator on the Internet, Tim O’ Reilly, is widely credited with having coined the term “Web 2.0” in 2004. O’Reilly’s 2005 article, “What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software” is considered a landmark reference on the topic (http://www.oreillynet.com). Numerous informative books on Web 2.0 have since been written. See for example Jones (2008) and Vossen and Hagerman (2007). See also http://en.wikipedia.org/wiki/Web_2.0

102

6 Concluding Remarks

will also be able to develop sophisticated standardized, high-throughput assessment tools that will be refined through dynamic user input. As the field embarks on this change, it will also be building an informatics infrastructure that will serve it for decades to come. Generations of neuropsychologists stand to benefit from a comprehensive global NI-cyber-infrastructure for neuropsychology. This networked system will feed data onto itself making for a constantly enriched, dynamic data/knowledge environment. The concept of utilizing computer networks for experimentation and knowledge generation is such a powerful one that it is a driving concept for the federally supported effort to revamp that Internet. The Global Environment for Network Innovations (GENI) is an NSF sponsored program that tests a range of ideas for computer network architecture, with a view to improving the existing Internet. What is unique about the project is that it is about much more than hardware infrastructure. It recognizes that optimal Internet infrastructure needs to incorporate network layers and novel communication platforms through which “the dynamic interactions of physical and social spheres” can be played out (www.geni.net). GENI engages researchers from academia and industry across a range of disciplines. It solicits proposals and provides test beds for new forms of network infrastructure. Neuropsychology can take every advantage of such programs to create for itself a digital sandbox – a network test space.

6.2 The Promising Politics of Informatics Infrastructure The self-generated, self defined force of the internet may be phenomenal but it is still influenced by broader social, economic and political factors. Communications infrastructure, for example, on which the internet depends, is affected by prevailing economic climate. The updating of national information grids to handle greater data volume and at higher speeds is a common concern and this aspect of internet development is often tied to federal spending and political agenda. The lack of a unified health information network (in the USA) is a matter that has been receiving great attention. Physicians in the USA do not have a nationwide system for patient health records: Only a small fraction of hospitals and physicians in private practice have implemented comprehensive information systems for medical records, and different computerized systems are used by different healthcare networks (see Jha et al. 2009). The result is that billions of dollars are spent annually on manual transfer of records or the regeneration of records. A bureaucratic environment, hazardous to patient heath, has emerged. This concern that has been expressed so strongly in clinical medicine (see Blumenthal 2002; Blumenthal and Glaser 2007; Jha et al. 2006) is also a concern for clinical neuropsychology as the field looks to developing an informatics infrastructure. The problem of the lack of a national heath information network (HIN) has been deemed so dire that the current president of the USA, soon after being elected, addressed the issue as part of his economic recovery plan:

6.3

Prologue to Neuropsychology for the 21st Century

103

“In addition to connecting our libraries and schools to the internet, we must also ensure that our hospitals are connected to each other through the internet. That is why the economic recovery plan I’m proposing will help modernize our health care system – and that won’t just save jobs, it will save lives. We will makes sure that every doctor’s office and hospital in this country is using cutting edge technology and electronic medical records so that we can cut red tape, prevent medical mistakes, and help save billions of dollars each year” (Obama 2008; excerpt from a radio address delivered on Dec. 6).

The promise drew applause but also words of caution. Some specialists on the topic responded in an open letter, highlighting among many points, the lack of interoperability among current systems, and the need for some basic steps such as extending high speed internet access to all practitioners.2 Undaunted by debates on the value of health information networks or the difficulty of the task, about $20 billion dollars have been allocated toward health information technology (HIT) through the American Recovery and Reinvestment Act of 2009. The initiative will be overseen by the newly created Office of the National Coordinator of Health Information Technology (see Blumenthal, 2009; www.recovery.gov). Discussions and debates around health information technology and electronic medical records continue and have broadened into the public domain.3 The issue is very pertinent to neuropsychology as neuropsychology considers large-scale informatics systems for communication and data sharing. A national medical informatics system that carries comprehensive health records could likely hold neuropsychological data as well. Whether or not it does have a place for neuropsychological data may depend on how quickly neuropsychologists ride into this frontier of health information networks. To be tuned to such considerations, the discussion on NI for neuropsychology needs to be all encompassing. It has to consider the broader infrastructure and when there are planned developments within that can potentially serve neuropsychology, the opportunity should be promptly examined.

6.3 Prologue to Neuropsychology for the 21st Century In the course of its historical development, neuropsychology has been very open to new technology – from brass instruments to EEG, from microcomputers to fMRI. The discipline has been very good at incorporating new technology into its existing

2 David

Kibbe, a senior advisor to the American Academy of Family Physicians, and Bruce Klepper, a healthcare market analyst, sent an open letter to Barack Obama in which they outlined key problems and made suggestions regarding electronic health records in the USA. (www.thehealthcareblog.com/the_health_care_blog/2008/12/where-should-fe.html) 3 For example, the National Public Radio program, All Things Considered, interviewed Dr. Ashish Jha (Harvard School of Public Health) on March 25, 2009, on the topic on HIN implementation in USA: “Electronic Medical Record Change Not Easy”. (http://www.npr.org/templates/story/ story.php?storyId=102360638)

104

6 Concluding Remarks

structure and then adjusting its focus or developing subspecialties based on insights gained through the technology. As great an impact that various technologies have had on the field, none has implied or required that the discipline of neuropsychology undergo an evaluation of so many of its fundamental operating principles – relating to its constructs of cognition, how it assesses these constructs, how its data are gathered and protected, how inferences are drawn from the data, how models of behavior are built, and how neuropsychologists communicate as a community. NI has fundamentally different implications in this regard. NI is much more than just a new technology. It is about a major change of paradigm on the treatment of neuroscience data at all levels. The implication for neuropsychology, especially clinical neuropsychology, is that its traditions and tool sets are going to be widely challenged. From its clinical assessment methods to its modes of training, clinical neuropsychology is at risk of severe incongruence with a surrounding mass of fine-grained neuroscience data. NI is the great unifying force for data in the brain sciences. It is undoubtedly a powerful, transformative force that has reoriented much of neuroscience. This is the brave new world in which neuropsychology has to function. The effect of NI on neuropsychology will not be ameliorating but instead greatly enlivening. It is because of the rich history of neuropsychology and its long line of remarkable pioneers that the discipline has reached such a high level of distinction. To continue this legacy, it must embark on entirely new trajectories set forth by technologies of our time. It should do so with vision and firm embrace. Neuropsychology in the 21st century will be defined by neuroinformatics.

References

A. Reference List (Journal Articles, Books, Book Chapters, etc.) Aboitiz, F. & Garcia, V.R. (1997). The evolutionary origin of the language areas in the human brain. A neuroanatomical perspective. Brain Research Reviews, 25, 381–396. Adams, K.M. & Heaton, R.K. (1987). Computerized neuropsychological assessment: Issues and applications. In J.N. Butcher (Ed.) Computerized Psychological Assessment. A Practitioner’s Guide. New York: Basic Books. Aldhous, P. (2000). Prospect of data sharing givers brain mappers a headache. Nature, 406, 445–446. Almli, C.R., Rivkin, M.J., & McKinstry, R.C. & Brain Development Cooperative Group. (2007). The NIH MRI study of normal brain development (Objective-2): Newborns, infants, toddlers, and preschoolers. NeuroImage, 35(1), 308–325. Amari, S., Beltrame, F., Bjaalie, J.G. et al. (2002). Neuroinformatics: The integration of shared databases and tools towards integrative neuroscience. Journal of Integrative Neuroscience, 1(2), 117–128. Andersen, R.A., Brotchie, P.R., & Mazzoni, P. (1992). Evidence for the lateral intraparietal area as the parietal eye field. Current Opinion in Neurobiology, 2, 840–846. Andersen, R.A., Essick, G.K., & Siegel, R.M. (1987). Neurons of area 7 activated by both visual stimuli and oculomotor behavior. Experimental Brain Research,67, 316–322. Andersen, R.A., Snyder L.H., Li C.S., & Stricanne, B. (1993). Coordinate transformations in the representation of spatial information. Current Opinion in Neurobiology, 3(2), 171–176. Arbib, M.A. (Ed.) (1995). The Handbook of Brain Theory and Neural Networks. Cambridge, MA: MIT Press. Arbib, M.A. & Caplan, D. (1979). Neurolinguistics must be computational. Behavioral and Brain Sciences, 2, 449–483. Arbib, M. & Bota, M. (2003). Language evolution: Neural homologies and neuroinformatics. Neural Networks, 16, 1237–1260. Axer, H., Jantzen, J., Berks, G., Keyserlingk, D.G.v. (2000). Aphasia Classification Using Neural Networks. In Proceedings of ESIT 2000, European Symposium on Intelligent Techniques, September 14–15, 2000, Aachen, Germany; ERUDIT (Ed.), pp. 111–115. Axer, H., Jantzen, J., Berks, G., Südfeld, D., Keyserlingk, D.G.v. (2000). The Aphasia Database on the Web: Description of a Model for Problems of Classification in Medicine. In Proceedings of ESIT 2000, European Symposium on Intelligent Techniques, September 14–15, 2000, Aachen, Germany; ERUDIT (Ed.), pp. 104–110. Axer, H., Jantzen, J., & Keyserlingk, D.G.v. (2000). An aphasia database on the internet: A model for computer-assisted analysis in aphasiology. Brain and Language, 75(3), 390–398. Bearden, C.E. & Freimer, N.B. (2006). Endophenotypes for psychiatric disorders: Ready for primetime? Trends in Genetics, 22(6), 306–313. V. Jagaroo, Neuroinformatics for Neuropsychology, C Springer Science+Business Media, LLC 2009 DOI 10.1007/978-1-4419-0060-9,

105

106

References

Behavioral Neuroscience Systems, LLC. (date) BrainCheckers: A multidimensional system with focus on mental status monitoring, and performance enhancement. Bellugi, U. & Klima, E.S. (1976). Two faces of sign: Iconic and abstract. Annals of The New York Academy of Sciences, 280, 514–538. Bellugi, U., Klima, E.S., & Poizner, H. (1988). Sign language and the brain. Research Publications: Association for Research in Nervous and Mental Disease, 66, 39–56. Benson, D.F. (1993). Aphasia. In K. Heilman, M. Kenneth, & E. Valenstein (Eds.) Clinical Neuropsychology (pp. 17–36). New York: Oxford University Press. Benton, A.L. (1992). Clinical neuropsychology: 1960–1990. Journal of Clinical and Experimental Neuropsychology, 14(3), 407–417. Benton, A.L. (1994). Neuropsychological assessment. Annual Reviews in Psychology, 45, 1–23. Bilder, R.M. (2007). Cognitive phenomics for neuropsychology. The Clinical Neuropsychologist, 21, 382–421. Bilder, R.M. (2008a). Phenomics: Building scaffolds for biological hypotheses in the post-genomic era. Biological Psychiatry, 1(63), 439–440. Bilder, R.M. (2008b). Phenomics for neuropsychology: Trans-disciplinary strategies for research on brain-behavior relations in the post-genomic era (Slides on compact disc). Presentation at the Annual International Neuropsychological Society Meeting, Waikoloa, HI, Feb. 6–9. Bilder, R.M. (2008c). Phenomics for neuropsychology: Trans-disciplinary strategies for research on brain-behavior relations in the post-genomic era (Audio recording on compact disc). Presentation at the Annual International Neuropsychological Society Meeting, Waikoloa, HI, Feb. 6–9. Bilder, R.M., Poldrack, R., Stott Parker, D., Reise, S.P., Jentsch, J.D., Cannon, T., London, E., Sabb, F., Foland, L., Rizk-Jackson, A., Kalar, D., Brown, N., Carstensen, A., & Freimer, N. (ain press). Cognitive phenomics. In S. Wood & C. Pantelis (Eds.) Handbook of Neuropsychology of Mental Disorders. Bilder, R.M., Sabb, F.W., Cannon, T.D., London, E.D., Jentsch, J.D., Parker, D.S., Poldrack, R.A., Evans, C., & Freimer, N.B. (bin press). Phenomics: The systematic study of phenotypes on a genome-wide scale. Bilder, R.M., Sabb, F.W., Parker, D.S., Kalar, D., Chu, W.W., Fox, J., & Poldrack, R.A. (cin press). Cognitive ontologies for neuropsychiatric phenomics research. Cognitive Neuropsychiatry. Binder, J. (1997). Neuroanatomy of language processing studied with functional MRI. Clinical Neuroscience, 4, 87–94. Binder, J. (2008), Clinical Applications of FMRI for Presurgical Mapping (Slides). Presentation at the Introductory fMRI Course, Medical College of Wisconsin in Milwaukee, WI, June 5–7. Bisiach, E., Bulgarelli, C., Sterzi, R., & Vallar, G. (1983). Line bisection and cognitive plasticity of unilateral neglect of space. Brain and Cognition, 2(1), 32–38. Bisiach, E., Geminiani, G., Berti, A., & Rusconi, M.L. (1990). Perceptual and premotor factors of unilateral neglect. Neurology, 40(8), 1278–1281. Bisiach, E. & Luzzatti, C. (1978). Unilateral neglect of representational space. Cortex, 14, 129–133. Bisiach, E., Pizzamiglio, L., Nico, D., & Antonucci, G. (1996). Beyond unilateral neglect. Brain: A Journal of Neurology, 119(3), 851–857. Bisiach E., & Rusconi, M.L. (1990). Break-down of perceptual awareness in unilateral neglect. Cortex, 26(4), 643–649. Bleiberg, J., Reeves, D., Elsmore, T., Wolkenberg, F., & Kelly, J. (2007). Brain checkers: Sports medicine battery V1.1. (User’s Manual). Behavioral Neurosciences Systems, LLC. Bleiberg, J., Roebuck-Spencer, T.M., & Frisch, D. (2004). ANAMTM Access database. (User’s Manual). National Rehabilitation Hospital. Blumenthal, D. (2002). Doctors in a wired World: Professionalism survive connectivity? The Milbank Quarterly, 80(3), 525–546.

References

107

Blumenthal, D. (2009). Stimulating the adoption of health information technology. The New England Journal of Medicine, 360(15), 1477–1479. Blumenthal, D. & Glaser, J.P. (2007). Information technology comes to medicine. The New England Journal of Medicine, 356(24), 2527–2534. Bower, J.M. & Beeman, D. (1998). The book of GENESIS: Exploring realistic models with the GEneral NEural SImulation System (2nd ed.). New York: Springer-Verlag New York, Inc. Bower, J.M., Beeman, D., & Hucka, M. (2003). The GENESIS simulation System. In M.A. Arbib (Ed.) The Handbook of Brain Theory and Neural Networks (2nd ed.). Cambridge, MA: The MIT Press Broca, P. (1861). Remarques sur le siege de la faculte de langage articule, suivie d’une observation d’aphemie (perte de la parole). Bulletins de la Societe Anatomigue, 6, 330–357. Broca, P. (1865). Sur la faculte du langage articule. Bulletin de la Societe d’Anthropologie, 6, 337–393. Burnett-Stuart, G., Halligan, P.W., & Marshall, J.C. (1991). A newtonian model of perceptual distortion in visuo-spacial neglect. Neuroreport, 2, 255–257. Burns, G. (2001a). Knowledge mechanics and the neuroscholar project: A new approach to neuroscientific theory. In M. Arbib & J. Grethe (Eds.) Computing the Brain, 1 (pp. 319–336). San Diego, CA: Academic Press. Burns, G.A. (2001b). Knowledge management of the neuroscientific literature: The data model and underlying strategy of the NeuroScholar system. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 356(1412), 1187–208. Burns, G.A.P.C., Khan, A.M., Ghandeharizadeh, S., O’Neill, M.A., & Chen, Y. (2003). Tools and approaches for the construction of knowledge models from the neuroscientific literature. Neuroinformatics, 1(1), 81–109. Bush, S., Naugle, R., & Johnson-Greene, D. (2002). Interface of information technology and neuropsychology: Ethical issues and recommendations. Clinical Neuropsychology, 16(4), 536–547. Buxton, R.B. (2001). Introduction to Functional Magnetic Resonance Imaging: Principles and Techniques. Oxford: Cambridge University Press. Cernich, A.N., Brennana, D.M., Barker, L.M., & Bleiberg, J. (2007). Sources of error in computerized neuropsychological assessment. Archives of Clinical Neuropsychology, 22S, S39–S48. Chalupa, L.M. & Werner, J.S. (2004). The Visual Neurosciences. Cambridge, MA: The MIT Press. Chomsky, N. (1957). Syntactic Structures.The Netherlands: Mouton & Co., N.V., Publishers, The Hague. Churchland, P. & Sejnowski, T.J. (1994).The Computational Brain (Computational Neuroscience).Cambridge, MA: The MIT Press. Chute, D.L. (2002). Neuropsychological technologies in rehabilitation. Journal of Head Trauma Rehabilitation, 17(5), 369–377. Cohen, M.S., Kosslyn, S.M., Breiter, H.C., DiGirolamo, G.J., Thompson, W.L., Anderson, A.K., Brookheimer, S.Y., Rosen, B.R., & Belliveau, J.W. (1996). Changes in cortical activity during mental rotation. A mapping study using functional MRI. Brain, 119, 89–100. Collins, F.S., Morgan, M., & Patrinos, A. (2003). The human genome project: Lessons from large scale biology. Science, 300 (5617), 286–290. Corina, D.P., Bellugi, U., & Reilly, J. (1999). Neuropsychological studies of linguistic and affective facial expressions in deaf signers. Language and Speech, 42, 307–331. Damasio, A. (1990). Synchronous activation in multiple cortical regions: A mechanism for recall. Seminars in the Neurosciences, 2, 287–296. Dayan, P. & Abbott, L.F. (2005). Theoretical neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: The MIT Press. Deacon, T.W. (1997). The Symbolic Species: The Co-evolution of Language and the Brain. New York: W.W. Norton & Company, Inc.

108

References

Dell, G. S., Burger, L.K., & Svec, W.R. (1997). Language production and serial order: A functional analysis and a model. Psychological Review, 104(1), 123–147. Dell, G.S., Martin, N., & Schwartz, M.F. (2007). A case-series test of the interactive two-step model of lexical access: Predicting word repetition from picture naming. Journal of Memory and Language, 56(4), 490–520. Deutsch, C.K., Hobbs, K., Price, S.F., & Gordon-Vaugh, K. (2000). Skewing of the brain midline in schizophrenia. Neuroreport, 11(18), 3985–3988. Deutsch, C.K. & Joseph, R.M. (2003) Brief report: Cognitive correlates of enlarged head circumference in children with autism. Journal of Autism and Developmental Disorders, 33(2), 209–215. Donnelly, N., Guest, R., Fairhurst, M., Potter, J., Deighton, A., & Patel, M. (1999). Developing algorithms to enhance the sensitivity of cancellation tests of visuospatial neglect. Behavior Research Methods, Instruments, & Computers, 31(4), 668–673. Downing, P.E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001). A cortical area selective for visual processing of the human body. Science, 293(5539), 2470–2473. Dreuw, P., Neidle, C., Athitsos, V., Sclaroff, S. & Ney, H. (2008, May). Benchmark databases for video-based automatic sign language recognition. Paper presented at the Sixth International Conference on Language Resources and Evaluation in Marrakech, Morocco. Duhamel, J.R., Colby, C.L., & Goldberg, M.E. (1991). Congruent representations of visual and somatosensory space in single neurons of monkey ventral intra-parietal cortex (area VIP). In J. Paillard (Ed.) Brain and Space (pp. 223–236). New York: Oxford University Press. Editorial (2000, September). “A debate over fMRI data sharing.” Nature Neuroscience, p. 845. Elsmore, T. & Reeves, D. (2004). ANAM readiness evaluation system (ARES): User’s Guide. Activity Research Services. R test system for palm OS Elsmore, T.F., Reeves, D.L., & Reeves, A.N. (2007). The ARES handheld computers. Archives of Clinical Neuropsychology, 22S, S135–S144. Emmorey, K. (2002). Language, Cognition, and the Brain. New Jersey: Lawrence Erlbaum Associates. Emmorey, K., Mehta, S., & Grabowski, T.J. (2007). The neural correlates of sign versus word production. Neuroimage, 36(1), 202–208. Emmorey, K. & Reilly, J.S. (1995). Theoretical issues relating language, gesture, and space: An overview. In K. Emmorey & J.S. Reilly (Eds.) Language, Gesture and Space (pp. 1–19). Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers. Epstein, R. & Kanwisher, N. (1998). A cortical representation of the local visual environment. Nature, 9(392), 598–601. Evans, A.C. & The Brain Development Cooperative Group (2004). NIH MRI study of normal brain development. Presented at Pediatric Functional Neuroimaging: A Trans-NIH Workshop, Bethesda, Maryland, May, 25. Evans, A.C. & the Brain Development Cooperative Group (2006). The NIH MRI study of normal brain development. NeuroImage, 30(1), 184–202. Farah, M.J. (2000). The cognitive Neuroscience of Vision. Oxford: Blackwell Publishers. Farkas, L.G. & Deutsch, C.K. (1996). Anthropometric determination of craniofacial morphology. American Journal of Medical Genetics, 2(65), 1–4. Fellbaum, C. (1998). WordNet: An Electronic Lexical Database. Cambridge, MA: The MIT Press. Felleman, D.J. & Van Essen, D.C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47. Flint, J. & Munafò, M.R. (2007). The endophenotype concept in psychiatric genetics. Psychological Medicine, 37, 163–180. Foygel, D., & Dell, G.S. (2000). Models of impaired lexical access in speech production. Journal of Memory and Language, 43(2), 182–216. Friedl, K.E., Grate, S.J., Proctor, S.P., Ness, J.W., Lukey, B.J., & Kane, R.L. (2007). Army research needs for automated neuropsychological tests: Monitoring soldier health and performance status. Archives of Clinical Neuropsychology, 22S, S7–S14.

References

109

Freimer, N. & Sabatti, C. (2003). The human phenome project. Nature Genetics, 34, 15–21. Funahashi, S., Bruce, C.J., & Goldman-Rakic, P.S. (1989). Mnemonic coding of visual space in the monkey s dorsolateral prefrontal cortex. Journal of Neurophysiology, 61(2), 331–349. Geschwind, N. (1965). Disconnexion syndromes in animals and man. Brain, 88, 237–294 and 585–644. Goldstein, G. (1997). The clinical utility of standardized or flexible battery approaches to neuropsychological assessment. In G. Goldstein & T.M. Incagnoli (Eds.)Contemporary Approaches to Neuropsychological Assessment (pp.67–92). New York: Plenum Press. Goldstein, G. (1998). Introduction to neuropsychological assessment. In G. Goldstein, P.D. Nussbaum & S. R. Beers (Eds.) Neuropsychology (pp. 1–8). New York: Plenum Press. Goldstein, G., Nussbaum, P.D., & Beers, S.R. (Eds.) (1998). Neuropsychology. New York: Plenum Press. Goodglass, H. (1993). Understanding Aphasia. San Diego, California: Academic Press, Inc. Goodglass, H. & Kaplan, E. (1979). Assessment of cognitive deficit in the brain-injured patient. In M.S. Gazzaniga (Ed.) Handbook of Behavioral Neurobiology: Volume 2, Neuropsychology (pp. 3–21). New York: Plenum Press. Gorin, F., Hogarth, M., & Gertz, M. (2001). The challenges and rewards of integrating diverse neuroscience information. The Neuroscientist, 7(1), 18–27. Gottesman, I.I. & Gould, T.D. (2003). The endophenotype concept in psychiatry: Etymology and strategic intentions. American Journal of Psychiatry, 160(4), 636–645. Grill-Spector, K. & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649–677. Gruber, T.R. (1995). Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies, 43, 907–928. Guenther, F.H. (1994). A neural network model of speech acquisition and motor equivalent speech production. Biological Cybernetics, 72, 43–53. Guenther, F.H. (1995). Speech sound acquisition, coarticulation, and rate effects in a neural network model of speech production. Psychological Review, 102, 594–621. Guenther, F.H. (2006). Cortical interactions underlying the production of speech sounds. Journal of Communication Disorders, 39, 350–365. Guenther, F.H., Ghosh, S.S., & Tourville, J.A. (2006). Neural modeling and imaging of the cortical interactions underlying syllable production. Brain and Language, 96(3), 280–301. Guenther, F.H., Hampson, M., & Johnson, D. (1998). A theoretical investigation of reference frames for the planning of speech movements. Psychological Review, 105(4), 611–33. Gur, R.C., Ragland, J.D., Moberg, P.J., Turner, T.H., Bilker, W.B., Kohler, C., Siegel, S.J., & Gur, R.E. (2001). Computerized neurocognitive scanning: I. Methodology and validation in healthy people. Neuropsychopharmacology, 25, 766–776. Hall, K.M. (1997). Establishing a national traumatic brain injury information system based upon a unified data set. Archives of Physical Medicine and Rehabilitation, 78, S5–S11 Halligan, P.W. & Marshall, J.C. (1991). Figural modulation of visuo-spatial neglect: A case study. Neuropsychologia, 29(7), 619–628. Halligan, P.W. & Marshall, J.C. (1995). Lateral and radial neglect as a function of spatial position: A case study. Neuropsychologia, 33(12), 1697–702. Halligan, P.W. & Marshall, J.C. (1994). Figural perception and parsing in visuo-spatial neglect. Neuroreport, 5, 537–539. Halligan, P.W. & Marshall, J.C. (1998). Visuospatial neglect: The ultimate deconstruction. Brain and Cognition, 37(3), 419–438. Heilman, K.M., Watson, R.T., Valenstein, E. (1993). Neglect and related disorders. In K. Heilman, M. Kenneth, & E. Valenstein (Eds.) Clinical Neuropsychology (3rd ed.) (pp. 279–336). New York: Oxford University Press. Heuttel, S.A., Song, A.W., & McCarthy, G. (2004). Functional Magnetic Resonance Imaging. Sunderland, MA: Sinauer Associates.

110

References

Hickok, G., Bellugi, U., & Klima, E.S. (1996). The neurobiology of sign language and its implications for the neural basis of language. Nature, 381(6584), 699–702. Hines, M. (1994). The NEURON simulation program. In J. Skrzypek (Ed.) Neural Network Simulation Environments (pp. 147–163). Norwell, MA: Kluwer. Hines, M.L. (1998). The neurosimulator NEURON. In C. Koch & I. Segev (Eds.) Methods in Neuronal Modeling (pp. 129–136). Cambridge, MA: The MIT Press. Hines, M.L. & Carnevale, N.T. (2001). NEURON: A tool for neuroscientists. The Neuroscientist, 7(2), 123–135. Hines, M.L. & Carnevale, N.T. (2003). The NEURON simulation environment. In M.A. Arbib (Ed.) The Handbook of Brain Theory and Neural Networks (2nd ed.) (pp. 769–773). Cambridge, MA: The MIT Press. Hobbs, A. (1999). Mapping variation in brain structure and function: Implications for rehabilitation. The Journal of Head Trauma Rehabilitation, 14(6), 616–621. Hubel, D.H. & Wiesel, T.N. (1962). Receptive fields, binocular interaction and functional architecture in the cat s visual cortex. Journal of Physiology, 160(1), 106–154. Hubel, D.W. and Wiesel, T.N. (1978) Anatomical demonstration of orientation columns in macaque monkey. Journal of Comparative Neurology, 177, 361–379. Huang, H.C. & Wang, T.Y. (2005). Toward a graphical analysis tool for computer-assisted assessment of visual search patterns. Paper presented at the Fifth IEEE International Conference on Advanced Learning Technologies, Kaohsiung, Taiwan. Huang H.C. & Wang T.Y. (2008). Visualized representation of visual search patterns for a visuospatial attention test. Behavioral Research Methods, 40(2), 383–90. Huerta, M.F. & Koslow, S.H. (1996). Neuroinformatics: Opportunities across disciplinary and national borders. Neuroimage, 4, S4–S6. Huerta, M.F., Koslow, S.H., & Leshner, A.I. (1993). The human brain project: An international resource. Trends in Neuroscience, 16(11), 436–438. Jagaroo, V. (1998). Allocentric and representational subprocesses in visuospatial function: Four focal lesion groups. Unpublished doctoral dissertation, Boston University School of Medicine, Boston. Jagaroo, V. (1999). Towards an analytic framework for the visuospatial domain: Spatial reference frames, cognitive operations and neural systems.The Journal of Clinical and Experimental Neuropsychology 21(1): 134–146. Jagaroo, V. (2002). Dynamic computational visual field matrices: A computerized mapping system for the analysis of visual perception, spatial processing and featural recognition. In C.H. Dagli, A.L. Buczak, J. Ghosh, M.J. Embrechts, O. Ersoy, & S.W. Kercel (Eds.) Smart Engineering System Design, Neural Networks and Fuzzy Logic,vol. 12 (pp. 81–90). New York: American Society of Mechanical Engineers Press. Jagaroo, V. (2004). Mental rotation and the parietal question in functional neuroimaging: A discussion of two views. European Journal of Cognitive Psychology, 16(5), 717–728. Jagaroo, V. & Wilkinson, K. (2008). Further considerations of visual cognitive neuroscience in aided AAC: The potential role of motion perception systems in maximizing design display. Augmentative and Alternative Communication, 24(1), 29–42. Jensen M. (2007). The new metrics of scholarly authority. The chronicle of higher education (The chronicle review), 53(41), B6 (June 15). Jha, A.K., Ferris, T.G., Donelan, K., DesRoches, C., Shields, A., Rosenbaum, S. & Blumenthal, D. (2006). How common are electronic health records in the United States? A summary of the evidence. Health Affairs, 25, 496–507 Jha, A.K., DesRoches, C., Campbell, E.G., Donelan, K., Rao, S.R., Ferris, T.G., Shields, A., Rosenbaum, S., & Blumenthal, D. (2009). Use of electronic health records in U.S. hospitals. The New England Journal of Medicine, 360(16), 1628–1638. Johnston, M.V. (1997). Toward a national information system for traumatic brain injury: Foreword and overview. Archives of Physical Medicine and Rehabilitation, 78(8), S1–S4.

References

111

Jones, B.L. (2008). Web 2.0 heroes: Interviews with 20 web 2.0 influencers. Indianapolis, IN: Wiley Publishing, Inc. Jonides J., Smith E.E., Koeppe R.A., Awh E., Minoshima S., & Mintun M.A. (1993). Spatial working memory in humans as revealed by PET. Nature,363(6430), 623–625. Just, M.A. & Carpenter, P.A. (1985). Cognitive coordinate systems: Accounts of mental rotation and individual differences in spatial ability. Psychological Review, 92(2), 137–172. Kalaska, J.F. (1991). Parietal cortex area 5: A neuronal representation of movement kinematics for kinaesthetic perception and movement control? In J. Paillard (Ed.) Brain and Space (pp. 133– 146). New York: Oxford University Press. Kane (2007). Editorial. Archives of Clinical Neuropsychology, 22S: S3–S5. Kane, R.L. & Kay, G.G. (1992). Computerized assessment in neuropsychology: A review of tests and test batteries. Neuropsychology Review, 3(1), 1–117. Kane, R.L. & Kay, G.G. (1997). Computer applications in neuropsychological assessment. In G. Goldstein & T.M. Incagnoli (Eds.) Contemporary Approaches to Neuropsychological Assessment: Critical Issues in Neuropsychology (pp. 359–392). New York: Plenum Press. Kane, R.L. & Reeves, D.L. (1997). Computerized test batteries. In A.M. Horton, D. Wedding & J. Webster (Eds.) The Neuropsychology Handbook (pp. 423–467). New York (NY): Springer. Kanwisher, N. (2000). Domain specificity in face perception. Nature Neuroscience, 3(8), 759–763. Kanwisher, N., McDermott, J., & Chun, M. (1997). The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302–4311. Kaplan, E. (1988). A process approach to neuropsychological assessment. In T. Boll & B.K. Bryant (Eds.) Clinical Neuropsychology and Brain Functioning: Research, Measurement, and Practice (pp. 129–167). Washington, DC: American Psychological Association. Klerman, G.L., Vaillant, G.E., Spitzer, R.L., & Michels, R. (1984). A debate on DSM-III. American Journal of Psychiatry, 141(4), 539–553. Klima, E.S. & Bellugi, U. (1979). The Signs of Language. Cambridge, MA: Harvard University Press. Kolb, B. & Whishaw, I.Q. (2003). Fundamentals of Human Neuropsychology. New York: Worth Publishers. Koslow, S.H. (2000). Should the neuroscience community make a paradigm shift to sharing primary data? Nature Neuroscience, 3(9), 863–865. Koslow, S.H. (2002). Sharing primary data: A threat of asset to discovery? Nature Reviews: Neuroscience, 3, 311–313. Kosslyn, S.M. (1987). Seeing and imagining in the cerebral hemispheres: A computational approach. Psychological Review, 94(2), 148–175. Kosslyn, S.M., Chabris, C.F., Marsolek, C.J., & Koenig, O. (1992). Categorical versus coordinate spatial relations: Computational analysis and computer simulations. Journal of Experimental Psychology: Human Perception and Performance, 18(2), 562–577. Kötter, R. (2001). Neuroscience database: Tools for exploring brain structure-function relationships. Philosophical Transactions of the Royal Society of London, 356(1412), 1111–1120. Kremen, W.S., Eisen , S.A., Tsuang, M.T., & Lyons, M.J. (2007). Is the wisconsin card sorting test a useful neurocognitive endophenotype? American Journal of Medical Genetics. Part B, Neuropsychiatric Genetics, 5(144B), 403–406. Lainhart, J.E., Bigler, E.D., Bocian, M., Coon, H., Dinh, E., Dawson, G., Deutsch, C.K., Dunn, M., Estes, A., Tager-Flusberg, H., Folstein, S., Hepburn, S., Hyman, S., McMahon, W., Minshew, N., Munson, J., Osann, K., Ozonoff, S., Rodier, P., Rogers, S., Sigman, M., Spence, M.A., Stodgell, C.J., & Volkmar, F. (2006). Head circumference and height in autism: A study by the collaborative program of excellence in autism. American Journal of Medical Genetics 1A(140), 21:2257–2274. Landauer, T.K. & Dumais, S.T. (1997). A solution to plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240.

112

References

Landauer, T.K., Foltz, P.W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259–284. Landauer, T.K., McNamara, D.S., Dennis, S., & Kintsch, W. (2007). Handbook of Latent Semantic Analysis. Mahwah, NJ: Lawrence Erlbaum Associates Publishers. Lavraˇc, N. (1999). Selected techniques for data mining in medicine. Artificial Intelligence in Medicine, 16(1), 3–23. Lee, A. Thompson, P. & Toga, A. (2000). Databasing the brain. Nature, 406, 822–825. Levin, B.L. (1990). Spatial cognition in parkinson disease. Alzheimer’s Disease and Associated Disorders, 4(3), 161–170. Lezak, M.D., Howieson, D.B. & Loring, D.W. (2004). Neuropsychological assessment 4th edition. New York: Oxford University Press. Lezak, M.D. (1995). Neuropsychological assessment. 3rd edition. New York: Oxford University Press. Lichtheim, L. (1885). On aphasia. Brain, 7, 433–484. Lieberman, P. (2002). Human Language and our Reptilian Brain. Cambridge, MA: Harvard University Press. Luciana, M. (2003). Practitioner review: Computerized assessment of neuropsychological function in children: Clinical and research applications of the Cambridge neuropsychological testing automated battery (CANTAB).Journal of Child Psychology and Psychiatry, 44(5), 649–663. MacLaughlin, D., Neidle, C., & Greenfield, D. (2000). SignStreamTM User’s Guide, Version 2.0. Boston, MA: American Sign Language Linguistic Research Project No. 9, Boston University. Mahner, M. & Kary, M. (1997). What exactly are genomes, genotypes and phenotypes? And what about phenomes? Journal of Theoretical Biology, 7(186), 55–63. Marshall, J.C. (1986). The description and interpretation of aphasic language disorder. Neuropsychologia, 24(1), 5–24. Matthysse, S., Levy, D.L., Kinney, D., Deutsch, C., Lajonchere, C., Yurgelun-Todd, D., Woods, B., & Holzman, P.S. (1992). Gene expression in mental illness: A navigation chart to future progress. Journal of Psychiatric Research, 26(4), 461–473. McClelland, J.D., Rumelhart, & the PDP Research Group (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations. Cambridge, MA: The MIT Press. McCullough S., Emmorey K., & Sereno M. (2005). Neural organization for recognition of grammatical and emotional facial expressions in deaf ASL signers and hearing nonsigners. Brain Research: Cognitive Brain Research, 22(2), 193–203. McNeill, D. & Pedelty, L.L. (1995). Right brain and gesture. In K. Emmorey & J. Reilly (Eds.), Language, Gesture and Space (pp. 63–87). New Jersey: Lawrence Erlbaum Associates. Meadche, A. (2002). Ontology learning for the semantic web. Boston: Kluwer Academic Publishers. Mesulam, M.M. (1990). Large-scale neurocognitive networks and distributed processing for attention, language, and memory. Annals of Neurology, 28, 597–613. Miller, C. (2001). Some reflections on the need for a common sign notation. Sign Language & Statistics, 4, 11–28. National Academy Press (1999). A Question of Balance: Privacy Rights and the Public Interest in Scientific and Technical Databases. National Research Council, Washington, D.C. Neidle, C. (2001). SignStreamTM : A database tool for research on visual-gestural language. Sign Language & Linguistics, 4, 203–214. Neidle, C. (2002b). SignStreamTM Annotation: Conventions used for the American sign language linguistic research project. Boston, MA: American Sign Language Linguistic Research Project Report No. 11, Boston University. Neidle, C. (2002a). SignStreamTM : A Database Tool for Research on Visual-Gestural Language. Journal of Sign Language and Linguistics, 4, 203–214.

References

113

Neidle, C., Sclaroff, S., & Athitsos, V. (2001). SignStreamTM : A tool for linguistic and computer vision research on visual-gestural language data. Behavior Research Methods and Instrumentation, 33(3), 311–320. Newcombe, F. and Ratcliff, G. (1990) Disorders of visuospatial analysis. In F. Boller & J. Grafman (Eds.) Handbook of Neuropsychology, Vol. 2. (pp. 333–356). Amsterdam: Elsevier. Nichelli, P., Rinaldi, M., & Cubelli, R. (1989). Selective spatial attention and length representation in normal subjects and in patients with unilateral spatial neglect. Brain and Cognition, 9(1), 57–70. O’Reilly, R.C., Munakata, Y., & McClelland, J.L. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. Cambridge, MA: The MIT Press. Organization for Economic Co-operation and Development (2002). Report on Neuroinformatics by The Global Science Forum Neuroinformatics Working Group (June). Pechura, C.M. & Martin, J.B. (Eds.) (1991). Mapping the brain and its functions: Integrating enabling technologies into neuroscience. Institute of Medicine (U.S.). Committee on a National Neural Circuitry Database. Washington, D.C.: National Academy Press. Perrett, D.I., Hietanen, J.K., Oram, M.W., Benson, P.J., & Rolls, E.T. (1992). Organization and functions of cells responsive to faces in the temporal cortex. Philosophical Transactions of the Royal Society of Biological Sciences, 335(1273), 23–30. Petersen, S.E., Fox, P.T., Posner, M.I., Mintum, M., & Raichle, M.E. (1988). Positron emission tomographic studies of the cortical anatomy of single-word processing. Nature, 331, 585–589. Petitto, L.A. & Bellugi, U. (1988). Spatial cognition and brain organization: Clues from the acquisition of a language in space. In J. Stiles-Davis, M. Kritchevsky, & U. Bellugi (Eds.) Spatial Cognition: Brain Bases and Development (pp.299–327). New Jersey: Lawrence Erlbaum Associates. Piatetsky-Shapiro, G., & Frawley, W.J. (Eds.) (1991). Knowledge Discovery in Databases (AAAI press copublication). Cambridge, MA: The MIT Press. Pinker, S. (1994). The Language Instinct: How the Mind Creates Language. New York: William Morrow and Company. Pizzuto, E., & Pietrandrea, P. (2001). The notation of signed texts: Open questions and indications for further research. Sign Language & Linguistics, 4, 29–45. Potter, J., Deighton, T., Patel, M., Fairhurst M, Guest R, & Donnelly N. (2000). Computer recording of standard tests of visual neglect in stroke patients. Clinical Rehabilitation, 14, 441–446. Poizner, H., Kaplan, E., Bellugi, U., & Padden, C.A. (1984). Visual-spatial processing in deaf brain-damaged signers. Brain and Cognition, 3(3), 281–306. Prather, J.C., Loback, D.F., Goodwin, L.K., Hales, J.W., Hage, M., & Hammond, W.E. (1997). Medical data mining: Knowledge discovery in a clinical data warehouse. Proceedings: A Conference of the American Medical Informatics Association. AMIA Fall Symposium, 101–105. R Genogram: Historical Reeves, D. L., Winter, K. P., Bleiberg, J., & Kane, R. L. (2007). ANAM perspectives, description, and current endeavors. Archives of Clinical Neuropsychology, 22S, S15–S37. Reeves, D., Elsmore, T., Wiederhold, M.D., Wood, D., Center, C., Spira, J., & Wiederhold, B.K. (2007). Brain checkers: Handheld computerized neuropsychological assessment in a virtual reality treatment protocol for combat PTSD. Annual Review of CyberTherapy and Telemedicine: Transforming Healthcare Through Technology, 5: 151–156. Reitan, R.M. (1964). Psychological deficits resulting from cerebral lesions in man. In J.M. Warren & K. Akert (Ed.), The Frontal Granular Cortex and Behavior (pp. 295–312). New York: McGraw-Hill Book Company. Richter, W., Ugurbil, K., Georgopoulos, A., Kim, S-G. (1991) Time-resolved fMRI of mental rotation. Neuroreport, 8(17), 3697–3702. Roland, P.E. & Friberg, L. (1985). Localization of cortical areas activated by thinking. Journal of Neurophysiology, 53(5), 1219–1243.

114

References

Rowe, P.M. (1995). Neural networks: A bridge between neuroscience and psychology. Molecular Medicine Today 1(4), 168–173. Sabb, F.W., Bearden, C.E., Glahn, D.C., Parker, D.S., Freimer, N., & Bilder, R.M. (2008). A collaborative knowledge base for cognitive phenomics. Molecular Psychiatry, 13, 350–360. Sadler, T.W. (2004). Langman’s Medical Embryology, 9th Ed. Philadelphia, PA: Lippincott, Williams & Wilkins. Sadler, J.Z., Hulgus, Y.F. & Agich, G.J. (1994). On values in recent American psychiatric classification. Journal of Medical Philosophy 19(3), 261–277. Schatz, P., Browndyke, J. (2002). Applications of computer-based neuropsychological assessment. The Journal of Head Trauma Rehabilitation, 17(5), 395–410. Schlegel, R.E., & Gilliland, K. (2007). Development and quality assurance of computer-based assessment batteries. Archives of Clinical Neuropsychology, 22S, S49–S61. Schultheis, M.T., Himelstein, J., & Rizzo, A.A. (2002). Virtual reality and neuropsychology: Upgrading the current tools. The Journal of Head Trauma Rehabilitation, 17(5), 378–394. Sergent, J. (1982). The cerebral balance of power: Confrontation or cooperation? Journal of Experimental Psychology: Human Perception and Performance, 8(2), 253–272. Sergent, J. (1991). Judgments of relative position and distance on representations of spatial relations. Journal of Experimental Psychology, Human Perception and Performance, 17(3), 762–780. Shepherd, G.M., Mirsky, J.S., Healy, M.D. et al. (1998). The human brain project: Neuroinformatics tools for integrating, searching and modeling multidisciplinary neuroscience data. Trends in Neuroscience, 21(11), 460–468. Small, M., Cowey, A., & Ellis, S. (1994). How lateralised is visuospatial neglect? Neuropsychologia, 32(4), 449–464. Smith, E.E. & Jonides, J. (1998). Neuroimaging analyses of human working memory. Proceedings of the National Academy of Science, 95, 12061–12068. Spitzer, R.L., Endicott, J., & Robins, E. (1975). Clinical criteria for psychiatric diagnosis and DSM-III.American Journal of Psychiatry, 132(11), 1187–1192. Spitzer, R.L., Williams, J.B., & Skodol, A.E. (1980). DSM-III: The major achievements and an overview. American Journal of Psychiatry, 137(2), 151–164. Spreen, O. & Strauss, E. (Eds.). (1998). A Compendium of Neuropsychological Tests: Administration, norms, and commentary. New York: Oxford University Press. Staab, S. & Studer, R. (2009). Handbook on Ontologies. New York: Springer. Stein, J.F. (1991). Space and the parietal association areas. In J. Paillard (Ed.) Brain and Space (pp. 185–222). New York: Oxford University Press. Sternberg, S. (1966). High-speed scanning in human memory. Science, 153(736), 652–654. Strauss, E., Sherman, M.S., & Spreen, O. (2006). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary. New York: Oxford University Press. Talairach, J. & Tournoux, P. (1993). Referentially Oriented Cerebral MRI Anatomy: An Atlas of Stereotaxic Anatomical Correlations for Gray and White Matter. New York: Thieme Medical Publishers. Talairach, J. & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system – an approach to cerebral imaging. New York: Thieme Medical Publishers. Tarr, M.J. & Gauthier, I. (2000). FFA: A flexible fusiform area for subordinate-level visual processing automatized by expertise. Nature Neuroscience, 3(8), 764–769. Thorne, D.R. (2006). Throughput: A simple performance index with desirable characteristics. Behavior Research Methods, 38(4), 569–573. Thorne, D.R., Genser, S.G., Sing, H.C., & Hegge, F.W. (1985). The Walter Reed performance assessment battery. Neurobehavioral Toxicology & Teratology, 7, 415–418. Treisman, A.M. (1982). Perceptual grouping and attention in visual search for features and for objects. Journal of Experimental Psychology: Human Perception and Performance, 8(2), 194–214.

References

115

Treisman, A.M. & Gelade, G. (1980). A Feature Integration Theory of Attention. Cognitive Psychology, 12, 97–136. Ungerleider, L.G. & Mishkin, M. (1982). Two Cortical Visual Systems. In D.J. Ingle, M.A. Goodale, & R.J.W. Mansfield (Eds.) Analysis of Visual Behavior (pp. 549–586). Cambridge: The MIT Press. Valler, G., Guariglia, C., Nico, D., & Bisiach, E. (1995). Spatial hemineglect in back space. Brain, 118, 467–472. Van Essen, D.C., Felleman, D.J., DeYoe, E.A., Olavarria, J., & Knierim, J. (1990). Modular and Hierarchical Organization of Extrastriate Visual Cortex in the Macaque Monkey. Cold Spring Harbor Symposia on Quantitative Biology, Vol. LV: 679–696. Cold Spring Harbor, NY: Cold Spring Harbor Laboratory Press. Van Horn, J.D. & Gazzaniga, M.S. (2002). Databasing fMRI studies–towards a ‘discovery science’ of brain function. Nature Reviews, Neuroscience, 3, 314–318. Vossen, G. & Hagemann, S. (2007) Unleashing web 2.0: From Concepts to Creativity. Burlington, MA: Morgan Kaufmann Publishers Walsh, K. (1987). Neuropsychology: A Clinical Approach. New York: Churchill Livingstone. Wang, T.Y., Huang, H.C., & Huang, H.S. (2006). Design and implementation of cancellation tasks for visual search strategies and visual attention in school children. Computers & Education, 47, 1–16. Wang, Y. (2002). On Cognitive Informatics (Keynote Speech). Proceedings 1st IEEE International Conference on Cognitive Informatics (ICCI ’02). Calgary, Canada: IEEE CS Press, pp. 34–42. Wang, Y. (2003a). Cognitive informatics: A new transdisciplinary research field. Brain and Mind: A Transdisciplinary Journal of Neuroscience and Neurophilosophy, 4(2), 115–127. Wang, Y. (2003b). On cognitive informatics. Brain and Mind: A Transdisciplinary Journal of Neuroscience and Neurophilosophy, 4(2), 151–167. Wang, Y. (2006). Cognitive informatics towards the future generation computers that think and feel. In Proceedings of the 5th IEEE International Conference on Cognitive Informatics (ICCI’06) (pp. 3–7). Beijing, China: IEEE CS Press. Wang, Y. (2007a). Cognitive informatics: Exploring the theoretical foundations for natural intelligence, neural informatics, autonomic computing, and agent systems. International Journal of Cognitive Informatics and Natural Intelligence, 1(1), i–x. Wang, Y. (2007b). The theoretical framework of cognitive informatics. International Journal of Cognitive Informatics and Natural Intelligence, 1(1), 1–27. Wernicke, C. (1874). Der Aphasische Symptomencomplex. Breslau: Cohn & Weigert. Williams, J.B. & Spitzer, R.L. (1982). Research diagnostic criteria and DSM-III: An annotated comparison. Archives of General Psychiatry, 39(11), 1283–1289. Wilson, S.L., & McMillan, T.M. (1992). Computer-based assessment in neuropsychology. In J. R. Crawford, D.M. Parker, & W. W. McKinlay (Eds.) A Handbook of Neuropsychological Assessment (pp. 413–431). Hove, UK: Lawrence Erlbaum Associates, Wong, S.T.C. & Koslow, S.H. (2001). Human brain program research progress in bioinformatics/neuroinformatics. Journal of the American Medical Informatics Association, 8(5), 510–511. Wong, S.T.C., Hoo, K.S., Cao, X., Tjandra, D., Fu, J.C., & Dillon, W.P. (2004). A neuroinformatics database system for disease-oriented neuroimaging research. Academic Radiology, 11(3), 345–358. Wong, S.T.C., Hoo, K.S., Knowlton, R.C., Laxer, K.D., Cao, X., Hawkins, R.A., et al. (2002). Design and applications of a multimodality image data warehouse framework. Journal of American Medical Informatics Association, 9(3), 239–254. Wong, S.T.C. & Huang, H.K. (1996). Design methods and architectural issues of integrated medical image data base systems. Computerized Medical Imaging and Graphics, 20(4), 285–299. Young, A.W., Hellawell, D.J., & Welch, J. (1992). Neglect and visual recognition. Brain, 115 Pt 1, 51–71.

116

References

Zhang, S., Zhang, C., & Wu, X. (2004). Knowledge Discovery in Multiple Databases. New York: Springer. Zobel, A. & Maier, W. (2004). Endophänotypen-ein neues Konzept zur biologischen characterisierung psychischer störungen. Der Nervenarzt, 75(3), 205–214. Zupan, B., Lavraˇc, N., & Keravnou, E. (1999). Data mining techniques and applications in medicine. Artificial Intelligence in Medicine, 16, 1–2.

B. Reference List (All websites / URLs) American Recovery and Reinvestment Act (2009). http://www.recovery.gov. Cited 30 Mar 2009 Athinoula A. Martinos Center for Biomedical Imaging. (2009). FreeSurfer. http://www.nmr.mgh. harvard.edu. Cited 15 May 2009. Barras, C. (2008). Brain implant helps stroke victim speak again. In: New Scientist (Tech). Brain implant helps stroke victim speak again. http://www.newscientist.com/article/dn14277-brainimplant-helps-stroke-victim-speak-again.html. Cited 11 Feb 2009. Biomedical Informatics Research Network. (2005). http://www.nbirn.net. Cited 22 Nov 2005. Boston University American Sign Language Linguistic Research Project. (2008). Database Access Interface. http://ling.bu.edu/asllrpdata/queryPages/) Cited 26 Sept 2008. Boston University American Sign Language Linguistic Research Project. (2008). SignStream. http://www.bu.edu/asllrp/SignStream/. Cited 26 Sept 2008. Boston University Department of Cognitive & Neural Systems, CNS Speech Lab. (2008). The DIVA Model. http://speechlab.bu.edu/diva.php. Cited 9 Jul 2008. Computational Systems Neuroscience Group. (2005). CoCoMac (Collations of Connectivity data on the Macaque brain). http://cocomac.org/home.asp. Cited 30 Nov 2005 Deutsch, C. (2008). Collaborative biological research in schizophrenia: Dysmorphology in schizophrenia. http://www.umassmed.edu/shriver/research/psychological/projects/cbrsds.aspx. Cited 8 May 2008. Duke University. (2005). NEURON for computer simulations of neurons and neural networks. http://neuron.duke.edu/. Cited 6 Dec 2005 ERUDIT. (2008). ERUDIT Aphasia tutorial. http://fuzzy.iau.dtu.dk/aphasia.nsf/htmlmedia/ index.html. Cited 21 Jul 2008. GENESIS. (2005). General neural simulation system. http://www.genesis- sim.org/GENESIS/. Cited Dec 6 2005 GENI. (2008). Global Environment for Network Innovations. http://www.geni.net/. Cited 9 Feb 2009 Information Sciences Institute, University of Southern California. (2009). Neuroscholar. http://www.neuroscholar.org/. Cited Jan 15 2009 International Consortium for Brain Mapping. (2005). http://www.loni.ucla.edu/ICBM/. Cited Nov 16 2005 International Neuroinformatics Coordinating Facility. (2006). The neuroinformatics site.org. http://www.incf.org. Cited 8 Dec 2006 International Neuroinformatics Coordinating Facility. (2006). The neuroinformatics site.org. http://www.neuroinf.org. Cited 8 Dec 2006 International Neuroinformatics Coordinating Facility. (2006). G-Node (German Neuroinformatics Node). http://www.neuroinf.de. Cited 8 Dec 2006 Jantzen J. (1999a). Aphasia and classification methods [PowerPoint slides]. Cited from http://fuzzy.iau. dtu.dk/tutor/apha/lecture1/index.htm. Cited 21 Jul 2008 Jantzen J. (1999b). Second test: Other features [PowerPoint slides]. Cited from http://fuzzy.iau. dtu.dk/tutor/apha/lecture3/index.htm. Cited 21 Jul 2008 Johnson, K.A. & Becker J.A. (1999). The whole brain atlas. http://www.med.harvard.edu/ AANLIB/home.html. Cited 1 Dec 2006

References

117

Kelly, M.P. Walter Reed Army Medical Center (2001). Internet Enabled Aviation Neuropsychological Assessment. http://www.stormingmedia.us/59/5911/A591104.html. Cited 15 May 2009 Kelly, M.P. Walter Reed Army Medical Center (2001). Internet Enabled Aviation Neuropsychological Assessment. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix= html&identifier=ADA401195 Cited 15 May 2009 Kibbe, D. & Klepper, B. (2008). An open letter to the Obama health team. http://www. thehealthcareblog.com/the_health_care_blog/2008/12/where-should-fe.html. Cited 2 Jan 2009 Lee, Y.S. (2007). Scientists seek to help ‘locked-in’ man speak. In: CNN Health. CNN. http://www.cnn.com/2007/HEALTH/conditions/12/14/locked.in/index.html. Cited 11 Feb 2009 McLean Hospital Mailman Research Center Psychology Research Lab. (2008). http://www.mclean.harvard.edu/research/mrc/psychlab.php. Cited 8 May 2008. National Institutes of Health. (2005). National Human Genome Research Institute. http://www.genome. gov/. Cited 9 Nov 2005 National Institutes of Health MRI Study of Normal Brain Development. http://www.bic.mni. mcgill.ca/nihpd/info/index.html. Cited 26 Jan 2009 National Institute of Mental Health. (2008). Analysis of functional neuroimages (AFNI). http://afni.nimh.nih.gov/. Cited 10 June 2008 National Institute of Mental Health. (2008). Neuroimaging informatics technology initiative. http://nifti.nimh.nih.gov/ Cited 10 June 2008 National Institute of Mental Health. (2005). Neuroinformatics: The human brain project. http://www.nimh.nih.gov/neuroinformatics/index.cfm. Cited 10 Nov 2005 National Public Radio (2009). All Things Considered: “Electronic Medical Record Change Not Easy” http://www.npr.org/templates/story/story.php?storyId=102360638 Cited Mar 25 2009 National Rehabilitation Hospital (2009) http://www.nrhrehab.org/Research/Projects/ATRC+2000/ ANAM/default.aspx. Cited 15 May 2009 Obama B. (2008). President-elect Barack Obama lays out key parts of economic recovery plan. http://change.gov/newsroom/entry/the_key_parts_of_the_jobs_plan/. Cited 14 Jan 2009. O’Reilly, T. (2005). What is web 2.0: Designing patterns and business models for the next generation of software. http://oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web20.html. Cited 16 Jan 2009 Organization for Economic Co-operation and Development. (2006). OECD Global science forum, neuroinformatics working group. http://www.neuroinformatics.nl/. Cited 1 Dec 2006 Press, B. & Olshausen, B. (2005). Xanat 2.0: A Graphical anatomical database. http://redwood. psych.cornell.edu/bruno/xanat/xanat.html. Cited 30 Nov 2005. Society of Neuroinformatics for Neuropsychology. http://www.scnn.org. Cited 15 Jun 2009. University of California-Davis. (2005). BrainMaps.org. http://brainmaps.org/ Cited 29 Nov 2005 University of California Los Angeles Center for Cognitive Phenomics. (2008). PubBrain. http://www.pubbrain.org/ Cited 6 Oct 2008 University of California Los Angeles Center for Cognitive Phenomics. (2008). PubGraph. http://www.pubgraph.org/ Cited 6 Oct 2008 University of California Los Angeles Semel Institute Consortium for Neuropsychiatric Phenomics. (2008). http://phenomics.ucla.edu. Cited 6 Oct 2008 University of California Los Angeles Semel Institute Consortium for Neuropsychiatric Phenomics. (2008). http://www.phenowiki.org/index.php5/Main_Page. Cited 6 Oct 2008 University of Texas at Austin Center for Learning and Memory. (2009). Synapse Web. http://synapses.clm.utexas.edu/learn/filo3D/howto.stm. Cited Jan 15 2009 US Army. (2009). Automated Neuropsychological Assessment Metrics. http://www.army medicine.army.mil/prr/anam.html. Cited 15 May 2009 U.S. Department of Energy Office of Science. (2005). Genomics programs of the U.S. Department of Energy Office of Science. http://genomics.energy.gov/ Cited 9 Nov 2005

118

References

Washington University in St. Louis. (2008). Computerized Anatomical Reconstruction and Editing Toolkit (CARET). http://brainmap.wustl.edu/. 6 June 2008 Wikipedia. (2009) Web 2.0. http://en.wikipedia.org/wiki/Web_2.0. Cited 16 Jan 2009 Yale University. (2008). NEURON for empirically-based computer simulations of neurons and neural networks. http://www.neuron.yale.edu/neuron/. Cited Dec 6 2005

Index

Note: Page numbers followed by f indicate figures; t, tables; n, notes.

A Aachen Aphasia test (AAT), 68 Adaptive networks, 17 Algorithm types in neural modeling, 17 Amari, Shun-ichi, 23 American Sign Language Linguistic Research Project, 70–71 Analysis of Functional Neuroimages (AFNI), 8–9 Analytic module (CVFN), 49, 51 f ANAM Access Database (AADB), 37–38 Animated priming effects, 53–54 Anomic aphasia, 68–69 Aphasia Database, The (AD) description and background of, 67 neural modeling and diagnoses with, 68–69 rationale and goals of, 67 structure and contents of, 68 tabular summary of, 30t Aphasia Modeling Project, 73 Aphasia/aphasic syndromes neuroinformatic applications for, 6, 58–61 types of, 68–69 Arbib, M.A., 59, 60 Artificial intelligence (AI), 16–17 Assessment, neuropsychological cognitive, 41–42 conventional options for, 28 internet-enabled and remote, 38 portable systems for, 38, 40–41 visuospatial, 44–46 Assessment batteries neuroinformatic applications for, 5–6 Wechsler battery of tests, 27–28 Attention deficit hyperactivity disorder (ADHD), 81 Autism, 77

Automated Neuropsychological Assessment Metrics (ANAM) description and background of, 35–38 informatics incorporated into, 36–41 query functions in, 38, 39 f in shared platform, 43 tabular summary of, 30t tabular types in, 36–37 B Back-propagation, 17 Baggett, M., 42 Behavioral data in children and adolescents, 11 Benton, A.L., 91 Bilder, R., 4, 75, 90–91, 101 Bioinformatics, defined, 1 Biomedical Informatics Research Network (BIRN), downloadable software from, 9–10 Bipolar disorder, 81 Boston Aphasia Diagnostic Exam, 60 Boston Visuospatial Quantitative Battery, 48 Bota, M., 60 Brain cortices of visuospatial function, 44–45 dysmorphology in, 77–78 function-specific areas of, 45 See also Mapping, brain Brain imaging construction, analysis, and morphometric tools for, 8–10 databases for, 10–12 five categories of, 10 BrainCheckers software package, 40 Broca, P., 58 Broca’s aphasia, 69

119

120 Broca-Wernicke Model of Language, 58 Browndyke, J., 26–27 C C. elegans (Caenorhabditis elegans), 22 California Verbal Learning Test II, 27 Caplan, D., 59 CellBuilder software package, 15 Children and adolescents behavioral data in, 11–12 brain development databases for, 11–12 Chimaeric stimuli, 52 Christensen, D., 42 Chute, D.L., 27 c ClinicView software package, 38 A Cognitive Analysis (CNP), 81 Cognitive assessment, 41–42 Cognitive informatics, 16n Cognitive neuroscience background of, 99 visuospatial research in, 46 CogScreen software package, 42 Collations of Connectivity Data on the Macaque Brain (CoCoMac), 12–14 Computational grid matrix, 48 f , 50–53 Computational modeling, see Neural modeling Computational Visual Field for Neglect (CVFN) coordinate address system of, 48–50 description and background of, 46–47 grid matrix advantages in, 48 f , 50–53 limitations of, 48 NI approach to, 48–50 tabular summary of, 30t Computer applications general for neuropsychology, 25–28 neuroinformatics for neuropsychology models of, 29–84, See also Neuroinformatics (NI) scope and function of, 28–29 Computer-Assisted Cancellation Test System (CACTS) description and background of, 53–54 design and architecture of, 54 four software models of, 54 kinematic and metric functions of, 54–57 outcome analysis module of, 54 tabular summary of, 30t Computerized Anatomical Reconstruction and Editing Toolkit (CARET), 9 Consortium for Neuropsychiatric Phenomics, The levels schematic in, 80 f

Index neuroinformatics in, 80–81 tools of, 81–82 Consortium for Neuropsychiatric Phenomics, The (CNP) description and background of, 78–79 mission and strategy of, 79–80 Coordinate computational kernel (CCK), 49, 51 f Coordinate recording database (CRD), 49, 51 f Cox, Robert, 8 Craniofacial dysmorphology, 76–77 Craniofacial features, neuroinformatic applications for, 6 D Data behavioral, in children and adolescents, 11 capture and manipulation of, 4t, 37 separated files for, 36 sharing issues in, 86–88 standardization of, 88–89 uniformity of output, 36 Data mining neural modeling role in, 16–17 privacy issues and, 86–88 process of, 16 sharing issues in, 89 taxonomies and ontologies for, 88 tools for, 18–19 Data processing, key aspects of, 4t Database Access Interface (DAI), 72 Databases for academic and clinical research, 18–20 of International Consortium of Brain Mapping (ICBM), 11 for mapping neural structure and connectivity patterns, 12–13 structure and functions of, 16–17 Dataman software package, 40 Demographic tables (ANAM), 38 Deutsch, C.K., 77 Diagnostic imaging, see Brain imaging; specific modalities Digitization of data, 3 Directions Into Velocities of Articulators (DIVA) description and background of, 61–62 feedback and feedforward process in speech acquisition, 63–65 feedback control subsystem, 64–65 NI role in, 66 online information on, 61 synopsis of

Index feedforward control subsystem, 63–64 infant babbling, 65 neural decoding and prosthetics for speech, 66 phonetic categories, 65 tabular summary of, 30t E Electronic medical records (EMRs), 103 Electrophysiology, 26 Embryogenesis, 77 Endophenotypes, 75 Error maps (DIVA), 64 European Network for Fuzzy Logic and Uncertainty Modeling in Information Technology (ERUDIT), 67 European Network on Intelligent Technologies for Smart Adaptive Systems (EUNITE), 67 F Felleman, D.J., 45 Fetal development, 77 Flight training, 41–42 FreeSurfer software package, 9–10 Freimer, N., 75, 76 Functional magnetic resonance imaging (fMRI) brain imaging tools for, 8–10 National fMRI Data Center, 86 speech and language data from, 59, 66 Funding sources, 20–21, 90 G Gahm, G., 42 Gazzaniga, M.S., 86–87 General Neural Simulation System (GENESIS), 14 Genomics, 2 Gestural communication, 6, 70–73 Gilliland, K., 93 Global aphasia, 69 Global Environment for Network Innovations (GENI), 102 Graphics database (CVFN), 49, 51 f Graphics kernel (CVFN), 49, 51 f Grid controller (CVFN), 49, 51 f Grillner, Sten, 23 Gyral surface contour maps, 9, 64

121 H Halligan, P.W., 47 Head injury research, 26 Header tables (ANAM), 37 Hemispatial neglect, 47 Hobbs, A., 26 Hooper Visual Organization Test, 44 Huang, H.C., 20 Human Brain Project (HBP) defined, 3 functions of, 21 overview of, 21 TBI rehabilitation in, 26 Human Genome Project (HGP), 2, 22 Human Phenome Project (HPP), 75 I Interfacing modalities for CoCoMac and CARET, 12 International Consortium for Brain Mapping (ICBM), subject database of, 11 International Neuroinformatics Coordinating Facility (INCF), 24 International Neuropsychological Society (INS), 3–4 Internet, vast influence of, 100–102 Internet-Enabled Neuropsychological Assessment of Army Aviators (IENAAA) description and background of, 41–42 informatics incorporated into, 42–46 in shared platform, 43 tabular summary of, 30t Item tables (ANAM), 37 J Jagaroo, V., 44, 47 Joseph, R.M., 77 Journal of Head Trauma Rehabilitation (2002), 26 K Kane, R.L., 92 Kay, G.G., 92 Kelly, M., 42 Knowledge discovery in Databases (KDD), see Data mining Koslow, S., 22, 83, 87 L Language assessment techniques for, 60 neuroinformatic applications for, 58–61, 73 neuropsychological models of, 58–59

122 Language (cont.) signs and gestures as, 60–61 Latent Semanitc Analysis (LSA), 74 Lezak, M., 33 Lichtheim, L., 58 Lotus Domino server, 68n Luria-Nebraska Neuropsychological Battery, 27 M Macaque monkeys brain cortices of visuospatial function in, 44–45 Collations of Connectivity Data on the Macaque Brain (CoCoMac), 12–13 Mapping, brain cortical areas of visuospatial processing, 45 in DIVA model, 63–64 of gyral surface contours, 9 of neural structure and connectivity patterns, 12–13 Marshall, J.C., 47 Memory Search software package, 35 Microsoft Access queries, 37–38, 49, 54 Microsoft Excel, 68 Microsoft SQL, 42 Military, U.S., Walter Reed Performance Assessment Battery (WRPAB), 35 Mishkin, M., 44 ModelDB software package, 15 Montreal Neurological Institute (MNI), 11 MRI Study of Normal Brain Development database, 11 N National Academy of Sciences Institute of Medicine (IOM), 22 National Center for Biotechnology Information (NCBI), 90 National Center for Sign Language and Gesture Resources (NCSLGR), 71, 72 National fMRI Data Center, 86 National Institutes of Health (NIH) MNI collaboration with, 11 MRI Study of Normal Brain Development database of, 11–12 National Center for Biotechnology Information (NCBI) of, 90 Neuroimaging Informatics Technology Initiative (NIfTI) of, 10 role in HBP, 22 National Science Foundation (NSF), 21–22 Neidle, C., 70–73

Index Neural connectivity patterns, 12–13 Neural modeling algorithm types used in, 17 defined, 14 tools for, 15 Neural networks, structure and functions of, 16–17 Neuroimaging Informatics Technology Initiative (NIfTI), 10 Neuroinformatics (NI) applications/models for, 7–24, 46 for assessment batteries, 5–6, 35 attitudes, reorganization, and training in, 91–93 data sharing issues in, 86–88 defined, 2 digitization of data in, 3 evolution of, 83, 85 frameworks for neuropsychology, 4 infrastructure management of, 20–24 neuropsychology-specific systems Aphasia Database, The (AD), 30t, 67–69 applications/models for, 29–84 Automated Neuropsychological Assessment Metrics (ANAM), 30t, 35–41 Computational Visual Field for Neglect (CVFN), 30t, 47–53 Computer-Assisted Cancellation Test System (CACTS), 30t, 53–57 Consortium for Neuropsychiatric Phenomics, The, 31t, 78–83 Directions Into Velocities of Articulators (DIVA), 30t, 61–67 Face Value Database Project, The, 31t Internet-Enabled Neuropsychological Assessment of Army Aviators (IENAAA), 30t, 41–46 scope and function of, 28–29 R software package, 31t, SignStream 70–73 tabular summary of, 30t, 31t See also as main entries privacy issues in, 86–88 Society for Neuroinformatics in Neuropsychology (SCNN), 95–97 storage/warehousing of, 20 throughput concept in, 88–89 Neuroinformatics Database System (NIDS), 19–20 Neuroinformatics Portal Pilot, The, 24 Neuroinformatics Site, The, 24

Index Neuroinformatics: The Integration of Shared Databases and Tools Towards Integrative Neuroscience (Amari), 23 Neuroinformatics Working Group (NWG), 23 Neuron software package, 13–14 Neurons 3D images of ultrastructure, 13 clustered into nodes, 16 simulation methods for, 14–18 structure and connectivity patterns of, 12–13 Neuropsychological Assessment (Lezak, 2004), 33 Neuropsychology changing landscape in, 100, 103–104 neuroinformatic frameworks for, 4 NeuroScholar project, 19 Nodes, neuronal, 16 Notes/Domino database, 68n O Obama, B., 103 Online information in BrainMaps.org, 13 for CNP tools, 81 on DIVA, 61 HBP website, 21 language-related, 73 for neuron simulation, 15 for NI sharing, 23–24 on OECD neuroinformatics working group, 23 R software package, 70 SignStream for Society for Neuroinformatics in Neuropsychology (SCNN), 95–97 in Synapse Web, 13 vast influence of, 100–101 See also specific databases/software Ontologies cognitive, 83 functions of, 18–19 need for, 89 O’Reilly, Tim, 101n Organization for Economic Cooperation and Development-Neuroinformatics Working Group (OECD-NIWG), 23–24 Outcome analysis module (CACTS), 54–57 P Parallel distributed processing, 15–16 Parameter modification in testing, 36 Patient profile database (CVFN), 49, 51 f

123 Personal digital assistants (PDAs), 40 Phenotypes/phenomics, neuropsychiatric defined, 6 description and background of, 75 neuropsychology role in, 75 research in, 76–83 Privacy issues, 86–88 Programme in International Neuroinformatics (PIN), 23 Proteonomics, 2 PubMed/MEDLINE database, 81 Q Query functions in AADB, 38, 39 f Query interfacing, 11 R Reconstruct software package, 13 Reeves, D., 36 Reitan, R., 91 Report on Neuroinformatics (2002), 23 Research, academic and clinical databases and knowledge discovery systems for, 18–20 Resolution of images in BrainMaps.org, 13 Rey Complex Figure Test, 27, 44, 48 Rheinisch-Westphalian Technical University (RWTH), 67, 73 S Sabatti, C., 75, 78 Schatz, P., 26–27 Schizophrenia, 77, 81, 82 f Schlegel, R.E., 93 Schultheis, M.T., 27 Semantic categories, 19 SenseLab software package, 15 Sharing platforms, 90 Sign language research, 61, 70–73 R software package SignStream database of, 71–72 description and background of, 71–72 structure and functions of, 71–73 tabular summary of, 31t Software packages and applications BrainCheckers, 40 CellBuilder, 15 c , 38 ClinicView CogScreen, 42 Dataman, 40 3D Slicer, 9–10 FreeSurfer, 9–10 Memory Search, 35

124 Software packages and applications (cont.) ModelDB, 15 Neuron, 13–14 Phenowiki, 81 PubBrain, 81 PubGraph, 81, 82 f Reconstruct, 13 SenseLab, 15 R SignStream , 70–73 SynWin, 42 Tower Puzzle, 35 Webfit, 74 Spatial neglect, 30t, 47–53 Speech motor skills, neuroinformatic applications for, 6 Speech production and acquisition (SPA), 61–62, 66 Speech/language neuropsychological models of, 58–61 NI role in, 58–61 production and acquisition (SPA) of, 61–62, 66 Standardization of data, 88–89 State maps (DIVA), 65 Sternberg, S., 35 Stimulus displacement/manipulation in CACTS tool, 57 in CVFN tool, 53–57 Summary tables (ANAM), 37 Synapse Web, 13 SynWin software package, 42 T Tabular types (ANAM), 38 Talairach space/system, 8 Temporal lobe epilepsy (TLE), 20 Testing, parameter modification in, 36 Three dimentional (3D) imaging generation of, 8

Index of neuronal ultrastructure, 13 Tower Puzzle software, 35 Tracer database, 12 Traumatic brain injury (TBI), 26, 42 U Ungerleider, L.G., 44 US Army Aeromedical Cognitive Assessment Tool (USA-ACAT), 42 V Van Essen, D.C., 45 Van Horn, J.D., 87 Virtual reality technology, 27 Visual search/attention patterns CACTS tool for, 53–57 neuroinformatic applications for, 7 Visual/gestural communication, neuroinformatic applications for, 6, 70–73 Visuospatial functioning defined, 45 NI assessment tools for, 47–48 tests for, 43–46 W Walter Reed Performance Assessment Battery (WRPAB), 35 Web 2.0, 101 Wernicke, C., 58 Wernicke’s aphasia, 68–69 Western Aphasia Battery Tap, 60 Whole Brain Atlas, 10–11 Wiki platforms, 101 R , 27 Wisconsin Card Sorting Test Wong, S.T.C., 18–19, 83 WordNet database, 74 X XANAT (a graphical anatomical database), 13