EXPLORING GEOVISUALIZATION
The maps on the cover are spatializations that use the landscape metaphor to represent the information in this book. The symbols represent the chapters, the relationships between them and the themes that they address as described in the Preface. Maps by Sara Fabrikant, Jo Wood and Jason Dykes.
EXPLORING GEOVISUALIZATION
Edited by
j. Dykes Department of Information Science City University, London, UK
A.M. MacEachren Departmentof Geography, Penn State University University Park, PA, USA
M.-J. Kraak Department of Geolnformation Processing International Institute of Geolnformation Science and Earth Observation (ITC), Enschede, The Netherlands
2005 tea
A C t
ELSEVIER
Published on behalf of the International Cartographic Association by Elsevier Amsterdam Paris - San
- Boston Diego
- Heidelberg
- San Francisco
- London
- New York-
- Singapore
- Sydney
Oxford - Tokyo
ELSEVIER B.V. Radarweg 29 P.O. Box 211, 1000 AE Amsterdam The Netherlands
ELSEVIER Inc. 525 B Street, Suite 1900 San Diego, CA 92101-4495 USA
ELSEVIER Ltd The Boulevard, Langford Lane Kidlington, Oxford OX5 1GB UK
ELSEVIER Ltd 84 Theobalds Road London WC 1X 8RR UK
9 2005 Elsevier Ltd. All rights reserved. This work is protected under copyright by Elsevier Ltd, and the following terms and conditions apply to its use: Photocopying Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish to make photocopies for non-profit educational classroom use. Permissions may be sought directly from Elsevier's Rights Department in Oxford, UK: phone (+44) 1865 843830, fax (+44) 1865 853333, e-mail:
[email protected]. Requests may also be completed on-line via the Elsevier homepage (http://www.elsevier.com/locate/permissions). In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; phone: (+1) (978) 7508400, fax: (+1) (978) 7504744, and in the UK through the Copyright Licensing Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London WIP 0LP, UK; phone: (+44) 20 7631 5555, fax: (+44) 20 7631 5500. Other countries may have a local reprographic rights agency for payments. Derivative Works Tables of contents may be reproduced for internal circulation, but permission of the Publisher is required for external resale or distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and translations. Electronic Storage or Usage Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or part of a chapter. Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher. Address permissions requests to: Elsevier's Rights Department, at the fax and e-mail addresses noted above. Notice No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made. First edition 2005 Reprinted 2007 Library of Congress Cataloging in Publication Data A catalog record is available from the Library of Congress. British Library Cataloguing in Publication Data A catalogue record is available from the British Library. ISBN-13:978-0-08-044531-1 ISBN-10:0-08-044531-4 (Hardbound) ISBN-10:0-08-044533-0 (CD-rom) O The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper). Printed in China.
Working together to grow libraries in developing countries www.elsevier.com I www.bookaid.org ] www.sabre.org
Contents Preface ................................................................................................................
ix
Digital Appendices ...........................................................................................
xiii
Acknowledgments .............................................................................................
xv
List of Contributors .........................................................................................
xvii
Introduction: Exploring Geovisualization 1. Exploring Geovisualization Jason Dykes, Alan M. MacEachren & Menno-Jan Kraak ....................... 3
Section A Geovisualization in Context: Perspectives from Related Disciplines 2. Information Visualization: Scope, Techniques and Opportunities for Geovisualization Daniel A. Keim, Christian Panse & Mike Sips ...................................... 23 3. Information Visualization and the Challenge of Universal Usability Catherine Plaisant .................................................................................... 53 4. Beyond Tools: Visual Support for the Entire Process of GIScience Mark Gahegan .........................................................................................
83
Section B Creating Instruments for Ideation: Software Approaches to Geovisualization Perspectives 5. Creating Instruments for Ideation: Software Approaches to Geovisualization Gennady Andrienko, Natalia Andrienko, Jason Dykes, David Mountain, Penny Noy, Mark Gahegan, Jonathan C. Roberts, Peter Rodgers & Martin Theus ............................................................. 103
vi
Exploring Geovisualization
6. Statistical Data Exploration and Geographical Information Visualization Martin Theus .......................................................................................... 127 7. Graph Drawing Techniques for Geographic Visualization Peter Rodgers ......................................................................................... 143 8. Exploratory Visualization with Multiple Linked Views Jonathan C. Roberts ............................................................................... 159 9. Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour David Mountain ..................................................................................... 181 10. Impact of Data and Task Characteristics on Design of Spatio-Temporal Data Visualization Tools Natalia Andrienko, Gennady Andrienko & Peter Gatalsky ................. 201 11. Using Multi-agent Systems for GKD Process Tracking and Steering: The Land Use Change Explorer Monica Wachowicz, Xu Ying & Arend Ligtenberg ............................ 223 12. Signature Exploration, a Means to Improve Comprehension and Choice within Complex Visualization Processes: Issues and Opportunities Penny Noy ............................................................................................. 243 13. Facilitating Interaction for Geovisualization Jason Dykes ........................................................................................... 265
Section C
Using 3D in Visualization
14. Using 3D in Visualization Jo Wood, Sabine Kirschenbauer, Jtirgen D611ner, Adriano Lopes & Lars Bodum .............................................................. 295 15. Multim im parvo Many Things in a Small Place Jo Wood ................................................................................................. 313 -
16. Geovisualization and Real-Time 3D Computer Graphics Jtirgen D611ner ....................................................................................... 325
Table of Contents
vii
17. Interactive Approaches to Contouring and Isosurfacing for Geovisualization Adriano Lopes & Ken Brodlie .............................................................. 345 18. Applying "True 3D" Techniques to Geovisualization: An Empirical Study Sabine Kirschenbauer ............................................................................ 363 19. Modelling Virtual Environments for Geovisualization: A Focus on Representation Lars Bodum ........................................................................................... 389 20. Web-based Dissemination and Visualization of Operational 3D Mesoscale Weather Models Lloyd A. Treinish .................................................................................. 403
Section D Connecting People, Data and Resources: Distributed Geovisualization 21. Connecting People, Data and Resources - Distributed Geovisualization Ken Brodlie, David Fairbairn, Zarine Kemp & Michael Schroeder ................................................................................. 425 22. Moving Geovisualization toward Support for Group Work Alan M. MacEachren ............................................................................ 445 23. Models of Collaborative Visualization Ken Brodlie ............................................................................................ 463 24. Intelligent Information Integration: From Infrastructure through Consistency Management to Information Visualization Michael Schroeder ................................................................................. 477 25. A Knowledge-based Collaborative Environment for Geovisualization: Ontologies for Multiple Perspectives on Distributed Data Resources Zarine Kemp .......................................................................................... 495 26. Geovisualization Issues in Public Transport Applications David Fairbairn ...................................................................................... 513 27. Presenting Route Instructions on Mobile Devices: From Textual Directions to 3D Visualization Volker Coors, Christian Elting, Christian Kray & Katri Laakso ......... 529
viii
Exploring Geovisualization
Section E Making Useful and Useable Geovisualization: Design and Evaluation Issues 28. Making Useful and Useable Geovisualization: Design and Evaluation Issues Sven Fuhrmann, Paula Ahonen-Rainio, Robert M. Edsall, Sara I. Fabrikant, Etien L. Koua, Carolina Tob6n, Colin Ware & Stephanie Wilson ........................................................... 553 29. 3D Geovisualization and the Structure of Visual Space Colin Ware & Matthew Plumlee ........................................................... 567 30. Applications of a Cognitively Informed Framework for the Design of Interactive Spatio-temporal Representations Robert M. Edsall & Laura R. Sidney .................................................... 577 31. User-centered Design of Collaborative Geovisualization Tools Sven Fuhrmann & William Pike ........................................................... 591 32. Towards Multi-variate Visualization of Metadata Describing Geographic Information Paula Ahonen-Rainio & Menno-Jan Kraak .......................................... 611 33. Evaluating Self-organizing Maps for Geovisualization Etien L. Koua & Menno-Jan Kraak ...................................................... 627 34. Evaluating Geographic Visualization Tools and Methods: An Approach and Experiment Based upon User Tasks Carolina Tob6n ...................................................................................... 645 35. Cognitively Plausible Information Visualization Sara Irina Fabrikant & Andr6 Skupin ................................................... 667
Conclusion" Advancing Geovisualization 36. Advancing Geovisualization Jason Dykes, Alan M. MacEachren & Menno-Jan Kraak ................... 693 Index ................................................................................................................ 705
Preface you judge a book by its cover? "Exploring Geovisualization" draws upon perspectives from disciplines including information science, computer science and cartography to discuss and advance the emerging field of geovisualization and maps the contributions from these fields on the cover of the book. One of the areas in which expertise from these domains has been usefully combined to develop techniques and augment knowledge is in the generation and mapping of information spaces or spatializations. These are maps that show the relationships between a series of documents in a collection according to the information contained in each. The cover of this book contains two such maps, on each of which the chapters within are displayed as a landmark. These are represented on the maps with a point symbol and a label identifying the first author of the chapter. The landmarks are arranged using a technique that places documents with more similar contents closer together on the page. The maps themselves fill a continuous information space in which a variety of topics are organized according to their semantic relationships. Where chapters are more closely related, landmarks are clustered in the information space and we can consider the document collection (the book in this case) to have a particular focus on the themes that the chapters address. It seems plausible to draw upon the metaphor of the landscape to map the presence or absence of information relating to themes within the information space. This can be achieved by representing the various thematic foci of the book as an undulating semantic surface with continually varying magnitudes. Where a number of chapters are relatively closely related the information landscape metaphorically piles up into mountains of information about a particular theme. The valleys between information peaks occur in areas of the information landscape associated with topics about which the book focuses less explicitly. The topographic shading scheme used in the map on the left draws further upon the metaphor to represent the "thematic density" of the book across our information landscape. Note however that the landscape and the distribution of the documents represent contributions to this book, and the documents themselves are only discrete samples of possible information sources within the information space - there are likely to be alternative information sources beyond the scope of this volume that can fill ix
Exploring Geovisualization
the information valleys. Indeed it could be argued that whilst the chapters that are clustered in our information space represent current research foci (as reported here), those in information valleys and isolated locations may represent topics that require additional research efforts and are the most 'cutting edge'. The map on the right splits the landscape up into discrete units and shades the information space according to the section of the book in which the chapter represented by the closest landmark occurs. This allows us to see how the sections of the book map into our information space and the themes that are represented by areas within it. The map also allows us to consider the ways in which the chapters within the sections relate to each other according to our spatialization. We hope that these graphics and the metaphor are interesting and that they will prompt some thought about both the book and the nature of maps and information spaces. Some of the decisions taken in developing these spatializations are subjective (though each was thoroughly discussed and we have been through a number of redesigns!) and any number of graphical realizations of the contents of the book might be developed. There are clear parallels here with conventional cartography that depicts the world around us as all map-makers draw upon the three major tenets of map design: theme, purpose and audience, to develop their products. The main difference with spatialization is that it is perhaps more difficult to compare maps of information to an objective truth than is the case with traditional cartography. Despite the complex transformations and abstractions that occur and are imbued with the influence of personal and socio-cultural preference and bias, most cartographers would accept the existence of a 'reality' that they are mapping, that shapes their work, and upon which to develop their design and assess error. When generating spatializations we do not have the notion of a physical standard upon which to base and evaluate our maps. In addition to their role in inspiring metaphors and stimulating thought and discussion we also hope that the maps on the cover provide both a pertinent starting point for the book and an overview of some aspects of its contents. Perhaps they offer a relatively novel opportunity to view and assess the scope of a book, or at least the relationships between the chapters within it, from its cover.
Details on the Cover Map The maps that we have used on the cover of the book utilize a topical or thematic density surface. The surface was produced by Sara Fabrikant and developed through discussions with the editors and a number of other colleagues. Jason Dykes and Jo Wood then used the LandSerf software to generate the maps, in discussion with Sara and others, by applying symbolism, shading and some cartographic exaggeration. The process consisted of a number of stages. Initially Latent Semantic Indexing (Deerwester et al., 1990) was used to determine the similarities between all chapters by comparing the chapter titles and keywords submitted by the authors.
Preface
xi
This seemed to produce more meaningful results than a comparison of the full-text of the abstracts. Principal Coordinate Ordination was then used to collapse the document similarity matrix into two-dimensional spatial coordinates. To generate a topical density surface that reflects the discrete nature of the input data (the book chapters in this case) a pycnophylactic surface was interpolated from these point locations (Tobler, 1979). The pycnophylactic reallocation approach is an areal interpolation technique that permits the construction of smooth density surfaces from more abrupt continuous data, such as those recorded in area-based enumeration units. The chapters represent such discrete boundaries within the information landscape of "Exploring Geovisualization". A voronoi tessellation was derived from the point locations to represent the maximal zone of influence for each of the 36 chapters. The voronoi boundaries were then used as break lines in the process of pycnophylactic interpolation to generate an estimation of the information landscape of the book. Full details on using the method to generate cognitively plausible information spaces are provided in the literature (see Fabrikant, 2001) and the development and application of a framework for doing so is presented later in Chapter 35 (Fabrikant and Skupin, this volume). The resultant density surface was further manipulated in LandSerf (Wood, 2004) to add structured noise and to apply shading and some cartographic enhancements. The noise was designed to add visual interest, to draw attention to the uncertainty inherent in the surface and to reinforce the landscape metaphor of information 'hills' and 'valleys'. This was achieved by combining the density surface with a random fractal surface of fractal dimension 2.1. The original surface was smoothed and rescaled so that the total fractal noise was approximately 4% of the 'true' variation. LandSerf was then used to generate vector contour lines at a vertical interval of approximately 10% of the total variation in relief. The colour scheme applied to the surface is based upon the work of the Swiss cartographer Eduard Imhof for the representation of relief (Imhof, 1965). Relief shading was also computed in LandSerf and the combination of the original surface with the noise, contours, topographic colour scheme and hill shading emphasizes the use of the landscape metaphor to represent density of information about the themes in our landscape. The voronoi polygons are shaded using a colour scheme derived from the examples, guidelines, and considerations suggested by Brewer (1994) and implemented in ColorBrewer (Harrower and Brewer, 2003). The colour scheme is also employed to differentiate between sections of the book in the digital appendices which provide a number of spatializations through which "Exploring Geovisualization" may be explored. The labels were added using the comprehensive text elements in SVG. LandSerf will export raster surfaces to PNG, JPEG and other graphics formats and vectors directly to SVG. Jason Dykes Sara Fabrikant Jo Wood
xii
Exploring Geovisualization
References Brewer, C. A., (1994) "Color use guidelines for mapping and visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography, Vol. 2. Oxford: Elsevier Science Ltd., pp. 123-148. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harschman, R., (1990) "Indexing by latent semantic analysis", Journal of the American Society of Information Science, 41, 391-407. Fabrikant, S. I., (2001) "Visualizing region and scale in semantic spaces", Proceedings, The 20th International Cartographic Conference, ICC 2001, Beiing, China, pp. 2522-2529. Harrower, M., and Brewer, C. A., (2003) "ColorBrewer.org: an online tool for selecting colour schemes for maps", The Cartographic Journal, 40(1), 27-37, online: http://www.colorbrewer.org Imhof, E., (1965) Kartographische Geliindedarstellung. Berlin: De Gruyter. Wood, J. (2004) LandSerf. online: http://www.landserf.org/(10/10/04) Tobler, W. R., (1979) "Smooth pycnophylactic interpolation for geographical regions", Journal of the American Statistical Association, 74(367), 519-530.
Digital Appendices It is quite a challenge to produce a book on geovisualization as colour, animation and dynamism are so important in the field and yet we are severely limited in terms of the extent to which we can draw upon these essential features of digital cartography when publishing on paper. We have therefore produced a series of digital appendices, included on the CD that accompanies this book. These contain the colour imagery submitted by each of the authors and should be used in conjunction with your reading of the book and consideration of the figures (particularly those that rely upon colour). The digital appendices are accessed through an interactive interface that draws upon a number of alternative spatializations of the contents of the book. These include those shown on the cover and discussed in the Preface. We also draw upon a network representation, rather than one that uses a continuous space to represent relationships between the chapters. The networks and their derivation are described in the introduction to the book (Dykes et al., this volume (Chapter 1)).
xiii
This Page Intentionally Left Blank
Acknowledgments Thanks are due to a number of individuals and organisations without whom this book would not have been possible. These include: the International Cartographic Association; the ICA Commission on Visualization and Virtual Environments; City University, London, and in particular The School of Informatics, Walter Sickert Hall and the Open Learning Centre; the participants at the London 'Exploring Geovisualization' workshop for contributing to a stimulating meeting - including Bob Spence, Ebad Banissi, Heiko Bleschschmeid and Steph Wilson; the various contributors to this volume whose efforts and patience are hugely appreciated; the reviewers whose comments have helped shape this book; series editor Bob McMaster; Elsevier Ltd.; Alden PrePress Services; TheresaMarie Rhyne and the ACM SIGGRAPH Carto Project; Werner Kuhn; the European Science Foundation (EURESCO Conferences); Stephanie Marsh; Jo Wood; Emma Dykes; Sara Fabrikant and Andr6 Skupin; MGI students at City University for comments on cover proposals; ITC, International Institute of Geoinformation Science and Earth Observation and the School of Informatics, City University, London for material support. For his role in developing these perspectives on geovisualization, MacEachren is pleased to acknowledge support over the past several years from the U.S. National Science Foundation (9983451, 9978052, 0113030, 0306845), from the Advanced Research and Development Activity (ARDA), and from the U.S. National Cancer Institute (NCI-CA95949).
XV
This Page Intentionally Left Blank
List of Contributors Paula Ahonen-Rainio Department of Surveying, Institute of Cartography and Geoinformatics, Helsinki University of Technology, PO Box 1200, Espoo, FIN-02015 HUT, Finland Gennady Andrienko Fraunhofer AiS.SPADE - Institute for Autonomous Intelligent Systems, Spatial Decision Support Team, Schloss Birlinghoven, Sankt-Augustin, D-53754, Germany Natalia Andrienko Fraunhofer AiS.SPADE - Institute for Autonomous Intelligent Systems, Spatial Decision Support Team, Schloss Birlinghoven, Sankt-Augustin, D-53754, Germany Lars Bodum Centre for 3D Geolnformation, Aalborg University, Niels Jernes Vej 14, DK-9220 Aalborg 0, Denmark Ken Brodlie School of Computing, University of Leeds, Leeds LS2 9JT, UK Volker Coors Stuttgart University of Applied Sciences, Schellingstr. 24, 70174 Stuttgart, Germany
Jtirgen D611ner Hasso Plattner Institute at the University of Potsdam, Helmert-Str. 2-3, 14482 Potsdam, Germany Jason Dykes Department of Information Science, City University, London EC 1V 0HB, UK Robert M. Edsall Department of Geography, Arizona State University, PO Box 870104, Tempe, AZ 85287, USA Christian Elting European Media Lab (EML), Schloss-Wolfsbrunnenweg 33, 69118 Heidelberg, Germany Sara Irina Fabrikant Department of Geography, University of California Santa Barbara, 3611 Ellison Santa Barbara, CA 93106, USA David Fairbairn School of Civil Engineering and GeoSciences, University of Newcastle Upon Tyne, Newcastle Upon Tyne NE1 7RU, UK Sven Fuhrmann GeoVISTA Center, Department of Geography, The Pennsylvania State University, 302 Walker Building, University Park, PA 16802, USA xvii
xviii
Exploring Geovisualization
Mark Gahegan GeoVISTA Center, Department of Geography, The Pennsylvania State University, University Park, PA 16802, USA Peter Gatalsky Fraunhofer AiS.SPADE - Institute for Autonomous Intelligent Systems, Spatial Decision Support Team, Schloss Birlinghoven, Sankt-Augustin, D-53754, Germany Daniel A. Keim Computer Science Institute University of Konstanz Box D78, Universit/itsstral3e 10 D-78457, Konstanz Germany Zarine Kemp Computing Laboratory, University of Kent, Canterbury, Kent CT2 7NF, UK Sabine Kirschenbauer Institute for Cartography, Dresden University of Technology, Dresden, Germany Etien L. Koua Department of Geo-Information Processing, International Institute for Geo-Information Science and Earth Observation (ITC), P.O. Box 6, 7500 AA Enschede, The Netherlands Menno-Jan Kraak Department of Geo-Information Processing, ITC, International Institute of Geoinformation Science and Earth Observation, P.O. Box 6, NL-7500 AA Enschede, The Netherlands Christian Kray German Research Center for AI (DFKI), Stuhlsatzenhausweg 3, 66123 Saarbrticken, Germany
Katri Laakso Nokia Research Center (NRC), It~imerenkatu 11-13, 00180 Helsinki, Finland Arend Ligtenberg Wageningen UR, Centre for Geo-Information, Droevendaalsesteeg 3, PO BOX 47, 6700 AA Wageningen, The Netherlands Adriano Lopes Department of Informatics of the Faculty of Science and Technology/CITI, New University of Lisbon, Lisbon, Portugal Alan M. MacEachren GeoVISTA Center, Department of Geography, Penn State University, 303 Walker, University Park, PA 16802, USA David Mountain Department of Information Science, City University, London EC 1V 0HB, UK Penny Noy School of Informatics, City University, London EC 1V 0HB, UK Christian Panse Computer Science Institute University of Konstanz Box D78, Universit~itsstrage 10 D-78457, Konstanz Germany William Pike GeoVISTA Center, Department of Geography, The Pennsylvania State University, 302 Walker Building, University Park, PA 16802, USA Catherine Plaisant Human-Computer Interaction Laboratory, University of Maryland, HCIL/UMIACS A.V. Williams Building, University of Maryland, College Park MD 20782, USA
List of Contributors Matthew Plumlee Data Visualization Research Lab, Center for Coastal and Ocean Mapping, University of New Hampshire, Durham, New Hampshire, USA Jonathan C. Roberts Computing Laboratory, University of Kent, Canterbury, Kent CT2 7NF, UK Peter Rodgers Computing Laboratory, University of Kent, Canterbury, Kent CT2 7NF, UK Michael Schroeder Department of Computing, City University, London, UK Laura R. Sidney Department of Geography, Arizona State University, Arizona, USA Mike Sips Computer Science Institute University of Konstanz Box D78, Universit~itsstral3e 10 D-78457, Konstanz Germany Andr6 Skupin Department of Geography, University of New Orleans, New Orleans, LA 70148, USA Martin Theus Department of Computer-Oriented Statistics and Data Analysis, University of Augsburg, Universit~itsstr. 14, 86135 Augsburg, Germany Carolina Tob6n Department of Geography and Centre for Advanced Spatial Analysis
xix
(CASA), University College London (UCL), 1-19 Torrington Place, Gower Street, London WC1E 6BT, UK Lloyd A. Treinish Mathematical Sciences, IBM Thomas J. Watson Research Center, 1101 Kitchawan Road, Yorktown Heights, NY 10598, USA Monica Wachowicz Wageningen UR, Centre for Geo-Information, Droevendaalsesteeg 3, PO BOX 47, 6700 AA Wageningen, The Netherlands Colin Ware Data Visualization Research Lab, Center for Coastal and Ocean Mapping (and Computer Science Department), University of New Hampshire, 24 Colovos Road, Durham, NH 03824, USA Stephanie Wilson Centre for HCI Design, City University, Northampton Square, London EC 1V 0HB, UK Jo Wood Department of Information Science, City University, London EC 1V 0HB, UK Xu Ying Wageningen UR, Centre for Geo-Information, Droevendaalsesteeg 3, PO BOX 47, 6700 AA Wageningen, The Netherlands
This Page Intentionally Left Blank
Introduction Exploring Geovisualization
1. Exploring Geovisualization Jason Dykes, Alan M. MacEachren & Menno-Jan Kraak ..............................................................................
3
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes,A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 1
Exploring Geovisualization Jason Dykes, Department of Information Science, City University, London EC 1V 0HB, UK Alan M. MacEachren, GeoVISTA Center, Department of Geography, Penn State University, 303 Walker, University Park, PA 16802, USA Menno-Jan Kraak, Department of GeoInformation Processing, ITC, International Institute of Geoinformation Science and Earth Observation, PO Box 6, NL-7500 AA, Enschede, The Netherlands
Keywords: geovisualization, GIScience, Cartography, EDA, Information Visualization, maps, multi-disciplinary, research agenda, challenges, workshop, collaboration
Abstract This introductory chapter makes the case for exploring geovisualization from multiple, inter-disciplinary perspectives and presents the structure of the book in which this is achieved. It does so by introducing geovisualization and drawing upon the work of the International Cartographic Association Commission on Visualization and Virtual Environments in reporting research foci and challenges in geovisualization and documenting recommendations for action through which these can be addressed. An examination of the nature of geovisualization and its various interfaces with cognate fields of academic study is identified as a key requirement. The objective is to foster communication, encourage collaboration and augment existing knowledge with that from relevant disciplines. The organization and structure of a geovisualization workshop in which these requirements were addressed is outlined. The outcomes of the workshop form the basis of this book. The book, in turn, supports and broadens the process of collaboration through reports on current research efforts and cross-disciplinary sharing of knowledge about our related fields of expertise. It contains a series of introductory contributions followed by sections entitled "creating instruments for ideation", "using 3D in visualization", "connecting people data and resources" and "making useful and usable geovisualization". Each of these sections is preceded by a collaboratively produced co-authored introduction to a particular focus for geovisualization. Crossreferences between these chapters and sections are common and many are explicitly identified in the text and supported by the digital appendices. Readers are encouraged to further relate concepts, issues and themes that are apparent between chapters in the book as they explore geovisualization and we collectively advance this evolving field. 3
J. Dykes, A.M. MacEachren & M.-J. Kraak
1.1
Geovisualization in Context
Geovisualization is an emerging field. It draws upon approaches from many disciplines, including Cartography, Scientific Visualization, Image Analysis, Information Visualization, Exploratory Data Analysis (EDA) and GIScience to provide theory, methods and tools for the visual exploration, analysis, synthesis and presentation of data that contains geographic information (MacEachren and Kraak, 2001a). The interactions across these disciplines are fluid, as are the boundaries delimiting the disciplines themselves. The art and science of Cartography has developed to embrace and support visualization. This has occurred informally through the efforts of cartographers to support map use and map users by employing the tools afforded by technical advances effectively and more formally through the Commissions of the International Cartographic Association (ICA) on map use, visualization and most recently visualization and virtual environments. Over the past decade, the ICA Commissions have focused on the use of highly interactive maps by individual experts to support thought processes that are motivated towards the discovery of unknowns in complex spatial data sets. Over time, the technologies and techniques that have supported these processes have become more readily available, information has become more accessible and a broadening range of types of high-quality spatial data have been recorded. The result is that greater numbers of geographic information users are now employing highly interactive techniques to achieve insight from a variety of spatial data sets. These users are not the traditional consumers of conventional Cartography and the uses to which they put their maps are evolving. Yet, whomever it involves and however it is used, geovisualization is about people, maps, process, and the acquisition of information and knowledge. It can lead to enlightenment, thought, decision making and information satisfaction, but can also result in frustration! When employed effectively, geovisualization offers the possibility for engaging, personal and specific interfaces to geographic information through which a range of users may be able to participate in a variety of activities that require and rely upon the geographic component of the information in hand. The increasing importance and use of spatial information and the map metaphor establishes geovisualization an essential element of 21st century information use, a genuine opportunity for 21st century Cartography and a requirement for modern map users.
1.2
Geovisualization Research
Considerable research activity in geovisualization is drawing upon the expertise available in a number of cognate disciplines to support the efforts of this variety of users of geographic information. This is the case whether the process of discovery is private or collective and whether it is related to the acquisition of established knowns or the search for insight to identify, explain and understand particular unknowns. The nature of geovisualization research is thus multi-faceted and associated research efforts are wide ranging. However, a number of broad themes can be identified
Exploring Geovisualization
5
that pervade recent activities and define an agenda for this evolving field. "Research Challenges in Geovisualization" (MacEachren and Kraak, 2001b) is a multi-authored effort coordinated by the ICA Commission on Visualization and Virtual Environments that delineates an international research agenda by identifying particular areas in which specific activity is required and has the potential for impacts well beyond Cartography. These focus areas are closely inter-related and should certainly not be regarded as discrete. They include research with an emphasis on representation, visualization-computation integration, interfaces and cognitive/usability issues and are summarized below to describe the nature and scope of current geovisualization research priorities.
1.2.1
Focus on representation
Fundamental questions about the representation of geographic phenomena arise as technological possibilities develop and the data available for depiction through advanced and interactive graphical realizations changes. Challenges include determining the limits and advantageous uses of both traditional and novel representation methods, creating meaningful graphics to represent very large, multi-variate spatio-temporal data sets (that may include both three spatial dimensions and time) and the development, use and continual evaluation of innovative tools that take advantage of interactivity, animation, hyper-linking, immersive environments, agents, multi-modal interfaces and dynamic object behaviors. Augmented and mixed reality applications and multi-modal representations of data are examples of technologically driven possibilities for which traditional approaches to geographic representation are unlikely to be adequate.
1.2.2
Focus on visualization-computation integration
The way that we construct knowledge from geospatial data can draw upon interactive visual representations that use some of these novel techniques. However, we are likely to be most successful in analyzing large complex spatial data sets if geovisualization is able to draw heavily and directly upon advances in Computer Science and computer graphics. The design, development and testing of software and hardware solutions that support the kinds of graphical interactivity that are specifically required by geovisualization is a key requirement. One aspect of this challenge involves integrating and adapting advances in computer graphics and Information Visualization associated with visual datamining for application to geographic data analysis. Such integration of tools and techniques would make it practical for us to participate in visually enabled knowledge construction across the process of GIScience. This process can benefit from closely linked software tools focused on uncovering patterns in complex structured geospatial data, the explanation of these structures and relationships through the development and testing of theory and the ultimate communication of this knowledge.
1.2.3
Focus on interfaces
Advances in geovisualization interface design are essential for visually enabled knowledge construction to have the greatest impact. Additionally, techniques need to be
6
J. Dykes, A.M. MacEachren & M.-J. Kraak
improved and developed to make geovisualization available to a wide range of users with a variety of requirements. Facilitating progress in the real world use of geospatial information and technology may require the adaptation of existing interface design (for example, see w1.2.4) and the development of new paradigms to provide high levels of interaction with advanced forms of (possibly novel) representation (for example, those relying upon augmented and mixed reality). Further knowledge is required concerning the possibilities afforded by multi-modal methods of accessing and interacting with information, and in using these and other means to create forms of interaction that are appropriate to particular combinations of user, task and circumstance. The various requirements presented by mobile devices, the desire to support group work, and the need to accommodate and support different groups and individuals make this a particularly challenging area of research.
1.2.4
Focus on cognitive/usability issues
Whatever the application, it is essential that we develop knowledge of whether the geovisualization techniques, tools and solutions that are produced actually work and under what circumstances this is the case. We must also be able to explain and even predict such outcomes. We can begin to achieve these objectives by studying the ways in which different users react to a range of new and established geovisualization methods. By drawing upon knowledge of the perceptual and cognitive processes involved, we may be able to generate a body of knowledge and associated theory relating task, technique and user type that establishes best practice. A whole range of users with different ages, cultures, sexes, levels of experience and sensory abilities may participate in visualization as individuals or in collaboration with others. Methods drawn from human computer interaction (HCI), such as the concept of "usability" may enable us, with associated knowledge, to both develop solutions for particular types of user and task and to approach the objective of "universal usability" in geovisualization. Each of these four areas of research focus has clear parallels with more traditional cartographic research, and can draw upon the knowledge derived from that work. Each can also benefit from knowledge obtained in domains beyond Cartography and GIScience. The need to undertake effective research in these areas is becoming increasingly urgent as maps are used more frequently, for more tasks, by more users, to provide access to and insight from more data.
1.3
Geovisualization Challenges
Our progress in each of these areas of geovisualization is dependent upon various fundamental issues that cut across the themes. They are established as a series of "geovisualization research challenges" in the ICA research agenda (MacEachren and Kraak, 2001). Each challenge relates to a current concern towards which the research efforts of the geovisualization community should be usefully applied.
Exploring Geovisualization
1.3.1
7
Research challenge 1 - experimential and multi-modal "maps"
Many of the questions being asked by geovisualizers relate to the increasingly experiential and multi-sensory representation technologies that are available. There is a need to develop the technologies and understanding that will enable geovisualization to use such modes of information access and manipulation effectively. Generating maps that do so will involve improving our models of the world so that they occupy volume and are dynamic - a considerable challenge when much geospatial information draws upon the traditional map metaphor in which space is regarded as flat and static. Much geographic information science and technology also starts with this assumption. Whilst such technical challenges are addressed, conceptual tensions associated with the way that realism and abstraction are utilized must be resolved. In some circumstances and applications visual and virtual realism are regarded as an ideal (and the challenge of rendering virtual spaces in real time is an active research problem, particularly in computer graphics). Yet the existing geovisualization paradigm is grounded in an assumption that abstraction is essential for achieving insight. It is vital that we explore and resolve the tensions between these perspectives if we are to make scientific progress and take advantage of developing technologies that offer exciting possibilities for virtual realism, multi-sensory representation and sophisticated modes of interaction, rather than be taken advantage of by them.
1.3.2
Research challenge 2 - large data sets
Geospatial data sets of progressively larger size and increasingly complex structure offer a continuing challenge for geovisualization as we aim to develop appropriate techniques, tools and approaches. Whilst the initial promise of visualization was based upon leveraging the power of human vision to extract meaning from complex data sets, many existing techniques do not scale well to the massive datasets that are increasingly common. Meeting this challenge will require advances in the methods used and their integration with geocomputational techniques that must also be enhanced and developed. Ultimately human understanding can then be used to steer an investigative process that draws upon visual processing, domain knowledge and advanced computational techniques to uncover patterns and relationships and determine their meaning. A key issue here is that many existing methods and tools do not encode geographic knowledge and meaning in effective ways. Knowledge construction using such tools can neither build easily from existing knowledge as a result, nor can they capture knowledge as it is generated. This is a challenge that demands not just new methods and tools, but requires a fundamental effort in theory building directed to the representation and management of geographic knowledge.
1.3.3
Research challenge 3 - group work
Advances in telecommunications and display technologies are making multi-user systems (through which group work can be supported) more effective, sophisticated, accessible and common. Most work with geographic information requires coordinated effort by groups, and yet the tradition in geovisualization research (and GIScience research more
8
J. Dykes, A.M. MacEachren & M.-J. Kraak
broadly) focuses on the individual expert, operating in isolation in the private domain. As a result, methods and tools have tended to be designed for individual use. Research effort is required to develop a new generation of techniques and competence to support same and different-place collaborative geovisualization. Attention must also be focused on the cognitive, social, and usability issues of visual display mediated dialogue, both amongst human collaborators and between human and computer. 1.3.4
Research challenge 4 - human-centered approach
Technology is a key driver of geovisualization, yet we can learn a great deal from disciplines that have developed and continue to employ a human-centered approach to their science. An important goal involves the development of an approach to geovisualization research that integrates work on technological advances leading toward more powerful and usable tools with work on human spatial cognition and the potential of visual (and other concrete) representations to enable thinking, learning, problem solving, and decision making. Approaches that can be characterized as "build and they will come" and "one tool fits all" dominate contemporary geoinformation technology. Yet a compelling need exists to develop both the theory and practice to support universal access and usability for geospatial data. This will require new approaches and methods that support the personalization of geovisualization tools to assist particular users and user groups with particular geovisualization tasks.
1.4
Exploring Geovisualization - Rationale
The geovisualization challenges outlined here indicate that the boundaries of Cartography are becoming less clear-cut, the nature of map use and users is broadening and the range of influence and sources of stimuli shaping Cartography is widening. The set of geovisualization challenges identified is a demanding one, yet the importance of addressing them is evident. The need for a multi-disciplinary approach to address such multi-faceted issues successfully is equally apparent. A concerted effort by a broad and deep community of scientists offering multiple linked perspectives is required: "cartographers cannot address these challenges alone" (MacEachren and Kraak, 2001 a). However, MacEachren and Kraak (2001a) also note that whilst "most of the challenges [facing geovisualization] are multi-faceted ones, requiring multi-disciplinary perspectives ... Little exchange of ideas occurs across disciplines working on related problems, with multiple sets of largely separate literature having few or no cross citations". A number of recommendations for action are made in "Research Challenges in Geovisualization" (MacEachren and Kraak, 2001 a) with the aim of coordinating efforts more effectively. A key proposal is the promotion of interdisciplinary communication and cross-disciplinary work to encourage more scientists to commit to multi-disciplinary research through which the challenges can be met. The investigation of geovisualization practice and issues from a number of disciplinary perspectives and communication between experts and practitioners
Exploring Geovisualization
9
in cognate disciplines are thus key objectives for advancing geovisualization. The need for such activity to be both supported by researchers and research infrastructures and valued by systems of recognition is also noted. Events at which geovisualization can be discussed and explored from a variety of perspectives, augmented with knowledge from relevant disciplines, and reported to the community are an essential means of addressing the geovisualization challenges. Doing so enables us to investigate the nature of geovisualization and its various interfaces with cognate fields, advance our knowledge and further define our discipline and practice. These activities will assist us in establishing inter-disciplinary awareness and encourage the required collective approach in the short term. Efforts to establish shared knowledge and identify commonalities and differences in understanding, techniques and approach can be supported in this way. In the longer term, the promotion and maintenance of such collaboration will enable us to advance the various research agenda. Exploring Geovisualization and the meeting and communication that formed the basis for this book are intended as key steps in this process as researchers with a variety of interests and a range of backgrounds aim to establish and document current research efforts and share knowledge about our practice, requirements and related fields of expertise.
1.5
Exploring Geovisualization - Approach
The concept of "radical collocation" is presented by Brodlie, this volume (Chapter 23). The term is used to describe a situation where the likely gains in productivity resulting from the ease of communication, coordination and organisation associated with actually being located in the same place results in teams of subject experts being "brought together into war rooms for an intensive piece of work". This is precisely what was suggested by the ICA Commission on Visualization and Virtual Environments to advance the research agenda: "workshops should be supported to bring individuals with varied disciplinary perspectives together to discuss research questions and approaches to addressing them in detail." (MacEachren and Kraak, 2001a) To achieve this aim, a cross-disciplinary workshop was organized in September 2002 at City University, London. The School of Informatics at City offers expertise in a number of related fields of study including GIScience, information science, Computer Science and HCI design. An open invitation was made to researchers in geovisualization and cognate disciplines to participate in the presentation and discussion of existing work in their fields, current trends, likely future developments and possibilities for the sharing of knowledge, techniques, practice and approach. The planned outcomes of the meeting included position papers representing the contributions of individuals and groups, discussion between researchers with common foci from different disciplines and this edited volume containing considered individual contributions and co-authored multi-disciplinary
10
J. Dykes, A.M. MacEachren & M.-J. Kraak
statements through which geovisualization and associated research efforts could be explored and presented to a wide audience. The contributors to the workshop fall into one of two categories: members of the ICA Commission on Visualization and Virtual Environments, a group with primary concerns relating to geographic information and Cartography, and "external experts" from cognate disciplines (primarily Computer and Information science). The former were asked to document their research to a wider community by identifying generic research objectives, a summary of the knowledge gained and some additional research questions that require cross-disciplinary expertise. Specific examples of research were requested of the ICA contributors with which to illustrate key geovisualization issues and identify current knowledge and practice. Contributions included various accounts of specific geovisualization methods (and their application), system descriptions, tests of geovisualization implementations and documented opportunities for research and collaboration. The external experts were asked to produce overviews of current themes in their particular field of expertise, to identify general areas where collaboration with the geovisualization community may be beneficial and to describe ways in which their discipline might gain from and/or contribute to advances in geovisualization, particularly regarding the specific geovisualization research challenges. To define the scope of the workshop and enable the organizers to identify topics upon which discussion could focus, participants were asked to specify their primary and secondary areas of expertise from a list of alternatives including Information Visualization, cognitive sciences, Computer Science, virtual environments, information science, HCI, education/pedagogy and data exploration/datamining. Drafts of individual contributions were also circulated prior to the workshop so that the organizers and participants were introduced to the work of all other contributors in advance of the meeting. The workshop itself took place between Wednesday 1 l th and Saturday 14th September 2002. Participants were arranged into groups by the organizers in advance of the meeting according to the self-selected areas of expertise, the nature of the papers submitted and through a process of discussion with those involved. The key objectives of the groups were to provide a mixture of backgrounds and areas of expertise but a common focus through which cross-disciplinary perspectives on particular pertinent topics could be addressed. At the workshop, the groups and their foci were discussed and consensus reached on four broad topics around which to structure the discussion through which geovisualization should be explored: 9 9 9 9
creating instruments for ideation; using 3D in visualization; connecting people data and resources; making useful and usable geovisualization.
These topics are intentionally overlapping due to the complexity of the issues being addressed and the objectives of the current exercise. Indeed many participants have expertise in more than one area. There is a loose correspondence between the focus topics and the cross-cutting themes identified in the research agenda report (MacEachren and
Exploring Geovisualization
11
Kraak, 2001; see w However, the goal was not to address these cross-cutting challenges in a comprehensive way - an objective that will require more than a short workshop. Instead, the groups and topics around which they were organized reflect the particular expertise of workshop participants as well as progress in some areas over the time since the 2001 report. During the workshop, plenary sessions at which each contributor made a brief presentation to the full body of participants allowed cross-fertilization of ideas. Parallel discussion sessions directed to the four identified topics supported focused, crossdisciplinary debate. The synergy that occurred as a consequence of this activity has resulted in these topics being used to structure this volume of contributions derived from the meeting (with one section for each topic). A primary charge to each of the groups was the development of a cross-disciplinary perspective that could be presented to the wider community as a context-providing introduction to the topic under discussion. Thus, each section in "Exploring Geovisualization" begins with a coauthored introductory chapter. Much of the workshop time was spent in discussion groups, developing the structure for these overview chapters, sketching out their contents and debating the issues that arose. These joint contributions are designed to enable us to record and comment upon aspects of our work that we can share and to identify ways in which we can work together for mutual benefit. They are designed to meet the key requirement of interdisciplinary collaboration by providing a shared perspective on the research topic, demonstrating some of the similarities and differences between the various disciplines represented and identifying potential for further collaboration. The draft chapters developed from the position papers and the group discussions were finalised shortly after the meeting following dialogue and feedback in a multi-disciplinary context in London. Group chapters were then further revised by a lead author through an iterative process of re-drafting. Both the group chapters and all individual chapters submitted for consideration in the volume underwent a two-tiered process of review. The editors provided detailed feedback to authors on initial drafts for chapters. Then, revised papers were subsequently reviewed formally by another of the editors and one anonymous external reviewer. Reviewers were informed of the objectives of Exploring Geovisualization and the fact that chapters were designed to be read by a relatively broad audience including graduate students and researchers in disciplines ranging from Cartography and other areas of GIScience, through Information Visualization and EDA in statistics, to Scientific Visualization and Computer Graphics. Contributors of accepted papers were asked to address the issues raised in the review process and document their responses. The result is the structured collection of 34 individual and joint contributions that enable us to explore the nature, scope and future of geovisualization from a number of perspectives.
1.6
Exploring Geovisualization - Structure
This book itself consists of an introductory section, followed by the sections relating to each of the four discussion topics and a conclusion. The links between geovisualization
12
J. Dykes, A.M. MacEachren & M.-J. Kraak
and Information Visualization dominate, which is unsurprising as the majority of contributors regarded "Information Visualization" as being a primary or secondary area of expertise. Three additional authors (not in attendance at the workshop) were invited to produce chapters for consideration in this book that discuss geovisualization at a general level from key perspectives, specifically those of GIScience, Information Visualization (InfoVis), and human computer interaction design (HCID). These scenesetting perspectives provide an introductory section, contextualizing the following sections that represent each of the workshop discussion topics.
1.6.1
Section A~Geovisualization in context
The chapters in Section A from the Computer Science Institute at the University of Konstanz document the current scope of Information Visualization, report upon techniques deemed relevant to geovisualization and identify areas of mutual opportunity. "Usability" emerges as a theme in a number of contributions in various sections and so Catherine Plaisant of the Human-Computer Interaction Laboratory at the University of Maryland builds upon this introduction to Information Visualization by identifying themes and challenges in usability. With this important contextual material on the nature of Information Visualization in place, the final chapter of our introductory section is a contribution by Mark Gahegan that grounds visualization in applications of GIScience. Gahegan relates opportunities for taking advantage of visualization methods and tools to the various forms of reasoning used throughout the geoscientific process.
1.6.2
Section B ~ C r e a t i n g instruments for ideation
Section B is the book's longest and focuses on software approaches to geovisualization and the development of instruments that support the EDA process specifically and ideation more generally. A number of the research challenges are relevant to this section in which experienced software authors and developers provide perspectives. In the coauthored introductory chapter, Gennady Andrienko and colleagues document a diverse range of software approaches to geovisualization and provide a structured explanation for the developments that are currently taking place. These include advances in technology, data availability, changing tasks, a variety of users, the use of expertise from associated disciplines and opportunities for interoperability. This co-authored perspective is followed by a chapter from Martin Theus, author of the Mondrian software, who identifies a number of links between statistical data exploration and geovisualization. Peter Rodgers then describes the potential and current use of graph drawing techniques from Information Visualization within geovisualization. Next, Jonathan Roberts reviews tools, methodologies and models for supporting multiple-linked views and suggests ways in which linked views may be coordinated and utilized in geovisualization instruments for exploratory work. This is followed by a contribution from David Mountain, who introduces approaches and software tools for exploring spatio-temporal behaviour with the objective of summarizing and explaining
Exploring Geovisualization
13
an individual's use of space. The knowledge gained can be a useful contributor to the provision of location dependent digital information. Natalia Andrienko and colleagues then draw upon task and data typologies when describing the way in which the characteristics of data and the nature of the task in hand drive their visualization tool design. This focus on tasks is complemented by attention to the user in a chapter by Monica Wachowicz and colleagues, who introduce the Land Use Change Explorer, a tool that exemplifies an approach to the process of geographic knowledge discovery by drawing upon autonomous agents to support decision-making in planning. In a chapter directed towards method rather than implemented tools, Penny Noy presents "signature exploration" - a means of aiding user comprehension when faced with complex and abstract graphics representing large structured data sets in the kinds of tools described in this section. Finally, Jason Dykes describes some of the approaches employed and software developed to support geovisualization and identifies a number of methods of achieving high levels of interaction with data to permit geovisualization through a variety of interfaces that minimize "visualization effort".
1.6.3
Section C---Using 3D in visualization
Our second topic relates to the use of"3D" in geovisualization and a range of experts offer perspectives on matters varying from theoretical issues to the development of rendering algorithms in section C. The research relates most strongly to geovisualization challenge 1 (see w1.3.1), but large data sets, collaborative work and usability issues are also addressed. By way of introduction, Jo Wood and colleagues describe the visualization pipeline and apply three-dimensionality to each of its five stages in exploring the various links between geovisualization, technology and cartographic theory. Three-dimensionality is considered in the stages of data management, data assembly, visual mapping, rendering and display. The artifacts produced and used at each stage are associated with various meanings of the term "3D" including the raw data, the assembled data, visual representations of the data, the image and the schemata used in interpretation. Jo Wood, then provides the initial individual chapter for the section in which he considers the influences of three components of scale on measurements made from surfaces. Rendering techniques are employed to represent these in a real-time dynamic 3D environment for geovisualization. Jiirgen DOllner then assesses the impact of realtime 3D computer graphics on geovisualization. Key techniques are identified that allow us to design and implement new geovisualization strategies, systems, and environments with improved visual expressiveness and interactivity. Next, Adriano Lopes and Ken Brodlie describe the evolution of contouring and isosurfacing techniques to account for the interactive requirements of geovisualization. Sabine Kirschenbauer follows this with discussion of an empirical study into the use of the Dresden 3D LC Display (D4D) - a flat autostereoscopic display for the visual presentation of geographic data in "true 3D". The results suggest that the visual capabilities of the user, their level of experience and the task in hand affect the success of such techniques. Lars Bodum then explores the scope of "virtual environments" by providing a typology that focuses on representational aspects involved in their
14
J. Dykes, A.M. MacEachren & M.-J. Kraak
construction. The level of abstraction and the temporal characteristic of the model are of particular concern here as the philosophical and technical issues associated with the design and use of virtual environments are explored. Finally, Lloyd Treinish offers a chapter solicited subsequent to the workshop to introduce and demonstrate the Webbased 3D geovisualization of complex spatio-temporal data in an operational environment. Techniques for the automated dissemination of animations of 3D weather models to a wide range of users are discussed and a number of approaches, products and implementations are presented.
1.6.4
Section D~Connecting people, data and resources
The third topic explored in this book is "distributed geovisualization", defined as geovisualization involving distributed data, resources, and people. By way of introduction to section D, Ken Brodlie and colleagues develop an example scenario to demonstrate the importance of connecting a range of people, data and resources to coordinate approaches and expertise in a geovisualization context. The scenario allows the group to demonstrate how a distributed approach to geovisualization can bring considerable benefits to science and society where there is a need to draw upon the combined force of data, resources and people from across the globe. Challenge 3 is addressed most directly (see w but each of the geovisualization challenges is considered under this topic. Alan MacEachren calls for multi-disciplinary collaboration when introducing a framework for geovisualization involving groupwork that draws upon research activity in the GeoVISTA Center at Penn State University. The framework discriminates between the use of visual representations in supporting groupwork as the object of collaboration, in providing support for dialogue and in supporting coordinated activity. In the next chapter, Ken Brodlie develops three system architecture models for Internet-based, realtime collaborative visualization. A number of tools for supporting real-time distributed work and experience of collaborative visualization are introduced with the inference that particular combinations of the approaches may be most appropriate. In the subsequent chapter, Michael Schroeder introduces enabling technologies for intelligent information integration. The relationships between geovisualization and developments in these areas, such as Grid computing, the Semantic Web and opportunities for visual datamining are discussed. In a complementary paper, Zarine Kemp focuses on the semantic requirements that enable distributed, disparate data resources to be shared by presenting a rationale for semantics to be integrated into interfaces for geovisualization. A prototype fisheries management system is introduced by way of demonstration. Taking a cartographic perspective on design of visualization to support everyday activity, David Fairbairn explores the relationships between geovisualization and distributed public transport information systems and relates general issues in geovisualization to this particular field with examples. Finally, in another chapter focused on distributed geovisualization relating to travel, Volker Coors and colleagues report the results of a pilot study designed to evaluate methods of assisting mobile users with route instructions that include text, sketches, spoken instructions and the use of 3D models. A series of technical issues are
Exploring Geovisualization
15
addressed and guidelines developed based upon the empirical results to determine the most suitable form of representation to use in a particular situation.
1.6.5
Section E~Making useful and useable geovisualization
The final discussion topic centers on issues of geovisualization design and evaluation, focusing most strongly on challenges associated with the development of theory and practice to support universal access and usability. In their introduction to section E, Sven Fuhrmann and colleagues with backgrounds in Computer Science, Information Visualization, GIScience, Geography and Cartography emphasize that geovisualization design is not just about technical issues. They explore various means of making geovisualization tools and techniques useful and usable from a user' s perspective and aim to bridge the gap between developers and users by introducing methods and discussing research questions in user-centered geovisualization tool design. In the first contribution, Colin Ware and Matthew Plumlee characterize the structure of visual space in three ways. They demonstrate how a consideration of the perceptual structure of space, the costs associated with gaining extra information by navigation and the cognitive mechanism used to visually interrogate geo-spatial representations can enable us to identify forms of navigation interface that will be most suitable for particular tasks. Then, Robert Edsall and Laura Sidney describe approaches for representing interactions with time in tools for geovisualization. Cognitive science and usability engineering are employed to inform the development of a framework that helps cartographers design appropriate geovisualization interfaces and environments to support insight. Next, Sven Fuhrmann and William Pike aim to engage users in geovisualization tool design and evaluation when considering the user-centered design process for collaborative tools that support interaction. A case study demonstrates how three approaches to supporting user-centered design between distributed users and tool developers have been employed to enhance remote collaboration in the environmental sciences. In the subsequent chapter, Paula Ahonen-Rainio and Menno-Jan Kraak make use of multi-variate visualization techniques in proposing an environment for the exploration of geographic metadata that is designed to support users in determining how well available datasets meet their needs. In a chapter focusing on integration of visualization with other analytical methods, Etien Koua and Menno-Jan Kraak present a prototype exploratory geovisualization environment that combines visual and computational analysis. The tool draws upon Information Visualization techniques and is based upon a usability framework developed to analyze the ways in which various abstract representations are used and understood in relation to more traditional maps. Carolina Tob6n then provides a complementary outline of the contributions that experimental design and HCI may make in producing geovisualization tools that are usable, useful and fit for purpose and develops an approach to tool design that is based upon user tasks. An experiment is presented that shows how multiple techniques can be combined to evaluate the effectiveness of a system and to address the complex issue of task definition in geovisualization. The results suggest that the approach can refine existing task characterizations to reflect the cognitive visual operations supported by geovisualization
16
J. Dykes, A.M. MacEachren & M.-J. Kraak
environments. Finally, Sara Fabrikant and Andrd Skupin consider information spaces based upon spatial metaphors and use geographic information theory and principles of ontological modeling to develop a framework for the construction of cognitively plausible semantic information spaces. Examples of the application of the framework to the design of cognitively adequate information spaces are provided with the aim of supporting and augmenting the internal visualization capabilities of those who use them. 1.7
Exploring Geovisualization--Visualization
The structure presented here and described in w1.6 is somewhat linear, due to the nature of the printed medium that we are using and the organisation of the discussion groups. Yet the inter-related nature of the topics under discussion here means that some chapters could logically be grouped in more than one section. Similarly, many workshop participants were keen to comment upon the activities of groups other than their own and plenty of cross-fertilization between groups occurred at the London meeting. Citations are used throughout the book to relationships between the contributions in their own use of the book. We also provide a number of "maps" of the contents of the book to draw attention to relationships between the chapters and to guide the reader through the interrelated concepts. These are spatializations of the semantic relationships between the various chapters. The cover of the book shows a topical density surface using principal coordinate ordination (see Preface). Semantic links between chapters in the book are shown in Figure 1.1. PathFinder Network Scaling (PFNet) solutions (Schvaneveldt, 1990) are used here to spatialize such semantic relationships between documents as detailed by Skupin and Fabrikant (2003) and discussed further by Fabrikant and Skupin, this volume (chapter 35). In this case, estimates of the similarities between the contents of each of the chapters in Exploring Geovisualization are used to establish the most notable links. These similarity estimates are derived by considering the textual content of the chapter titles and keywords using Latent semantic Indexing (LSI) (Deerwester et al., 1990). 1 The PFNet algorithm then generates a topological network that preserves these semantic similarity relationships identified by LSI. The semantic network can be subsequently transformed and reproduced graphically using appropriate layout techniques (see Rodgers, this volume (Chapter 7)) as shown in Figure 1.1. Two parameters are used to generate a PFNet: the q-parameter, which constrains the maximal path length (number of links between points) adhering to the triangle inequality criterion (and is thus an integer between 2 and n - 1); the r-parameter, a value between 2 and infinity, which defines the metric used for computing the distance paths and is inversely proportional to the number of links in the network generated. Within certain constraints, q and r can be experimented with to 1 A series of network were generated using the full abstracts, but the relationships depicted were judged to be less meaningful than those presented here. Possible explanatory factors that may have resulted in relationships that did not relate whollyto the chapterthemesinclude the length of abstract, the style of the abstract and the first language of the author. On reflection the use of a closed list in generating chapter keywords may well have improved the PFNets.
I
r Figure 1.l. Exploring Geovisualization-PFNet solutions showing the relationships between chapters in this book according to titles and keywords. (a) PFNet (infinity,35)-in essence a minimum spanning tree showing only the most salient connections. (b) PFNet (infinity,2)-a highly linked network showing some of the more subtle relationships between chapters. The digital appendices include a number of PFNet solutions that readers can use to navigate between related chapters.
18
J. Dykes, A.M. MacEachren & M.-J. Kraak
produce suitable networks and any solution that uses a particular combination of q and r is referred to as PFN (r,q). The minimum-spanning tree, an undirected graph of minimal length that spans all the nodes of the network, is generated when q = n - 1. This is shown in Figure 1.1a, where 35 links are depicted connecting all 36 nodes. An alternative network with more linkages is shown in Figure 1.lb. Weaker links are included in the PFNet solution in this case. Fabrikant (2001) describes better connected documents within spatializations such as those shown in Figure 1.1 as "semantic anchor points for information exploration". We hope that the authors and readers of Exploring Geovisualization will find these static graphics useful in mapping out some of the semantic relationships between the chapters and the work that they represent. The digital supplement to the book contains a number of PFNet solutions. These maps take advantage of the digital medium to provide interactive interfaces to the digital appendices that will help readers relate chapters and navigate through the information presented in an exploratory manner.
1.8
Exploring Geovisualization - Conclusion
The result of the exercise reported here is a broad and considered volume of work with opinions and influences from a range of disciplines. We do not argue that this body of material is a comprehensive statement on the status of geovisualization and related fields. But what we have cooperatively developed is a wide-ranging collection of contributions that are clearly informed by existing descriptions of research requirements (MacEachren and Kraak, 2001). These contributions have been produced by a group of authors from diverse disciplines with a variety of perspectives and have evolved following discussion amongst colleagues from a number of fields. As a result, this volume provides a platform for exploring the nature of geovisualization, its interdependencies with other academic fields of study and viceversa. This is important as we advance the evolving field of geovisualization in terms of our collective knowledge, the techniques that we use, the tasks we address and the ways in which we address them. It is also important that knowledge is shared and collaborative work is planned, completed and reported as relationships between practitioners in increasingly related disciplines develop, enabling us to make progress with the various research challenges. Our collective perspectives have expanded considerably through this examination of geovisualization and in a concluding chapter we reflect on the contributions presented and on ways that we might continue advancing geovisualization. We hope and anticipate that the reader will have an equally enlightening experience as they use the chapters that follow to explore geovisualization.
Acknowledgements The contributions made by Andr6 Skupin and in particular Sara Fabrikant in developing, generating and advising upon the pathfinder network scaling solutions are greatly appreciated and acknowledged with thanks.
Exploring Geovisualization
19
References Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harschman, R., (1990) "Indexing by latent semantic analysis", Journal of the American Society of Information Science, 41, 391-407. Fabrikant, S. I. (2001) Visualizing region and scale in semantic spaces. Proceedings of the 20th International Cartographic Conference, ICC 2001, Beijing, China, pp. 2522-2529. MacEachren, A. M., and Kraak, M. J., (200 l a) "Research challenges in geovisualization, Cartography and Geographic Information Science", Special Issue on Geovisualization, 28(1), 3-12. MacEachren, A. M. and Kraak, M. -J. (eds.) (2001b) "Research challenges in geovisualization, Cartography and Geographic Information Science", Special Issue on Geovisualization, 28(1), American Congress on Mapping and Surveying, 80 pp. Schvaneveldt, R. W., (ed.), (1990) Pathfinder Associative Networks: Studies in Knowledge Organization. Norwood, NJ: Ablex. Skupin, A., and Fabrikant, S. I., (2003) "Spatialization methods: a cartographic research agenda for non-geographic information visualization", Cartography and Geographic Information Science, 30(2), 95-119.
This Page Intentionally Left Blank
Section A Geovisualization in Context Perspectives from Related Disciplines
2. Information Visualization: Scope, Techniques and Opportunities for Geovisualization Daniel A. Keim, Christian Panse & Mike Sips ............................................................................................. 23 3. Information Visualization and the Challenge of Universal Usability Catherine Plaisant ........................................................................................................................................... 53 4. Beyond Tools: Visual Support for the Entire Process of GIScience Mark Gahegan ................................................................................................................................................ 83
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 2
Information Visualization: Scope, Techniques and Opportunities for Geovisualization Daniel A. Keim, Christian Panse, Mike Sips, Computer Science Institute, University of Konstanz, Konstanz, Germany
Keywords: Information Visualization, visual datamining, visual data exploration, geovisualization
Abstract Never before in history has data been generated at such high volumes as it is today. Exploring and analyzing the vast volumes of data has become increasingly difficult. Information visualization and visual datamining can help to deal with the flood of information. The advantage of visual data exploration is that the user is directly involved in the datamining process. There are a large number of Information Visualization techniques that have been developed over the last two decades to support the exploration of large data sets. In this chapter, we provide an overview of Information Visualization and visual datamining techniques, and illustrate them using a few examples. We demonstrate that the application of Information Visualization methods provides new ways of analyzing geospatial data.
2.1
Introduction
The progress made in hardware technology allows today's computer systems to store very large amounts of data. Researchers from the University of Berkeley estimate that every year about 2 exabyte ( = 2 million terabytes) of data are generated, of which a large portion is available in digital form (growth rate is about 50%). This means that in the next 3 years, more data will be generated than in all of human history to date. The data are often automatically recorded via sensors and monitoring systems. Almost all transactions of every day life, such as purchases made with credit card, Web pages visited or telephone calls are recorded by computers. In many application domains, it might also be useful that much of the data include geospatial referencing. For example, credit card purchase transactions include both the address of the place of purchase and of the purchaser, telephone records include addresses and sometimes coordinates or at least cell phone zones, data from satellite remote sensed data contain coordinates or other 23
24
D.A. Keim, C. Panse & M. Sips
geographic indexing, census data and other government statistics contain addresses and/or indexes for places, and information about property ownership contains addresses, relative location info, and sometimes coordinates. These data are collected because people believe that they are a potential source of valuable information, providing a competitive advantage (at some point) to their holders. Usually many parameters are recorded, resulting in data with a high dimensionality. With today's data management systems, it is only possible to view quite small portions of the data. If the data are presented textually, the amount of data that can be displayed is in the range of some one hundred data items, but this is like a drop in the ocean when dealing with data sets containing millions of data items. Having no possibility to adequately explore the large amounts of data that have been collected because of their potential usefulness, the data become useless and the databases become data "dumps".
2.1.1
Benefits of visual data exploration
For datamining to be effective, it is important to include the human in the data exploration process and combine the flexibility, creativity, and general knowledge of the human with the enormous storage capacity and the computational power of today's computers Visual data exploration aims at integrating the human into the data exploration process, applying human perceptual abilities to the analysis of large data sets available in today' s computer systems. The basic idea of visual data exploration is to present the data in some visual form, allowing the user to gain insight into the data, draw conclusions, and directly interact with the data. Visual datamining techniques have proven to be of high value in exploratory data analysis, and have a high potential for exploring large databases. Visual data exploration is especially useful when little is known about the data and the exploration goals are vague. Since the user is directly involved in the exploration process, shifting and adjusting the exploration goals is automatically done if necessary. Visual data exploration can be seen as a hypothesis generation process, for example, (see Gahegan, this volume (Chapter 4)); the visual representations of the data allow the user to gain insight into the data and come up with new hypotheses. The verification of the hypotheses can also be achieved through data visualization, but may also be accomplished using automatic techniques from statistics, pattern recognition, or machine learning. In addition to the direct involvement of the user, the main advantages of visual data exploration over automatic datamining techniques are that: 9 9
9
visual data exploration can easily deal with highly non-homogeneous and noisy data; visual data exploration is intuitive and requires much less understanding of complex mathematical or statistical algorithms or parameters than other methods; visualization can provide a qualitative overview of the data, allowing data phenomena to be isolated for further quantitative analysis.
Scope, Techniques and Opportunities for Geovisualization
25
Visual data exploration does, however, include some underlying data processing, dimension reduction, re-mapping from one space to another, etc., which typically does require some basic understanding of the underlying algorithm to understand it properly. As a result, visual data exploration can lead to quicker and often superior results, especially in cases where automatic algorithms fail. In addition, visual data exploration techniques provide a much higher degree of confidence in the findings of the exploration. This fact leads to a high demand for visual exploration techniques and makes them indispensable in conjunction with automatic exploration techniques.
2.1.2
Visual exploration paradigm
Visual data exploration usually follows a three-step process: overview first, zoom and filter, and then details-on-demand. This has been termed the "information seeking mantra" (Shneiderman, 1996). First, the user needs to get an overview of the data. In the overview, the user identifies interesting patterns or groups in the data and focuses on one or more of them. For analyzing the patterns, the user needs to drill-down and access details of the data. Visualization technology may be used for all three steps of the data exploration process. Visualization techniques are useful for showing an overview and allowing the user to identify interesting subsets. In this step, it is important to keep the overview visualization while focusing on the subset using another visualization technique (for discussion of the perceptual factors in designing such multi-scale displays, for example, (see Ware and Plumlee, this volume (Chapter 29)). An alternative is to distort the overview visualization in order to focus on the interesting subsets. This can be performed by dedicating a larger percentage of the display to the interesting subsets while decreasing screen utilization for uninteresting data. This idea is also discussed by Plaisant (this volume, Chapter 3) and illustrated by D611ner (this volume, Chapter 16). To further explore the interesting subsets, the user needs a drill-down capability in order to observe the details about the data. Note that visualization technology does not only provide the base visualization techniques for all three steps but also bridges the gaps between the steps. The design and implementation of instruments created for supporting this kind of activity are addressed in section B of this book.
2.2
Classification of Visual Datamining Techniques
Information visualization focuses on data sets lacking inherent 2D or 3D semantics and therefore also lacking a standard mapping of the abstract data onto the physical screen space. There are a number of well known techniques for visualizing such data sets, such as X - Y plots, line plots, and histograms. These techniques are useful for data exploration but are limited to relatively small and low-dimensional data sets. In the last decade, a large number of novel Information Visualization techniques have been developed, allowing visualizations of multi-dimensional data sets without inherent two- or 3D semantics. Good overviews of the approaches can be found in a number of recent books (e.g., Card et al., 1999; Ware, 2000; Spence, 2001; Schumann and Mtiller, 2000). The techniques can be classified based on three criteria (Figure 2.1): the data to be visualized,
D.A. Keim, C. Panse & M. Sips
26
Data to be Visualized 1. one-dimensional 2. two-dimensional
|
~
VisualizatioTechni n que
3 multi-dimensional
4. text/web
ked Display
~_
5 hierarchies/graphs [
/'~ / "
6 algorithm/software I / "
Dense Pixel Display
Geometrically-transformed Display
Standard 2D/3D Display
I/ /~" /
I Standard
I
I
I
',
I
Projection Filtering Zoom Distortion Link&Brush Interaction Technique
Figure 2.1. Classification of information visualization techniques. the visualization technique, and the interaction technique used. The data type to be visualized (Shneiderman, 1996) may be: 9 9 9 9 9 9
1D data, such as temporal (time-series) data. 2D data, such as geographical maps. Multi-dimensional data, such as relational tables. Text and hypertext, such as news articles and web documents. Hierarchies and graphs, such as telephone calls and web documents. Algorithms and software, such as debugging operations.
The visualization technique used may be classified as: 9 9 9 9 9
Standard 2D/3D displays, such as bar charts and X - Y plots. G e o m e t r i c a l l y transformed displays, such as landscapes and parallel coordinates. Icon-based displays, such as needle icons and star icons. Dense pixel displays, such as the recursive pattern and circle segments. Stacked displays, such as treemaps and dimensional stacking.
The third element of the classification is the interaction technique used. Interaction techniques allow users to directly navigate and modify the visualizations, as well as select subsets of the data for further operations. Examples include: 9 9 9 9
Dynamic projection Interactive filtering Interactive zooming Interactive distortion
Scope, Techniques and Opportunities for Geovisualization
9 9
27
Interactive linking Brushing
Note that the three criteria upon which we base our classification - data type to be visualized, visualization technique, and interaction technique - can be assumed to be orthogonal. This means that any of the visualization techniques may be used in conjunction with any of the interaction techniques for any data type. Note also that a specific system may be designed to support different data types and that it may use a combination of visualization and interaction techniques.
2.3
Data Type To Be Visualized
In Information Visualization, the data usually comprise a large number of records, each consisting of a number of variables or dimensions. Each record corresponds to an observation, measurement, or transaction. Examples are customer properties, e-commerce transactions, and sensor output from physical experiments. The number of attributes can differ from data set to data set; one particular physical experiment, for example, can be described by five variables, while another may need hundreds of variables. We call the number of variables the dimensionality of the data set. Data sets may be 1D, 2D, multi-dimensional or may have more complex data types such as text/hypertext or hierarchies/graphs. Sometimes, a distinction is made between grid dimensions and the dimensions that may have arbitrary values.
2.3.1
1D data
1D data usually have one dense dimension. A typical example of 1D data is temporal data. Note that with each point of time, one or multiple data values may be associated. An example is a time series of stock prices (Figure 2.4) or the time series of news data used in the ThemeRiver (Havre et al., 2000).
2.3.2
2D data
2D data usually have two dense dimensions. A typical example is geometry-related planar data in the Euclidian plane. In general, planar data define quantities of distances, or distances and angles, which determine the position of a point on a reference plane to which the surface of a 3D object has been projected. A typical example is geographical data, where the two distinct dimensions are longitude and latitude. Longitude and Latitude describe locations on a 3D surface and some transformation is required to project the relationships between the locations specified in this way on a plane. Besides, depending upon the Cartography used, various characteristics of the relationships between locations are either preserved or lost. After the projection, the geographical data can be stored as 2D data with x/y-dimensions. X - Y plots are a typical method for showing 2D data and maps are a special type of X - Y plot for showing geographical data. Although it seems easy to deal with temporal or geographic data on 2D devices, caution is advised. If the number of records to be visualized is large, temporal axes and maps get quickly cluttered - and may not help to understand the data. There are several approaches to draw
28
D.A. Keim, C. Panse & M. Sips
upon with dense geographic data already in common use, such as Gridfit (Keim and Herrmann, 1998; see Figure 2.6 in w and PixelMap (Keim et al., 2003c; see Figure 2.7 in w
2.3.3
Multi-dimensional data
Many data sets consist of more than three attributes and therefore do not allow a simple visualization as 2D or 3D plots. Examples of multi-dimensional (or multi-variate) data are tables from relational databases, which often have from tens to hundreds of columns (or attributes). Since there is no simple mapping of the attributes to the two dimensions of the screen, more sophisticated visualization techniques are needed, such as parallel coordinates plots (Inselberg and Dimsdale, 1990; Figure 2.12a). A very different method for data with geospatial attributes is to generate multi-variate glyphs anchored on a 2.5D surface representation (Gahegan, 1998).
2.3.4
Text and hypertext
Not all data types can be described in terms of dimensionality. In the age of the World Wide Web, one important data type is text and hypertext, as well as multi-media Web page contents. These data types differ in that they cannot be easily described by numbers, and therefore most of the standard visualization techniques cannot be applied. In most cases, a transformation of the data into description vectors is necessary before visualization techniques can be used. An example for a simple transformation is word counting, which is often combined with a principal component analysis (PCA) or multidimensional scaling to reduce the dimensionality to two or three, for example, (see Fabrikant and Skupin, this volume (Chapter 35).
2.3.5
Hierarchies and graphs
Data records often have some relationship to other pieces of information. These relationships may be ordered, hierarchical, or arbitrary networks of relations. Graphs are widely used to represent such interdependencies. A graph consists of a set of objects, called nodes, and connections between these objects, called edges or links. Examples are the e-mail inter-relationships among people, their shopping behavior, the file structure of the hard disk or the hyperlinks in the World Wide Web. There are a number of specific visualization techniques that deal with hierarchical and graphical data. An example is given in Figure 2.2 that shows a snapshot of the Internet core (see also Cooperative Association for Internet Data Analysis, 2003). A fine overview of hierarchical Information Visualization techniques can be found in Chen (1999), an overview of Web visualization techniques is presented in Dodge (2001) and an overview book on all aspects related to graph drawing is by Battista et al. (1999).
2.3.6
Algorithms and software
Algorithms and software constitute another class of data. Coping with large software projects is a challenge. The goal of software visualization is to support software
Scope, Techniques and Opportunities for Geovisualization
29
Figure 2.2. Cooperative Association for Internet Data Analysis (CAIDA), k claffy "Skitter Graph" Internet Map, 2000.
92000 Regents of the University of California. Courtesy University of California.
Figure 2.4. Dense pixel displays: recursive pattern technique showing 50 stocks in the Frankfurt AIIgemeine Zeitung (Frankfurt Stock Index Jan 1975-April 1995). The technique maps each stock value to a colored pixel; high values correspond to bright colors. 92004 IEEE.
D.A. Keim, C. Panse & M. Sips
30
development by helping to understand algorithms (e.g., by showing the flow of information in a program), to enhance the understanding of written code (e.g., by representing the structure of thousands of source code lines as graphs), and to support the programmer in debugging the code (e.g., by visualizing errors). There are a large number of tools and systems that support these tasks. Comprehensive overviews of software visualization can be found in Trilk (2001) and Stasko et al. (1998). Examples of visualization, and visually enabled steering, of process models that depict geographic processes include the AGP-System for ocean flow model visualization (Johannsen and Moorhead, 1995), and visualization methods/simulation steering for the 3D turbulence model of Lake Erie (Marshall et al., 1987).
2.4
Visualization Techniques
There are a large number of visualization techniques that can be used for visualizing data. In addition to standard 2D/3D-techniques such as X - Y (X-Y-Z) plots, bar charts, line graphs, and simple maps, there are a number of more sophisticated classes of visualization techniques. The classes correspond to basic visualization principles that may be combined in order to implement a specific visualization system.
2.4.1
Geometrically transformed displays
Geometrically transformed display techniques aim at finding "interesting" transformations of multi-dimensional data sets. This class of geometric display methods includes techniques from exploratory statistics (for example, (see Theus, this volume (Chapter 6))) such as scatter plot matrices (Andrews, 1972; Cleveland, 1993) and techniques that can be subsumed under the term "projection pursuit" (Huber, 1985). Other geometric projection techniques include prosection views (Furnas and Buja, 1994; Spence et al., 1995), hyperslice (van Wijk and van Liere, 1993) and the parallel coordinates technique (Inselberg and Dimsdale, 1990). Parallel coordinates plots map the k-dimensional space onto the two display dimensions by using k axes that are parallel to each other (either horizontally or vertically oriented), evenly spaced across the display. The axes correspond to the dimensions and are usually linearly scaled from the minimum to the maximum value of the corresponding dimension. Each data item is presented as a chain of connected line segments, intersecting each of the axes at the location corresponding to the value of the considered dimensions (Figure 2.12a). A geographic extension of the parallel coordinates technique is to build them in 3D, with maps as orthogonal planes to the otherwise flat 2D parallel coordinates (Edsall, 1999).
2.4.2
Iconic displays
Another class of visual data exploration techniques includes iconic display methods. The idea here is to map the attribute values of a multi-dimensional data item to the features of an icon. Icons can be arbitrarily defined. They may for example comprise: simple faces (Chernoff, 1973); needle icons (Abello and Korn, 2001; Keim, 2000); star icons (Ward, 1994); stick figure icons (Pickett and Grinstein, 1988); color icons (Levkowitz, 1991; Keim and Kriegel, 1994); TileBars (Hearst, 1995). The visualization is generated by
Scope, Techniques and Opportunities for Geovisualization
31
mapping the attribute values of each data record to the features of the icons. In case of the stick figure technique, for example, two dimensions are mapped to the display dimensions and the remaining dimensions are mapped to the angles and/or limb length of the stick figure icon. If the data items are relatively dense with respect to the two display dimensions, the resulting visualization presents texture patterns that vary according to the characteristics of the data and are therefore detectable by pre-attentive perception. Figure 2.3 shows an example of this class of techniques. Each data point is represented by a star icon/glyph, where each data dimension controls the length of a ray emanating from the center of the icon. In this example, the positions of the icons are determined using PCA to convey more information about data relations. Other data attributes could also be mapped to icon position.
2.4.3
Dense pixel displays
The rationale behind the development of dense pixel techniques is to map each dimension value to a colored pixel and group the pixels belonging to each dimension into adjacent areas. More precisely, dense pixel displays divide the screen into multiple subwindows. For data sets with m dimensions (attributes), the screen is partitioned into m subwindows, one for each of the dimensions. Since in general dense pixel displays use one pixel per
$
@
Figure 2.3. The iris data set, displayed using star glyphs positioned based on the first two principal components (from XmdvTool; Ward, 1994). Reproduced with permission of M. Ward, Worcester Polytechnic Institute 92004 IEEE.
32
D.A. Keim, C. Panse & M. Sips
data value, the techniques allow the visualization of the largest amount of data possible on current displays (up to 1,000,000 data values on a 1,000,000 pixel screen). Although, it seems relatively straightforward to create dense pixel displays, there are some important questions to be taken into account.
Color mapping Firstly, finding a path through a color space that maximizes the numbers of just noticeable differences (JNDs), but at the same time, is intuitive for the application domain is a difficult task. The advantage of color over grey scales is that the number of JNDs is much higher.
Pixel arrangement If each data value is represented by one pixel, the main question is how to arrange the pixels on the screen within the subwindows. Note, that only a good arrangement due to the density of the pixel display will allow a discovery of clusters, local correlations, dependencies, other interesting relationships among the dimensions, and hot spots. For the arrangement of pixels, we have to distinguish between data sets that have a natural ordering of the data objects, such as time series, and data sets without inherent ordering, such as is the case when visualizing query result sets. Even if the data has a natural ordering to one attribute, there are many possible arrangements. One straightforward possibility is to arrange the data items from left to right in a line-by-line fashion. Another possibility is to arrange the data items top-down in a column-by-column fashion. If these arrangements are done pixel wise, in general, the resulting visualizations do not provide useful results. More useful are techniques that provide a better clustering of closely related data items and allow the user to influence the arrangement of the data. Techniques that support the clustering properties are screenfilling curves. Another technique that supports clustering is the recursive pattern technique (Keim et al., 1995a,b). This technique is based upon a generic recursive back-and-forth arrangement of the pixels and is particularly aimed at representing datasets with a natural order according to one attribute (e.g., time-series data). The user may specify parameters for each recursion level, and thereby control the arrangement of the pixels to form semantically meaningful substructures. The base element on each recursion level is a pattern of height hi and width wi as specified by the user. First, the elements correspond to single pixels that are arranged within a rectangle of height h l and width Wl from left to right, then below backwards from right to left, then again forward from left to right, and so on. The same basic arrangement is done on all recursion levels with the only difference being that the basic elements that are arranged on level i are the patterns resulting from the level i - 1 arrangements. In Figure 2.4, an example of a recursive pattern visualization is shown. The visualization shows 20 years of financial data (January 1974-April 1995) with the daily prices of the 50 stocks contained in the Frankfurt Stock Index (FAZ).
Shape of subwindows The next important question is whether there exists an alternative to the regular partition of the screen into rectangular subwindows. The rectangular shape of the subwindows
Scope, Techniques and Opportunities for Geovisualization
33
supports screen usage effectively, but on the other hand, the rectangular shape leads to a dispersal of the pixels belonging to one data object over the whole screen. The subwindows of each dimension can be relatively distant, especially for data sets with many dimensions. This can make it particularly difficult to detect clusters, correlations, and interesting patterns. An alternative shape of the subwindows is the circle segments technique (Ankerst et al., 1996) in which the data dimensions are displayed as segments of a circle.
Ordering of dimensions Finally, the last question to consider in dense pixel displays is the ordering of the dimensions. This is actually a case of a more general issue that arises for a number of other visualization techniques, such as the parallel coordinate plots. The basic problem is that the data dimensions have to be arranged in some 1D or 2D ordering on the screen. The ordering of dimensions, however, has a major impact on the expressiveness of the visualization. If a different ordering of the dimensions is chosen, the resulting visualization may change completely and lead to diverse interpretations. More details about designing pixel-oriented visualization techniques can be found in Keim (2000) and Keim and Kriegel (1995) and all techniques have been implemented in the VisDBSystem (Keim and Kriegel, 1994).
2.4.4
Stacked displays
Stacked display techniques are tailored to present data partitioned in an hierarchical fashion. In the case of multi-dimensional data, the data dimensions to be used for partitioning the data and building the hierarchy have to be appropriately selected. An example of a stacked display technique is dimensional stacking (LeBlanc et al., 1990). The notion is to embed one coordinate system inside another coordinate system, i.e., two attributes form the outer coordinate system, two other attributes are embedded into the outer coordinate system, and so on. The display is generated by dividing the outermost level coordinate system into rectangular cells and within the cells the next two attributes are used to span the second level coordinate system. This process may be repeated multiple times. The usefulness of the resulting visualization largely depends on the data distribution of the outer coordinates and therefore the dimensions that are used for defining the outer coordinate system have to be carefully selected. A rule of thumb is to choose the most important dimensions first. A dimensional stacking visualization of mining data with longitude and latitude mapped to the outer x and y axes, as well as ore grade and depth mapped to the inner x and y axes is shown in Figure 2.5. Other examples of stacked display techniques include Worlds-within-Worlds (Feiner and Beshers, 1990a,b), Treemap (Johnson and Shneiderman, 1991; Shneiderman, 1992), and Cone Trees (Robertson et al., 1991).
2.5
Visual Datamining on Geospatial Data
Geospatial data are relevant to a large number of applications. Examples include weather measurements such as temperature, rainfall, wind-speed, etc., measured at a large number
34
D.A. Keim, C. Panse & M. Sips
.:.!. i ~ ~.
~.i..i ~ 5.L&.oi s
!iiiilii i ?i:i.i:i!~:iiii~ iii i:
i i !,!iii
:~
~:~' '"~ '~'~
,i + ~,..~.j.',F[:~Li..
...........
~............... ~ ~
ii ii!i iiii~i iiii:i:ii~:
iiii/ii~iiii:iiii~i~i~ii~)i~it ,i::~:;~;,:~:,:~i i~:iiii:il ~ir::i:i ~ i~i:i~:i::i ii:il;iiiii~iiii:iiii; i~ ii i i ilia: iiii]iii ::i~ili:~ii:: :ii'::::i ::::::::::::::::::::::::: ~+~"i!:!i~i~~~+ :i~:i~li~:i:!:~ii :::::::::::::::::::::::::::::: ~!~-!!i ii!fi~[~.~,i+~i.~**~
::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::::::::::
Figure 2.5. Dimensional stacking visualization of drill hole mining data with longitude and latitude mapped to the outer x-, y-axes and one grade and depth mapped to the inner x-, y-axes. Reproduced with permission of M. Ward, Worcester Polytechnic Institute 92004 IEEE.
of locations, use of connecting nodes in telephone business, load of a large number of Internet nodes at different locations, air pollution in cities, etc. Nowadays, there exist a large number of applications, in which it is important to analyze relationships that involve geographic location. Examples include global climate modeling (measurements such as temperature, rainfall, and wind-speed), environmental records, customer analysis, telephone calls, credit card payments, and crime data. Spatial datamining is the branch of datamining that deals with spatial (location) data. However, to analyze the huge amounts of geospatial data obtained from large databases such as credit card payments, telephone calls, environmental records (that often amount to terabytes of data), it is almost impossible for users to examine the spatial data sets in detail and extract interesting knowledge or general characteristics. Automated datamining algorithms are indispensable for analyzing large geospatial data sets, but often fall short of providing completely satisfactory results. Interactive datamining based on a synthesis of automatic and visual datamining techniques usually does not only yields better results, but also results in a higher degree of user satisfaction and confidence in the findings (Han and Kamber, 1986). Although automatic approaches have been developed for mining geospatial data (Han and Kamber, 1986), they are often no better than simple visualizations of the geospatial data on a map.
2.5.1
Visualization strategy
The visualization strategy for geospatial data can be straightforward. The geospatial data points described by longitude and latitude are displayed on the 2D Euclidian plain using a 2D projection. The two Euclidian plain dimensions x and y are directly mapped to the two physical screen dimensions. The resulting visualization depends on the spatial dimension or extent of the described phenomena and objects. A fine overview can be found in Keim et al. (2003d). Since the geospatial locations of the data are not uniformly distributed on a plane, however, the display will usually be sparsely populated in some regions while in other regions of the display a high degree
Scope, Techniques and Opportunities for Geovisualization
35
of overplotting occurs. There are several approaches to cope with dense geospatial data (Geisler, 2003). One widely used method is 2.5D visualization showing data points aggregated up to map regions. This technique is commercially available in systems such as VisualInsight's In3D (Advizor Solutions Inc, 2003) and ESRI's ArcView (Environmental Systems Research Institute, 2003). An alternative that shows more detail is a visualization of individual data points as bars on a map. This technique is embodied in systems such as SGI's MineSet (Silicon Graphics Inc, 2002) and AT&T's Swift 3D (Keim et al., 1999). A problem here is that a large number of data points are plotted at the same position, and therefore only a small portion of the data is actually displayed. Moreover, due to occlusion in 3D, a significant fraction of the data may not be visible unless the viewpoint is changed.
2.5.2
The VisualPoints-System
The VisualPoints-System solves the problem of over-plotting pixels using the Gridfit algorithm (Keim and Herrmann, 1998). An example is shown in Figure 2.6. The Gridfit algorithm is designed to hierarchically partition the data space. In each step, the data set is partitioned into four subsets containing the data points that belong to the four equally sized subregions. Since the data points may not fit into the four equally sized subregions, we have to determine a new extent of the four subregions (without changing the four subsets of data points) such that the data points of each subset can be visualized in the corresponding subregion. For an efficient implementation of the algorithm, a quadtreelike data structure is used to manage the required information and to support the recursive partitioning process. The partitioning process works as follows: starting with the root of the quadtree, in each step the data space is partitioned into four subregions. The goal of the partitioning is that the area occupied by each of the subregions (in pixels) is larger than the number of pixels to be placed within these subregions. A problem of VisualPoints is that in areas with high overlap, the repositioning depends on the ordering of the points in the database. That is, the first data item found in the database is placed at its correct position, and subsequent overlapping data points are moved to nearby free positions, and so locally appear quasi-random in their placement.
2.5.3
Fast-PixelMap-Technique
The PixelMap-Technique solves the problem of displaying dense point sets on maps, by combining clustering and visualization techniques (Keim et al., 2003c). First, the FastPixelMap algorithm approximates a 2D kernel-density-estimation in the two geographical dimensions performing a recursive partitioning of the dataset and the 2D screen space by using split operations according to the geographical parameters of the data points and the extensions of the 2D screen space. The goal is to: (i) find areas with density in the two geographical dimensions and to (ii) allocate enough pixels on the screen to place all data points, especially of dense regions, at unique positions that are close to each other. The top-down partitioning of the dataset and 2D screen space results in distortion of certain map regions. That means, however, virtually empty areas
36
D.A. Keim, C. Panse & M. Sips
will be shrinking and dense areas will be expanding to achieve pixel coherence. For an efficient partitioning of the dataset and the 2D screen space and an efficient scaling to new boundaries, a new data structure called Fast-PixelMap is used. The Fast-PixelMap data structure is a combination of a gridfile and a quadtree that realizes the split operations in the data and the 2D screen space. The Fast-PixelMap data structure enables an efficient determination of the old (boundaries of the gridfile partition in the dataset) and the new boundaries (those of the quadtree partition in the 2D screen space) of each partition. The old and the new boundaries determine the local rescaling of certain map regions. More precisely, all data points within the old boundaries will be relocated to the new positions within the new boundaries. The rescaling reduces the size of virtually empty regions and unleashes unused pixels for dense regions. Second, the Fast-PixelMap algorithm approximates a 3D kernel-densityestimation-based clustering in the 3D performing an array-based clustering for each dataset partition. After rescaling all data points to the new boundaries, the iterative positioning of data points (pixel placement step), starting with the densest regions and within the dense regions the smallest cluster is chosen first. To determine the placement sequence, we sort all final gridfile partitions (leaves of the Fast-PixelMap data structure) according to the number of data points they contain. The clustering is a crucial preprocessing step to making important information visible and achieving pixel coherence, which refers to the similarity of adjacent pixels. High pixel coherence makes small pixel clusters perceivable. The final step of the pixel placement is a sophisticated algorithm that places all data points of a gridfile partition to pixels on the output map in order to provide visualizations that are as position-, distance-, and cluster-preserving as possible. An example showing income levels based upon US-Census data is displayed in Figure 2.7.
2.5.4
Cartogram techniques
A cartogram can be seen as a generalization of a familiar land-covering choropleth map. In this interpretation, an arbitrary parameter vector gives the intended sizes of the regions displayed on the map, so a familiar land-covering choropleth is simply a cartogram with sizes proportional to land area. In addition to the classical applications mentioned above, a key motivation for cartograms as a general Information Visualization technique is to have a method for trading off shape and area adjustments. For example, in a conventional choropleth map, high values are often concentrated in highly populated areas, and low values may be spread out across sparsely populated areas. Such maps therefore tend to highlight patterns in less dense areas where few people live. In contrast, cartograms display areas in relation to an additional parameter, such as population. Patterns may then be displayed in proportion to this parameter (e.g., the number of people involved) instead of the raw size of the area involved. Example applications in the literature include population demographics (Tobler, 1976; Raisz, 1962), election results (Kocmoud and House, 1998), and epidemiology (Gusein-Zade and Tikunov, 1995). Because cartograms are difficult to make by hand, the study of automated methods is of interest (Dent, 1996).
Scope, Techniques and Opportunities for Geovisualization
37
Figure 2.6. Call volume of AT&T customers at one specified time (midnight EST). Each pixel represents one of 22:000 telephone switches. A uni-color map is used to show the call volume.
Figure 2.7. Comparison of traditional map versus PixelMap. New York State Median Household Income. 1999. This map displays cluster regions, for example on the East side of Central Park in Manhattan, where inhabitants with high income live, or on the East side of Brooklyn, where inhabitants with low income live.
38
D.A. Keim, C. Panse & M. Sips
One of the latest cartogram generation algorithm is the CartoDraw-algorithm which was recently proposed as a practical approach to cartogram generation (Keim et al., 2003a,b). The CartoDraw method incrementally repositions the vertices of the map's polygons by means of scanlines. Local changes are applied if they reduce total area error without introducing excessive shape error. Scanlines may be determined automatically, or entered interactively. The main search loop runs over a given set of scanlines. For each, it computes a candidate transformation of the polygons, and checks it for topology and shape preservation. If the candidate passes the tests, it is made persistent; otherwise it is discarded. The scanline processing order depends on their potential for reducing area error. The algorithm runs until the area error improvement over all scanlines falls below a given threshold. A result of the CartoDraw algorithm can be seen in Figure 2.8.
2.6
Interaction Techniques
In addition to the visualization techniques themselves, effective data exploration requires the use of one or more methods of interaction. Interaction techniques allow the data analyst to directly interact with the visualizations and dynamically change the visualizations according to the exploration objectives. Besides, they also make it possible to relate and combine multiple independent visualizations.
Figure 2.8. US State population cartogram showing the presidential election result of 2000. The area of the states in the cartograms corresponds to the electoral voters, the color corresponds to the percentage of the vote. A bipolar colormap depicts the candidate winning each state.
Scope, Techniques and Opportunities for Geovisualization
39
Interaction techniques can be categorized based upon the effects they have on the display. Navigation techniques focus on modifying the projection of the data onto the screen, using either manual or automated methods. View enhancement methods allow users to adjust the level of detail on part or all of the visualization, or modify the mapping to emphasize some subset of the data. Selection techniques provide users with the ability to isolate a subset of the displayed data for operations such as highlighting, filtering, and quantitative analysis. Selection can be done directly on the visualization (direct manipulation) or via dialog boxes or other query mechanisms (indirect manipulation). Some examples of interaction techniques are described below.
2.6.1
Dynamic projection
Dynamic projection is an automated navigation operation in which the projections are dynamically changed in order to explore a multi-dimensional data set. A classic example is the GrandTour system (Asimov, 1985) that aims to show all interesting 2D projections of a multi-dimensional data set as a series of scatter plots. Note that the number of possible projections is exponential in accordance with the number of dimensions, i.e., it is intractable for large dimensionality. The sequence of projections shown can be random, manual, pre-computed, or data driven. Systems supporting dynamic projection techniques include Xgobi (Swayne et al., 1992; Buja et al., 1996), XLispStat (Tierney, 1991) and ExplorN (Carr et al., 1996).
2.6.2
Interactive filtering
Interactive filtering is a combination of selection and view enhancement. In exploring large data sets, it is important to interactively partition the data set into segments and focus on interesting subsets. This can be done by a direct selection of the desired subset (browsing) or by a specification of properties of the desired subset (querying). Browsing is very difficult for very large data sets and querying often does not produce the desired results, but on the other hand, these tools offer many advantages over traditional controls. Therefore, a number of interactive selection techniques have been developed to improve interactive filtering in data exploration. An example of a tool that can be used for interactive filtering is the Magic Lens (Bier et al., 1993; Fishkin and Stone, 1995) in which a tool similar to a magnifying glass is used to support the filtering of the data directly in the visualization. The data under the magnifying glass is processed by the filter, and the result is displayed differently from the remaining data set, for a 3D example, (see D611ner, this volume (Chapter 16)). Users may also use several lenses with different filters, e.g., magnify, then apply a blur by carefully stacking multiple lenses, which work by applying their effects from back to front. Each lens acts as a filter that screens on some attribute of the data. The filter function only modifies the view only within the lens window. When the lenses overlap, their filters are combined. Other examples of interactive filtering techniques and tools are InfoCrystal (Spoerri, 1993), Dynamic Queries (Ahlberg and Shneiderman, 1994; Eick, 1994; Goldstein and Roth, 1994), Polaris (Stolte et al., 2001), Linked Micromap Plots, and Conditioned Choropleth Maps (Carr et al., 2000a,b; Figure 2.9). Additionally, a type of
40
D.A. Keim, C. Panse & M. Sips
Figure 2.9. A conditioned choropleth map of prostate data of the USA. The mapped prostate data is conditioned by male and female lung cancer. Figure created using Conditioned Choropleth Mapping System (MacEachren, 2003) developed under the project Conditioned Choropleth Maps: Dynamic Multi-variate Representations of Statistical Data supervised by Alan MacEachren.
Figure 2.11. Combination of zooming, distortion, and filtering techniques in the area of geographic visualization (Keahey, 1998). Reproduced with permission of A. Keahey, Visintuit. 92004 IEEE.
Scope, Techniques and Opportunities for Geovisualization
41
interactive filtering using a parallel coordinate plot linked to a map is illustrated in MacEachren et al. (2003a,b) and in Gahegan et al. (2002a,b).
2.6.3
Zooming
Zooming is a well-known view modification technique that is widely used in a number of applications. In dealing with large amounts of data, it is important to present the data in a highly compressed form to provide an overview of the data, but at the same time, allowing a variable display of the data at different resolutions. Zooming does not only mean increasing the size of data objects in the display, but also that the data representation may automatically change to present more details at higher zoom levels. The objects may, for example, be represented as single pixels at a low zoom level, as icons at an intermediate zoom level, and as labeled objects at a high resolution. An interesting example applying the zooming idea to large tabular data sets is the Table Lens approach (Rao and Card, 1994). Achieving an overview of large tabular data sets is difficult if the data is displayed in textual form. The Table Lens represents each numerical value through a small bar instead. All bars have a one-pixel height and the lengths are determined by the attribute values. This means that the number of rows on the display can be nearly as large as the vertical resolution and the number of columns depends on the maximum width of the bars for each attribute. The initial view allows the user to detect patterns, correlations, and outliers in the data set. In order to explore a region of interest the user can zoom in, with the result that the affected rows (or columns) are displayed in more detail, possibly even in textual form. Figure 2.10 shows an example of a baseball database with a few rows being selected in full detail. Other examples of techniques and systems that use interactive zooming include PAD+ + (Perlin and Fox, 1993; Bederson, 1994; Bederson and Hollan, 1994), IVEE/Spotfire (Ahlberg and Wistrand, 1995a,b) and DataSpace (Anupam et al., 1995). A comparison of fisheye and zooming techniques can be found in Schaffer et al. (1993).
2.6.4
Distortion
Distortion is a view modification technique that supports the data exploration process by preserving an overview of the data during drill-down operations. In this technique, portions of the data are shown with a high level of detail while others are displayed with lower levels of detail. Popular distortion techniques are hyperbolic and spherical distortions. These are often used on hierarchies or graphs but may be also applied to any other visualization technique. An example of spherical distortions is provided in the Scalable Frame-work paper (Kreuseler and Schumann, 2001). An overview of distortion techniques is provided by Leung and Apperley (1994) and Carpendale et al. (1997). Examples of distortion techniques include bifocal displays (Spence and Apperley, 1982), perspective wall (Mackinlay et al., 1991), graphical fisheye views (Furnas, 1986; Sarkar and Brown, 1994), hyperbolic visualization (Lamping et al., 1995; Munzner and Burchard, 1995), and hyperbox (Alpern and Carter, 1991). Some of the uses of distortion with map displays that combine zooming, distortion and filtering can be found
42
D.A. Keim, C. Panse & M. Sips
Figure 2.10. Table lens showing a table of baseball players performance/classification statistics for 1986. Reproduced with permission from R. Rao, Xerox PARC. 9ACM.
in Keahey (1998). Figure 2.11 shows the effect of distorting part of a familiar landcoveting map to display more detail while preserving context from the rest of the display.
2.6.5
Brushing and linking
Brushing is an interactive selection process that is often, but not always, combined with linking, a process to communicate the selected data to other views of the data set. There are many possibilities to visualize multi-dimensional data, each with their own strengths and weaknesses. The idea of linking and brushing is to combine different visualization methods to overcome the shortcomings of individual techniques. Scatter plots of different projections, for example, may be combined by coloring and linking subsets of points in all projections. In a similar fashion, linking and brushing can be applied to visualizations generated by all visualization techniques described above. As a result, the brushed points are highlighted in all visualizations, making it possible to detect dependencies and correlations. Interactive changes made in one visualization are also automatically reflected in the other visualizations. Note that connecting multiple visualizations through interactive linking and brushing provides more information than considering the component visualizations independently. Typical examples of visualization techniques that have been combined by linking and brushing are multiple scatter plots, bar charts, parallel coordinates, pixel displays, and
Scope, Techniques and Opportunities for Geovisualization
43
Figure 2.12. (a) Parallel coordinates plots. (b) Scatter plot matrix. These figures display US Census data for 50 states. A linked brushing between these multi-variate visualization techniques is used. The plots are linked and brushed by color, which represents the state wise winner party of the 2000 US presidential election (from Ihaka and Gentleman, 1996). The plots show example relationships including an inverse correlation between High School Grad rates and levels of Illiteracy as well as suggesting a tendency towards Republican party voting in states with a low or a high HS Grade.
44
D.A. Keim, C. Panse & M. Sips
maps. Most interactive data exploration systems allow some form of linking and brushing. Examples include Polaris (Stolte et al., 2001) and the Scalable Framework (Kreuseler and Schumann, 2001). Other tools and systems include S-Plus (Becker et al., 1988), XGobi (Swayne et al., 1992; Becker et al., 1996), XmdvTool (Ward, 1994; Figure 2.12), and DataDesk (Velleman, 1997; Wilhelm et al., 1995). An early example of brushing and linking applied to geospatial data was proposed by Monmonier (1989). Most subsequent geovisualization tool kits incorporate brushing and linking. These include cdv (Dykes, 1998), Descartes (Andrienko and Andrienko, 1999a-f) and GeoVISTA Studio (Gahegan et al., 2002a,b; MacEachren et al., 2003a,b; Cook et al., 1997).
2.6.6
Alternative classification from GI-Science
Crampton (2002) proposes an alternative classification of interaction techniques developed in GI Science. The interaction methods are placed in the framework of geovisualization in order to extend the geovisualization emphasis on exploratory, interactive and private functions of spatial displays. Crampton defines a hierarchy of the interaction techniques based on their increasing sophistication. Four categories of interactivity are proposed: the data, the data representation, the temporal dimension, and contextualizing interaction. The following benefits of both classifications can be observed. Firstly, interactivity types allow the user to directly interact with the visualization, dynamically change the visualization, and relate and combine independent visualization techniques in order to build interactive environments. Secondly, interactivity types allow the data analysts as well as the cartographers to compare and critique different views of the data and provide the human with a mechanism for an effective exploration and understanding of the data. In contrast to the alternative classification from GI Science, the classification presented here from the perspective of Information Visualization is based on the visual information seeking mantra (see w This means that the interaction techniques enable the user to modify the visualization for further analysis operations and that there is a strong relationship to exploration and datamining tasks. These interaction techniques can be completely summarized in the category interaction with data. The other categories of Crampton' s classification are more oriented on an interactive data representation, e.g., selection of view points, scaling, etc., and temporal features of the visualization such as fly-throughs and navigations. These interaction techniques are more related to representation tasks and computer graphics research.
2.7
Conclusion
The exploration of large data sets is an important but difficult problem. Information visualization techniques can be useful in solving this problem. Visual data exploration has significant potential, and many applications such as fraud detection and datamining can use Information Visualization technology for an improved data analysis. A significant proportion of all data being stored and processed has geospatial attributes. The task of geovisualization is to view geospatial data and to give insight into
Scope, Techniques and Opportunities for Geovisualization
45
geo-processes and geo-phenomena. In this area, maps are well-known tools for many centuries, but they have several drawbacks from the visualization standpoint. For example, when drawing large geospatial data sets on the screen, the display will usually be sparsely populated in some regions while in other regions of the display, high degree of overplotting occurs, since the geospatial locations are highly non-uniformly distributed in the plane. One of the challenges today is to find out how to deploy efficient visualization strategies to represent large geospatial data sets. Avenues for future work include the tight integration of visualization techniques from the Information Visualization area with methods from the GI Science and with traditional techniques from such disciplines as Statistics, Machine learning, Operations Research, and Simulation. Integration of visualization techniques and these more established methods would combine fast automatic datamining algorithms with the intuitive power of the human mind, improving the quality and speed of the datamining process. The intention of this chapter is to share ideas and connecting methods from both the Information Visualization and geovisualization disciplines. The ultimate goal is to bring the power of integrated visualization technology from both areas to every desktop to allow a better, faster and more intuitive exploration of very large geospatial data resources. This will not only be valuable in an economic sense but will also stimulate and delight the user.
Acknowledgments We gratefully acknowledge Stephen North from the AT&T Laboratories for his contribution to this research. "Skitter Graph" Internet Map 9 Regents of the University of California. Courtesy University of California. This chapter draws upon a framework that has been previously published (Keim, 2001).
References Abello, J., and Korn, J., (2001) "MGV: a system for visualizing massive multi-digraphs", Transactions on Visualization and Computer Graphics. Advizor Solutions Inc., (2003) Data Visualization Tools and Analytic Business Technology - ADVIZOR Solutions Inc. Online: http://www.advizorsolutions.com/ (23/10/03). Ahlberg, C., and Shneiderman, B., (1994) "Visual information seeking: tight coupling of dynamic query filters with starfield displays", In: Adelson, B., Dumais, S., and Olson, J. S., (eds.), CHI'94 Human Factors in Computing Systems. Boston, MA: ACM, pp. 313-317, (24-28 April). Ahlberg, C., and Wistrand, E., (1995a) "IVEE: an environment for automatic creation of dynamic queries applications", Proceedings of CHI '95: Human Factors in Computing Systems. ACM. Ahlberg, C., and Wistrand, E., (1995b) "IVEE: an Information Visualization and exploration environment", Proceedings International Symposium on Information Visualization, Atlanta, GA, pp. 66-73.
46
D.A. Keim, C. Panse & M. Sips
Alpern, B., and Carter, L., (199 l) "Hyperbox", Proceedings Visualization '91, San Diego, CA, pp. 133-139. Andrews, D. F., (1972) "Plots of high-dimensional data", Biometrics, 29, 125-136. Andrienko, G. L., and Andrienko, N. V., (1999a) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Andrienko, G., and Andrienko, N., (1999b). Seasonal migration of white storks, 19981999. Online: http://www.ais.fhg.de/and/java/birds/(23/10/03). Andrienko, G., and Andrienko, N., (1999c) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999d) "Knowledge-based visualization to support spatial data mining", Proceedings of Intelligent Data Analysis. Berlin: Springer Verlag, pp. 149-160. Andrienko, G., and Andrienko, N., (1999e) "Making a GIS intelligent: CommonGIS project view", AGILE'99 Conference, Rome, April 15-17, pp. 19-24. Andrienko, G. and Andrienko, N., (1999f) Seasonal migration of white storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(03/10/02). Ankerst, M., Keim, D. A., and Kriegel, H.-P., (1996) "Circle segments: a technique for visually exploring large multidimensional data sets", Proceedings Visualization 96, Hot Topic Session, San Francisco, CA. Anupam, V., Dar, S., Leibfried, T., and Petajan, E., (1995) "DataSpace: {3D} visualization of large databases", Proceedings International Symposium on Information Visualization, Atlanta, GA, pp. 82-88. Asimov, D., (1985) "The grand tour: a tool for viewing multidimensional data", SIAM Journal of Science & Statistical Computation, 6, 128-143. Becker, R., Chambers, J. M., and Wilks, A. R., (1988) The New S Language: A Programming Environment for Data Analysis and Statistics. Pacific Grove, CA: Wadsworth. Becker, R. A., Cleveland, W. S., and Shyu, M.-J., (1996) "The visual design and control of trellis display", Journal of Computational and Graphical Statistics, 5(2), 123-155. Bederson, B., (1994) "Pad+ +: advances in multiscale interfaces", Proceedings Human Factors in Computing Systems CHI'94 Conference, Boston, MA, p. 315. Bederson, B. B., and Hollan, J. D., (1994) "PAD+ § a zooming graphical user interface for exploring alternate interface physics", Proceedings User Interfaces Software and Technology'94. New York: ACM, pp. 17-27. Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose, T., (1993) "Toolglass and magic lenses: the see-through interface", Proceedings SIGGRAPH'93, Anaheim, CA, pp. 73-80. Buja, A., Swayne, D. F., and Cook, D., (1996) "Interactive high-dimensional data visualization", Journal of Computational and Graphical Statistics, 5(1), 78-99.
Scope, Techniques and Opportunities for Geovisualization
47
CAIDA (Cooperative Association for Internet Data Analysis), (2003) Visualising internet topology at a macroscopic scale. Online: http://www.caida.org/analysis/topology/as_core_network (23/10/03). Card, S. K., Mackinlay, J. D., and Shneiderman, B., (eds.), (1999) Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann Publishers. Carpendale, M. S. T., Cowperthwaite, D. J., and Fracchia, F. D., (1997) "IEEE computer graphics and applications, special issue on Information Visualization", IEEE Journal Press, 17(4), 42-51, (July 1997). Carr, D. B., Wegman, E. J., and Luo, Q., (1996) ExplorN: design considerations past and present, Technical Report, (No. 129). Centerfor Computational Statistics, George Mason University. Carr, D. B., Olsen, A. R., Pierson, S. M., and Jean-Yves, P. C., (2000a) "Using linked micromap plots to characterize omernik ecoregions", Data Mining and Knowledge Discovery. Hingham, MA: Kluwer Academic. Carr, D. B., Wallin, J. F., and Carr, D. A., (2000b) "Two new templates for epidemiology applications: linked micromap plots and conditioned choropleth maps", Statistics in Medicine. New York: John Wiley & Son. Chert, C., (1999) Information Visualization and Virtual Environments. London: Springer, p. 223. Chernoff, H., (1973) "The use of faces to represent points in k-dimensional space graphically", Journal of The American Statistical Association, 68(342), 361-368. Cleveland, W. S., (1993) Visualizing Data. Summit, NJ: Hobart Press. Cook, D., Symanzik, J., Majure, J. J., and Cressie, N., (1997) "Dynamic graphics in a GIS: more examples using linked software", Computers and Geosciences. Crampton, J. W., (2002) "Interactivity types in geographic visualization", Cartography and Geographic Information Science, 29(2), 85-98. Dent, B. D., (1999) Cartography: Thematic Map Design. Boston: WCB/McGraw-Hill. Di Battista, G., Eades, P., Tamassia, R. and Tollis, I. G., (1999) GraphDrawing: Algorithms for the Visualization of Graphs. Upper Saddle River, NJ: Prentice Hall. Dodge, M., (2001) An Atlas of Cyberspace. Online: http://www.cybergeography.org/atlas Dykes, J. A., (1998) "Cartographic visualization: exploratory spatial data analysis with local indicators of spatial association using Tcl/Tk and cdv", The Statistician, 47(3), 485-497. Edsall, R. M., (1999) "The dynamic parallel coordinate plot: visualizing multivariate geographic data", Proceedings 19th International Cartographic Association Conference, Ottawa, pp. 89-97. Eick, S. G., (1994) "Data visualization sliders", Proceedings ACM UIST, 119-120. Environmental Systems Research Institute, (2003) ArcGIS - ArcView. Online: http://www.esri.com/software/arcgis/arcview/index.html (23/10/03). Feiner, S., and Beshers, C., (1990a) "Visualizing n-dimensional virtual worlds with n-vision", Computer Graphics, 24(2), 37-38.
48
D.A. Keim, C. Panse & M. Sips
Feiner, S., and Beshers, C., (1990b) "Worlds within worlds: metaphors for exploring n-dimensional virtual worlds", Proceedings of UIST'90, pp. 76-83. Fishkin, K., and Stone, M. C., (1995) "Enhanced dynamic queries via movable filters", Proceedings Human Factors in Computing Systems CHI'95 Conference, Denver, CO, pp. 415-420. Furnas, G. W., (1986) "Generalized fisheye views", Proceedings CHI'86 Conference: Human Factors in Computing Systems. New York: ACM Press, pp. 16-23. Furnas, G. W., and Buja, A., (1994) "Prosections views: dimensional inference through sections and projections", Journal of Computational and Graphical Statistics, 3(4), 323-353. Gahegan, M., (1998) "Scatterplots and scenes: visualization techniques for exploratory spatial analysis", Computers, Environment and Urban Systems, 22(1), 43-56. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002a) "GeoVISTA Studio: a geocomputational workbench", Computers, Environment and Urban Systems, 26, 267-292. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002b) "Introducing GeoVISTA Studio: an integrated suite of visualization and computational methods for exploration and knowledge construction in geography", Computers, Environment and Urban Systems, 26(4), 267-292. Geisler, G., (2003) Making information more accessible: A survey of information, visualization applications and techniques. Online: http://www.ils.unc.edu/geisg/info/infovis/paper.html (01/02/03). Goldstein, J., and Roth, S. F., (1994) "Using aggregation and dynamic queries for exploring large data sets", Proceedings Human Factors in Computing Systems CHI'94 Conference, Boston, MA, pp. 23-29. Gusein-Zade, S., and Tikunov, V., (1995) "Map transformations", Geography Review, 9(1), 19-23. Han, J., and Kamber, M., (1986) Data Mining: Concepts and Techniques. Morgan Kaufman. Havre, S., Hetzler, B., Nowell, L., (2000) "ThemeRiver: visualizing theme changes over time", Proceedings of lEEE Symposium on Information Visualization, InfoVis 2000, Salt Lake City, Utah, October 9-10, 2000, pp. 115-123. Hearst, M., (1995) "TileBars: visualization of term distribution information in full text information access", Proceedings of A CM Human Factors in Computing Systems Conference (CHI'95), pp. 59-66. Huber, P. J., (1985) "The annals of statistics", Projection Pursuit, 13(2), 435-474. Ihaka, R., and Gentleman, R., (1996) "R: a language for data analysis and graphics", Journal of Computational and Graphical Statistics, 5(3), 299-314. Inselberg, A., and Dimsdale, B., (1990) "Parallel coordinates: a tool for visualizing multi-dimensional geometry", Proceedings Visualization 90, San Francisco, CA, pp. 361-370. Johannsen, A., and Moorhead, R. J., (1995) "AGP: ocean model flow visualization", IEEE Computer Graphics and Applications, 15(4), 28-33.
Scope, Techniques and Opportunities for Geovisualization
49
Johnson, B., and Shneiderman, B., (1991) "Tree-maps: a space-filling approach to the visualization of hierarchical information structures", Proceedings IEEE Visualization'91, Piscataway, NJ: IEEE, pp. 284-291. Keahey, T. A., (1998) "The generalized detail-in-context problem", Proceedings IEEE Visualization. Keim, D. A., (2000) "Designing pixel-oriented visualization techniques: theory and applications", IEEE Transactions on Visualization and Computer Graphics (TVCG), 6(1), 59-78, (January-March 2000). Keim, D. A., (2001) "Visual exploration of large data sets", Communications of the ACM (CA CM), 44(8), 38-44. Keim, D.A. and Herrmann, A., (1998) "The gridfit algorithm: an efficient and effective approach to visualizing large amounts of spatial data", IEEE Visualization '98. Research Triangle Park, NC: IEEE Press, pp. 181-188. Keim, D. A., and Kriegel, H.-P., (1994) "VisDB: database exploration using multidimensional visualization", Computer Graphics & Applications, 6, 40-49. Keim, D. A., and Kriegel, H.-P., (1995) "Issues in visualizing large databases", Proceedings Conference on Visual Database Systems (VDB'95). Lausanne: Schweiz, pp. 203-214. Keim, D., Bergeron, D., and Pickett, R., (1995a) "Test data sets for evaluating data visualization techniques", Perceptual Issues in Visualization. Springer, pp. 9-21. Keim, D. A., Kriegel, H. P., and Ankerst, M., (1995b) "Recursive pattern: a technique for visualizing very large amounts of data", Proceedings Visualization 95, Atlanta, GA, pp. 279-286. Keim, D. A., Koutsofios, E., and North, S. C., (1999) "Visual exploration of large telecommunication data sets", Proceedings Workshop on User Interfaces In Data Intensive Systems (Invited Talk), Edinburgh, UK, pp. 12-20. Keim, D. A., North, S. C., and Panse, C., (2003a) "CartoDraw: a fast algorithm for generating contiguous cartograms", Transaction on Visualization and Computer Graphics. Keim, D. A., North, S. C., Panse, C., and Schneidewind, J., (2003b) "Visualizing geographic information: VisualPoints vs CartoDraw", Palgrave Macmillan Information Visualization, 2(1), 58-67, (March 2003). Keim, D. A., North, S. C., Panse, C. and Sips, M., (2003c) PixelMaps: a new visual data mining approach for analyzing large spatial data sets. Florham Park: Information Visualization Research Group, AT&T Laboratories, (Dez 2003). Online: http://dbvis.fmi.uni-konstanz.de/sips/papers/03/KNPS03_TECHREPORT.pdf Keim, D. A., Panse, C., and Sips, M., (2003) "Visual data mining of large spatial data sets", 3rd Workshop on Databases in Networked Information Systems, Aizu, Japan. Kocmoud, C. J., and House, D. H., (1998) "Continuous cartogram construction", Proceedings IEEE Visualization, pp. 197-204. Kreuseler, N. L. M., Lopez, N., and Schumann, H., (2001) "A scalable framework for Information Visualization", Proceedings of IEEE Symposium on Information Visualization, InfoVis 2000, Salt Lake City, Utah, October 9-10, 2000. IEEE Press, pp. 27-38.
50
D.A. Keim, C. Panse & M. Sips
Lamping, J., Rao, R., and Pirolli, P., (1995) "A focus + context technique based on hyperbolic geometry for visualizing large hierarchies", Proceedings of CHI'95 Human Factors in Computing Systems, Denver, CO, May 1995. New York: ACM Press, pp. 401-408. LeBlanc, J., Ward, M. O., and Wittels, N., (1990) "Exploring N-dimensional databases", Proceedings Visualization'90, San Francisco, CA, pp. 230-239. Leung, Y., and Apperley, M., (1994) "A review and taxonomy of distortion-oriented presentation techniques", Proceedings Human Factors in Computing Systems CHI'94 Conference, Boston, MA, pp. 126-160. Levkowitz, H., (1991) "Color icons: merging color and texture perception for integrated visualization of multiple parameters", Proceedings Visualization 91, San Diego, CA, pp. 22-25. MacEachren, A.M., (2003) Conditioned choropleth mapping systemproject: conditioned choropleth maps: dynamic multivariate representations of statistical data. Online: http://www.geovista.psu.edu/grants/dg-qg/CCmaps/(June 2003). MacEachren, A. M., Dai, X., Hardisty, F., Guo, D., and Lengerich, G., (2003a) "Exploring high-D spaces with multiform matrices and small multiples", International Symposium on Information Visualization, Seattle, pp. 31-38. MacEachren, A. M., Hardisty, F., Dai, X., and Pickle, L., (2003b) "Geospatial statistics supporting visual analysis of federal geospatial statistics", Digital Government Table of Contents, pp. 59-60. Mackinlay, J. D., Robertson, G. G., and Card, S. K., (1991) "The perspective wall: detail and context smoothly integrated", Proceedings Human Factors in Computing Systems CHI'91 Conference, New Orleans, LA, pp. 173-179. Marshall, R. E., Kempf, J. L., Scott Dyer, D. and Yen, C. C., (1987) "Visualization methods and simulation steering for a 3D turbulence model of Lake Erie", Symposium on Interactive 3D Graphics, Computer Graphics, pp. 89-97. Monmonier, M., (1989) "Geographic brushing: enhancing exploratory analysis of the scatterplot matrix", Geographical Analysis, 21 (1), 81-84. Munzner, T., and Burchard, P., (1995) "Visualizing the structure of the World Wide Web in 3D hyperbolic space", Proceedings VRML'95 Symposium, Special Issue of Computer Graphics, ACM SIGGRAPH, San Diego, CA, pp. 33-38. Perlin, K. and Fox, D., (1993) "Pad: an alternative approach to the computer interface", In: Kajiya, J. T., (ed.), Proceedings of the 20th Annual A CM Conference on Computer Graphics (SIGGRAPH '93), Anaheim, CA: ACM Press, pp. 57-64. (Aug. 2-6). Pickett, R. M., and Grinstein, G. G., (1988) "Iconographic displays for visualizing multidimensional data", Proceedings IEEE Conferene on Systems, Man and Cybernetics, Piscataway, NJ: IEEE Press, pp. 514-519. Raisz, E., (1962) Principles of Cartography. London: McGraw-Hill, pp. 215-221. Rao, R., and Card, S. K., (1994) "The table lens: merging graphical and symbolic representation in an interactive focus + context visualization for tabular information", Proceedings Human Factors in Computing Systems CHI 94 Conference, Boston, MA. New York: ACM, pp. 318-322.
Scope, Techniques and Opportunities for Geovisualization
51
Robertson, G. G., Mackinlay, J. D., and Card, S. K., (1991) "Cone trees: animated {3D} visualizations of hierarchical information", Proceedings Human Factors in Computing Systems CHI 91 Conference, New Orleans, LA, pp. 189-194. Sarkar, M., and Brown, M., (1994) "Graphical fisheye views", Communications of the ACM, 37(12), 73-84. Schaffer, D., Zuo, Z., Bartram, L., Dill J., Dubs, S., Greenberg, S., and Roseman, M., (1993) "Comparing fisheye and full-zoom techniques for navigation of hierarchically clustered networks", Proceedings Graphics Interface (GI'93) Canadian Information Processing Society. Toronto, Ontario: Graphics Press, pp. 87-96. Schumann, H., and Mfiller, W., (2000) Visualisierung: Grundlagen und Allgemeine Methoden. Berlin: Springer Verlag. Shneiderman, B., (1992)"Tree visualization with Treemaps: a {2D} space-filling approach", A CM Transactions on Graphics, 11 (1), 92-99. Shneiderman, B., (1996) "The eyes have it: a task by data type taxonomy for Information Visualizations", Proceedings IEEE Symposium on Visual Languages. Washington: IEEE Computer Society Press, pp. 336-343. Silicon Graphics Inc., (2002). SGI - Products: Software: MineSet Update. Online: http://www.sgi.com/software/mineset.html (01/02/02). Spence, R., (2001) Information Visualization. Harlow: Addison-Wesley/ACM Press Books, 206 pp. Spence, R., and Apperley, M., (1982) "Data base navigation: an office environment for the professional", Behaviour and Information Technology, 1(1), 43-54. Spence, R., Tweedie, L., Dawkes, H., and Su, H., (1995) "Visualization for functional design", Proceedings International Symposium on Information Visualization (InfoVis'95), pp. 4-10. Spoerri, A., (1993) "InfoCrystal: a visual tool for information retrieval", Proceedings Visualization'93, San Jose, CA, pp. 150-157. Stasko, J. T., Domingue, J. B., Brown, M. H., and Price, B. A., (eds.), (1998) Software Visualization. Cambridge, MA: MIT Press. Stolte, C., Tang, D., and Hanrahan, P., (2001) "Polaris: a system for query, analysis and visualization of multi-dimensional relational databases", Transactions on Visualization and Computer Graphics. Swayne, D. F., Cook, D., and Buja, A., (1992) "XGobi: interactive dynamic data visualization in the X Window System", Journal of Computational and Graphical Statistics, 7 (1), 113-130. Tierney, L., (1991) LispStat: An Object-Orientated Environment for Statistical Computing and Dynamic Graphics. New York: Wiley. Tobler, W. R., (1976) "Cartograms and cartosplines", Proceedings of the 1976 Workshop on Automated Cartography and Epidemiology, Washington, District of Columbia, pp. 53-58. van Wijk, J. J., and van Liere, R. D., (1993) "Hyperslice", Visualization'93, San Jose, CA, pp. 119-125.
52
D.A. Keim, C. Panse & M. Sips
Velleman, P. F., (1997) Learning Data Analysis with DATA DESK 5.0. New York: Addison-Wesley. Ward, M., (1994) "XmdvTool: integrating multiple methods for visualizing multivariate data", Proceedings Visualization'94. Washington: IEEE Computer Society Press, pp. 326-333. Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, p. 384. Wilhelm, A., Unwin, A. R., and Theus, M., (1995) "Software for interactive statistical graphics - a review", Proceedings Int. Softstat 95 Conference Heidelberg, Germany.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 3
Information Visualization and the Challenge of Universal Usability Catherine Plaisant, Human-Computer Interaction Laboratory, University of Maryland, HCIL/UMIACS A.V. Williams Building, College Park, MD 20782, USA
Keywords: Information Visualization, user interface, universal usability, blind users, help
Abstract Information Visualization aims to provide compact graphical presentations and user interfaces for interactively manipulating large numbers of items. We present a simple "data by tasks taxonomy" then discuss the challenges of providing universal usability, with example applications using geo-referenced data. Information Visualization has been shown to be a powerful visual thinking or decision tool but it is becoming important for services to reach and empower every citizen. Technological advances are needed to deal with user diversity (age, language, disabilities, etc.) but also with the variety of technology used (screen size, network speed, etc.) and the gaps in user's knowledge (general knowledge, knowledge of the application domain, of the interface syntax or semantic). We present examples that illustrate how those challenges can be addressed.
3.1
Introduction "If a picture is worth a thousand words, then an interface is worth a thousand pictures." (Shneiderman, 2001)
Designers are discovering how to use rapid and high-resolution color displays to present and manipulate large amounts of information in compact and user-controlled ways. Information Visualization can be defined as the use of computer-supported interactive visual representation of abstract data to amplify cognition (Card et al., 1999). The abstract characteristic of the data is what distinguishes Information Visualization from scientific visualization. Information Visualization is more likely to be used to display database content (for example, recorded stock values, health statistics) than output of models or simulations, but this distinction is not always important. The display of geo-referenced data is often a hybrid visualization that combines abstract and concrete data. In fact several of the most famous examples of Information Visualization 53
54
C. Plaisant
include maps, from the 1861 representation of the ill-fated Napoleon' s Russian campaign by Minard (see Tufte (1983) and Kraak (undated)) to the interactive HomeFinder application shown in Figure 3.1 that introduced the concept of dynamic queries (Ahlberg et al., 1992). Information Visualization aims to provide compact graphical presentations and user interfaces for interactively manipulating large numbers of items (102-106), possibly extracted from far larger datasets (Card et al., 1999; Spence, 2001; Ware, 2000; Chen, 2002; Bederson and Shneiderman, 2003). Also sometimes called visual datamining, it uses the enormous visual bandwidth and the remarkable human visual system to enable users to make discoveries, take decisions, or propose explanations about patterns, groups of items, or individual items. Perceptual psychologists, statisticians, and graphic designers (Tufte, 1983) offer valuable advice about presenting static information, but advances in processor speed, graphic devices and dynamic displays takes user-interface designers well beyond current wisdom.
Figure 3.1. The HomeFinder. Houses for sale appear as yellow dots on a stylized map of Washington DC. As users adjust sliders and click on buttons to set their search criteria immediate feedback (within 10 ms) is provided on the map showing the houses that match the query, and allowing users to pose hundreds of queries in a few seconds and rapidly explore the dataset.
Information Visualization and the Challenge of Universal Usability
55
This chapter presents a simple "data by tasks taxonomy" then discusses the challenges of providing "universal usability", with example applications using georeferenced data.
3.2
Data by Task Taxonomy
This data by tasks taxonomy includes seven basic data types and seven basic tasks (Shneiderman, 1998). The data being explored are usually multi-dimensional but most designs highlight 1, 2 or 3D that are used to define the general visual structure of the visualization. The taxonomy is organized based on those selected dimensions. This simplification is useful to describe the visualizations that have been developed and to characterize the types of problems that users encounter when using those visualizations. Examples were chosen to complement those provided by Keim et al., this volume (Chapter 2) in which a more comprehensive taxonomy for visualization of highdimensional data is the primary focus.
3.2.1
Data types
1- 2- or 3D data: Linear data types (1D) include lists, documents, program source code, and the like that are organized sequentially. Interface-design issues include what overview, scrolling, or selection methods can be used. User tasks might be to find the number of items or to see items that exhibit certain attributes (Eick et al., 1992). Planar data (2D) can be represented by geographic maps, floor plans, and newspaper layouts. User tasks are to find adjacent items, containing items and paths between items, and to perform the seven basic tasks (see below). Real-world 3D objects such as the human body or buildings have volumetric elements and connections with other elements. Users' tasks deal with adjacency plus above-below and inside-outside relationships. In 3D applications, users must understand and control their position and orientation when viewing the objects, and must be able to compensate for the serious problems of occlusion. While many examples of successful 3D computer graphics and scientific visualizations exist (Nielson et al., 1997) there are still very few Information Visualization examples in three (abstract) dimensions. Designers have used 3D representations to present attractive overviews of data that was not intrinsically 3D, such as sets of documents, as shown in Figure 3.2 (Wise et al., 1995). But controlled experiments trying to measure the benefits of 3D in such conditions have had mixed results (Cockburn and McKenzie, 2002). These mixed results complement similar findings from earlier cartographic experiments and related efforts to assess use of the third dimension to depict non-spatial attributes on maps (Kraak, 1989; Dorling, 1992) Temporal data: Time series are very common and merit a data type that is separate from 1D data. The distinctions are that items have a start and finish time, and may overlap. Timelines have been widely used, from the line plots used by Minard (see Tufte, 1983) to summaries of heterogeneous data such as LifeLines, as shown in Figure 3.3 (Plaisant et al., 1996). Frequent tasks include finding all events before, after, or during some time period or moment, and in some cases comparing periodical phenomena (Carlis and Konstan, 1998). Space-time data have also been a focus
56
C. Plaisant
Figure 3.2. ThemesViewTM (formally ThemeScape) shows a 3D map representing the results of a search in a large corpus of documents. Proximity indicates similarity of the topics, whilst height reflects the number of documents and frequency of terms. Commercial applications also exist (OmniViz Inc, 2003) Reproduced with permission of Pacific Northwest National Laboratory.
Figure 3.3. LifeLines present a summary of personal records (here a medical record), showing several facets of the records and using line thickness and color to map data attributes on the display.
Information Visualization and the Challenge of Universal Usability
57
of attention in geovisualization for more than a decade (Szego, 1987; DiBiase et al., 1992; Kraak and MacEachren, 1994; Kwan, 2000) and some recent advances are discussed by Andrienko et al., this volume (Chapter 10). Multi-dimensional data: Most relational- and statistical-database contents are conveniently manipulated as multi-dimensional data, in which items with n attributes become points in an n-dimensional space. The interface representation can be dynamic 2D scattergrams (possibly a map, as is the case in Figure 3.1) with each additional dimension controlled by a slider or button using dynamic queries (Ahlberg et al., 1992). A 3D scattergram is possible, but disorientation and occlusion are severe problems. Parallel coordinates plots (Inselberg, 1985) are one of the few truly multi-dimensional techniques and have been shown to be a powerful analysis tool. Familiarity, training and practice in using the technique will help a user become a "multi-dimensional detective" (Inselberg, 1997). Less powerful but more accessible to novice users is the Table Lens (Rao and Card, 1994; Inxight Software Inc., 2002), which uses a spreadsheet metaphor. Other examples include VisDB for multi-dimensional database visualization (Keim and Kriegel, 1994), interactive mosaic displays (Friendly, 1994; Theus, 2002a,b), the Attribute Explorer (Tweedie et al., 1996) and the scatter plot or prosection matrices of Becker and Cleveland (1987). Interactive geovisualization software also utilizes multidimensional visualization techniques (Andrienko and Andrienko, 1999a-f; Gahegan et al., 2002a,b; MacEachren et al., 2003a,b) as emphasized in section B of this volume (see Andrienko et al., this volume (Chapter 5) and subsequent chapters). Hierarchical data: Hierarchies or tree structures are collections of items, in which each item (except the root) has a link to one parent item. Examples include taxonomies, file structures, organization charts and disease classifications. Items and the links between parent and child can have multiple attributes. Tasks can be topological (for example, a query asking which branch of the company has more employees?) or attribute based (such as an attempt to find the largest old files on a hard disk). Interface representations of trees can use the outline style of indented labels used in tables of contents, node-and-link diagrams. Examples include the Hyperbolic tree (Lamping et al., 1995) commercialized by Inxight Software Inc (2002) or SpaceTree (Plaisant et al., 2002; Grosjean et al., 2002). A third possibility is to use a space filling representation such as Treemap (Johnson and Shneiderman, 1991; Bederson et al., 2002) as shown in Figure 3.4. A number of commercial applications of the Treemi~p technique are available including Shneiderman (1998), the Map of the Market (SmartMoney.com, 2003) or e-business (The Hive Group Inc., 2003). Network data: When relationships among items cannot be captured conveniently with a regular tree structure, items are linked to an arbitrary number of other items in a network. In addition to performing the basic tasks applied to items and links, network users often want to know about shortest or least costly paths connecting two items or traversing the entire network. Common representations include node-andlink diagrams (but layout algorithms are often so complex that user interaction remains limited for large networks), and square matrices of items with the value of a link attribute in the row and column representing a link. Network visualization is an old but still imperfect art because of the complexity of relationships and user tasks, for example,
58
C. Plaisant
Figure 3.4. An example of Treemap is the Map of the Market (SmartMoney.com, 2003). Rectangles each represent a stock and are organized by industry groups. The rectangle size is proportional to the market capitalization and the color indicates the percentage gain or loss for the given time period. Reproduced with permission of SmartMoney.com.
Figure 3.5. Dynamic labeling of items is still a challenge. Here Excentric Labels in incMap show the labels of all the items inside the focus circle, revealing hidden items and allowing their selection. Reproduced with permission of N Space Labs, Inc. (http://www.incmap.com).
Information Visualization and the Challenge of Universal Usability
59
(see Rodgers, this volume (Chapter 7)). It is used in a number of useful geographic applications and is being incorporated into software for geovisualization, for examples, (see Mountain, this volume (Chapter 9)) and Fairbairn, this volume (Chapter 26).
3.2.2
High-level task types
Having considered the range of data types available along with some methods that have been developed for graphically representing them, we can consider a number of highlevel tasks that apply to all data types Overview task: Gaining an overview of the data might include gauging the number of items and the range and distribution of the attribute values, or estimating how much things have changed since last time the user reviewed the data. Overview strategies include zoomed-out views adjoining the detail views, for example, (see Ware and Plumlee, this volume (Chapter 29)). A movable field-of-view box can be used to control the contents of the detail view. Intermediate views allow larger zoom factors. Another popular approach is the fisheye strategy originally described by Furnas (1986). It provides overview and details in a single combined view by using distortion based on a degree of interest function. It is effective when zoom factors are small and deformation is acceptable to users (for example, when orientation and distance measurement are not important). Zoom task: Users need to control the zoom focus and the zoom factor. Smooth zooming helps users to preserve their sense of position and context. Manual zooming is powerful but users tend to get lost so application-controlled zooming to preset levels is more likely to be usable, for example, see Ware and Plumlee, this volume (Chapter 29). Pad-+-+, now called Picolo is a popular zooming user interface toolkit that uses semantic zooming (Bederson, 1994; Bederson et al., 2000). Semantic zooming (Perlin and Fox, 1993) is commonly used with maps, where the same area can be displayed with different features and amount of details at different zoom ratios. There are key parallels with real time cartographic generalization here (Weibel and Jones, 1998). Constant density zooming (Woodruff et al., 1998) is an example of technique to maximize the number and readability of items on the display. Wood, this volume (Chapter 15) uses mipmapping to display surface characteristics according to the scale at which any part of a surface is viewed in a real-time 3D application and D611ner, this volume (Chapter 16) identifies some associated issues in computer graphics. Filter task: Dynamic queries allow users to quickly focus on their interests by eliminating unwanted items. Other techniques include sorting, grouping or highlighting followed by hiding, or locating items similar to an item of interest. Theus, this volume (Chapter 6) provides some examples of "selection" from the perspective of statistical graphics. Details-on-demand task: Once a collection has been trimmed, users need to review the details of single items or groups. The usual approach is to simply click on an item and review details in a separate or popup window. Dynamic labeling remains an important challenge. Excentric labeling (Fekete and Plaisant, 1999) is an approach
60
C. Plaisant
illustrated in Figure 3.5 through the Geographic Map of the Market in which geovisualization techniques and those of Information Visualization are integrated (N Space Labs Inc., 2003). Relate task: Users need to view relationships between items, as is the case with the LifeLines shown in Figure 3.3, where users can click on a medication and see the related visit report, prescriptions, and laboratory test results. Linking and brushing techniques (Cleveland, 1994) and the Influence Explorer (Tweedie et al., 1996) emphasize the exploration of relationships. Many applications combine multiple visualization techniques that are tightly coupled (for example, (see Roberts, this volume (Chapter 8))). This is a key strength of visualization tools, and a number of chapters in this publication demonstrate examples, particularly those concerned with "Creating Instruments for Ideation" (for example, (see Andrienko et al., this volume (Chapter 5))). Various tools are being developed in various academic and application domains to allow users to specify the visualizations they need and how the interaction between the visualizations should be controlled (North et al., 2002). History task: It is rare that a single user action produces the desired outcome. Keeping the history of actions allows users to retrace their steps, save useful exploration "recipes" and apply them to updated datasets later on. Roberts, this volume (Chapter 8) considers these issues at an operational level and Gahegan, this volume (Chapter 4) addresses the conceptual, scientific and motivational challenges that underlie support for saving and sharing entire analysis strategies. Extract and Report task: Users often need to save subsets of the data or particular views of the data into reports. They may also want to publish "cleansed" data with a simplified subset of the tool's features for others to review.
3.3
The Coming of Age of Information Visualization - Toward Universal Usability
Although maps have been used for hundreds of years, it is important to realize that most Information Visualization techniques are very recent and, if we exclude elementary tools such as pie or bar charts, these tools have mostly been used by researchers, data analysts and experts. Only recently have we seen novel designs become widely used by the public. Researchers recognize that there is no "one size fits all" visualization and that specialized techniques are needed to tackle special data (for example, genomic data or spatio-temporal data) as well as special user needs (such as collaborative analysis or real time monitoring). Two general challenges need to be faced by Information Visualization before we can anticipate more widespread uptake of many of the techniques that have been developed: the ability to handle larger volumes of data and the ability to effectively support more diverse users. The first challenge relates to the limited number of items most tools available can manage. Many innovative prototypes can only deal with a hundred or a thousand items, or have difficulties maintaining real time interactivity when dealing with larger numbers. Yet examples dealing with millions of items without aggregation have been achieved as shown in Figure 3.6 (Fekete and Plaisant, 2002). Keim
P, i
a -
,T-
-.)+-----
. : I . --..
Figure3.6. A Treemap displaying a million files from a large file system, without aggregation (Fekete and Plaisant, 2002). Careful examination of the high resolution display reveals patterns and special algorithms preserve the interactivity of the rich overviews and filtering tools.
R
R
62
C. Plaisant
et al. (2001) demonstrate that Information Visualization is not yet close to reaching the limits of human visual abilities but that new techniques are needed to tackle the large quantities of information. Keim et al., this volume (Chapter 2) provide an overview and some additional solutions. Dealing with real data also involves data cleansing or dealing with missing and uncertain values. The second challenge is focused on more specifically here: how do we make Information Visualization accessible to a wider group of diverse users? In fact a growing number of applications are being developed for what might be the most challenging user group - the general public. Let us take the example of the enormous amounts of georeferenced data accumulated by government agencies and to be made available to every citizen. These data are useful to senior citizens looking for a place to settle after retirement, business analysts, managers considering relocation, etc. They can also be useful to drivers on the road and researchers working on complex tasks on their high-end computer workstations. To address the problem of universal usability (Shneiderman, 2000; Shneiderman and Hochheiser, 2001), designers need to consider larger and more diverse groups of users. Technological advances are needed not only to deal with user diversity (age, language, disabilities, etc.) but also with the variety of technology used (screen size, network speed, platform, etc.) and any specific gaps in a user' s knowledge (e.g., general knowledge, knowledge of the application domain, of the interface syntax or semantic). Information Visualization has been shown to be a powerful visual thinking or decision tool for some average users receiving a minimum of training (Williamson and Shneiderman, 1992; Lindwarm et al., 1998; Chen and Czerwinski, 2000). But it is becoming important for services to reach and empower every citizen regardless of their background, technical disadvantages or personal disabilities. Universal usability is an ambitious long-term goal but many practical steps can be taken toward achieving it. Focusing on this problem will most likely lead to better technologies for everyone (simpler interfaces, faster downloads, etc). Several examples are reviewed below that illustrate how projects have addressed different aspects of the universal usability challenge.
3.3.1
Improving general usability
Of course the first step is to improve the general usability of the interface. Many texts are available to guide designers toward the creation of empowering user interfaces (Shneiderman, 1997; Preece et al., 2002). Usability testing can then verify that user needs and usability goals have been met and that users are able to use the system on their own with a sense of control and satisfaction (Dumas and Redish, 1999; Kuniavsky, 2003). Specific guidelines for map design are also available (Pickle, 2003) and Tob6n, this volume (Chapter 34) draws upon the usability approach to assess geovisualization map use, for example, (see also Fuhrmann et al., this volume (Chapter 28)). The success of the Map of the Market (Figure 3.4) illustrates that general techniques initially described as "difficult to understand" (i.e., Treemap) can become accessible interfaces. The popularity of this application with the general public might be attributed in part to the fact that the data is familiar, and important, to users who visit
Information Visualization and the Challenge of Universal Usability
63
a financial website. The careful simplification and refinement of the initial Treemap interface also plays a part. The names and groupings of stocks are familiar to users of the SmartMoney Website. The hierarchy is simple, being shallow and of fixed depth, and the color mapping is natural to most users (using green for gains or increased values of the stocks, red for losses, with alternative colors for those with color impairment). An elegant new algorithm has recently resulted in a better aspect ratio for all the rectangles making the display more readable and pleasing. Labeling and links to details were optimized for the particular application. Finally it was carefully written in lightweight Java so that it could actually run on most users' machines without having to download special plugins or worry about Java versioning. This combination of factors has been effective in making the map usable by the general public, as indicated by the large number of return users the Website enjoys.
3.3.2
Helping users get started
Designers need to consider whether their application is to be used by first-time users or by experts who will have invested time and effort into learning the application. A major aspect of accessibility is concerned with enabling users to get started with an application (Andrienko et al., 2002). Even the average professional analyst or researcher may need help, but a major challenge is to help ordinary citizens use Information Visualization applications successfully. Commercial Web sites can decide which population they target but Digital government applications might well be faced by the greatest challenge since they are supposed to be usable by everyone! This implies that most information consumers will be first time users and also that they will have very varied backgrounds and levels of education. They may also have limited time or interest in learning to use the system, but paradoxically may have very high expectations of the services they seek to use. Users want an answer to their question, not necessarily to learn all that a tool can do for them. Let us consider a particular example, that of DataMap (Plaisant, 1993), which was previously named Dynamap or Ymap. Dang et al. (2001) review three approaches that were explored to help users get started with the DataMap interface. DataMap (Figure 3.7) is an interactive visualization tool developed by our Human-Computer Interaction Laboratory at the University of Maryland and is scheduled to be released by the US Census Bureau on CDs and later on the Web. Users can click on a map of the US to display facts about the areas, select multiple areas to compare, zoom on the map or use dynamic queries to filter the map according to a list of criteria. Finally, they can use a scatter plot, tightly coupled to the map and table, to see relationships between two criteria. Our numerous demonstrations and tests in the lab had made us confident that average users could understand and use DataMap after a minute of two of demonstration, but usability tests at the Census Bureau with novice "off-the-street" users receiving no training revealed that many users had difficulties getting started. Some would struggle with certain details of the interface, while others would simply dismiss the interface saying, "It' s just too complicated". They had problems not only zooming and selecting multiple regions but also with the sliders. The problems ranged from recognizing that one could move the slider thumbs to understanding what the animation meant. Several users were even puzzled
64
C. Plaisant
Figure 3.7. (a) DataMap allows users to zoom on a state or county map, select areas or filter them out with dynamic queries. A tightly coupled scatter plot provides an alternative to compare areas. (b) The revised interface (reproduced with permission of Chris North, Virginia Tech).
Information Visualization and the Challenge of Universal Usability
65
by the interactive scatter plot, some not guessing that dots represented regions and some not knowing how to read a scatter plot at all. The first step was to revise the interface. This was done by a team of colleagues at Virginia Tech led by Chris North and is illustrated in Figure 3.7b. Most problems found in the usability study led to refinements of the interface. This resulted in a number of changes including: reducing the number of attribute sliders with the possibility of adding more with a control panel; making slider thumbs easier to recognize and use; limiting zooming to avoid zooming into empty areas; simplifying window resizing; using an appropriate projection for the map; using fonts consistently and selecting more readable typefaces; using color coding to show the relationship between the sliders position and the parts of the map highlighted or hidden. Some aspects of the interfaces still needed explanations. Since DataMap is meant for public access, it was clear that users would be unlikely to read manuals or go through long tutorials, so we explored alternative approaches to providing help (Plaisant et al., 2003.) We considered three approaches. The first approach is to create a multi-layered design that provides a simpler interface to get started, while complex features are accessed progressively as users move through the layers of the interface. The second approach is called Integrated Initial Guidance (IIG) or "sticky note help". It provides help within the working interface, right at the start of the application. Using the metaphor of sticky notes overlaid on top of the functional interface it locates the main widgets, demonstrates their manipulation, and explains the resulting actions by replaying recorded sets of actions. The third approach uses video demonstrations of the interface. The videos use sound and entirely overlap the interface to give the illusion of being integrated in the interface. 1
Approach 1" multi-layered designs Some applications might be suited for a multi-layer design that allows users to start with a simple interface with limited views on information and choices to make and then to move to more complex (and more functional) levels once the basic functions are understood. This is the case for DataMap, which can easily be organized in a three-layer design (Figure 3.8): 9 9 9
layer 1: map and table only; layer 2: map and table, plus dynamic query filters; layer 3: map and table and dynamic query, plus scatter plot.
The interface initial view hides level 2 and level 3 functionality. When users go up one level, the appropriate parts of the interface are added with an animated transition, which help users see what is changing. Multi-layered interfaces, termed "level-structured interfaces" when introduced to Information Visualization (Shneiderman, 1998), are commonly encountered in hardware devices like VCRs that hide advanced function buttons and controls behind a small door, or in search interfaces that often provide a Note that this workon Help was done in parallel with Chris North's workon the revisionof the interface, so the help techniques are shown on the old interface. Those two projects will be merged eventually.
66
C. Plaisant
Figure 3.8. Three different levels of the level structured approach. (a) the first level shows only the map and the data table; (b) second level shows slider bar along with map and table; (c) the third level adds scatter plot. An introduction panel lists the features of the level and indicates the location of the three level buttons.
Information Visualization and the Challenge of Universal Usability
67
simple and advanced search. This approach has been underused in software applications and could be generalized easily. The benefits are that users can get started more easily. The risk is that some users will never discover the advanced features they do not know are there.
Approach 2: integrated initial guidance or "sticky note help" IIG implements a metaphor of sticky notes inside the interface itself, thereby allowing users to use the interface or run automated demonstrations while reading the sticky notes overlaid on the functional interface (Figure 3.9). Sticky notes highlight the main functions of the interface, show the location of the main interface widgets whose use can be demonstrated and explained with a "show me" button, and provide lists of simple to complex example tasks which lead to demonstrations of advanced interface widget functions. This allows users to make use of the help in many different ways. Some users will try out all the example tasks, while others may never use any. Some users will
Figure 3.9. Using sticky notes. (a) Three IIG sticky notes lead to explanations of how to use the map, zoom, select multiple states and move to the next level. Example tasks are listed in the note at the top right. (b) The first example task was chosen. "Compare the populations of three west coast states". The next steps are shown in transparency until the user executes the current one, or asks to see an automated "show me" demonstration.
68
C. Plaisant
use the show me feature while others will execute the steps themselves, guided by the directions on the sticky notes and using the show me function only when they failed to guess what to do.
Approach 3: autonomous video or animated demonstrations Video demonstrations are used increasingly to introduce the main features of complex visual interfaces. They can be launched from the application but are really run outside the interface itself. Macromedia Flash can be used to create demonstrations but simple recording tools (such as Camtasia Studio from TechSmith) are useful to create video explanations in a matter of hours. Effectiveness is increased by the use of speech, which leads to more compact explanations and lively demonstrations (but subtitles will be needed to accommodate users with hearing difficulties). Highlighting the mouse cursor and making mouse clicks or key presses visible is important. Low-resolution videos can be effective and fast to download but we found that videos that entirely cover the interface give users the impression that they are watching the real interface, therefore reducing the transfer of learning to the interface. Of these three different approaches, the sticky note technique was found very effective during informal user testing but it was fairly difficult to design and build. On the other hand, the video demonstrations were easier to create and fairly effective when kept short and mapped in direct ways to users tasks. The multi-layered approach was appreciated by the majority of our test users and might be the most powerful approach to helping users get started with complex public access applications, as long as care is given to guide users to the advanced levels.
3.3.3
Addressing the hardware diversity
Because programmers and development teams usually work with high-end equipment and fast network connections it is important to address the range of devices and network speed available in people's homes and businesses when producing visualization software for public access. Issues may vary from application to application so we only give an example illustrating how addressing hardware problems does not necessarily result in a "lowest common denominator" application but can benefit everyone. Let us consider the example of DataMap once again. The Java version of DataMap loaded and ran quickly enough on standard PCs and broadband connections but - as with other interactive map tools - would take minutes to download with a 56K modem and could not maintain the interactivity needed for dynamic queries (i.e., 10 ms feedback) on low-end PCs. Traditionally, interactive maps use vector graphics to draw the maps which means downloading the vector data and the software to interpret it. Instead, we created a new version of DataMap that encoded geographic knowledge into raster images quickly delivered to the client (Zhao et al., 2003). Algorithms were devised to perform sub-second dynamic query, panning and zooming, with no or minimum server support. This technique leads to download times that were often counted in seconds instead of minutes for modem users, being between 5 and 10 times shorter that those experienced with Web GIS approaches that are based on vector geographic data, while
Information Visualization and the Challenge of Universal Usability
69
keeping the needed interactivity. The client-side was also implemented as light-weight Java applets to avoid version problems. This example illustrates that it is possible to find technological advances that benefit general public users with slow modem network connections and low-end machines, as well as users with fast T-1 connections and fast machines, therefore advancing the goals of Universal Usability.
3.3.4
Addressing the needs of users with visual impairments
Color blindness Color impairment is a very common condition that should not be overlooked (Rosenthal and Phillips, 1997; Olson and Brewer, 1997) This problem can more easily be addressed by limiting the use of color, using double encoding when appropriate (for example, by using symbols that vary in both shape and color), providing alternative color palettes to choose from, or allowing users to customize the colors themselves. For example the Mapof-the-Market illustrated in Figure 3.4 provides two choices of color schemes: red-green and blue-yellow. Various tools are available to both simulate color vision impairments and to optimize graphics for some of the various forms of color impairment that exist, including Vischeck (Dougherty and Wade, 2002). ColorBrewer (Brewer and Harrower, 2002) offers guidelines on color schemes that work for those with color vision impairment.
Low Vision and blindness Approximately four million people in the US are blind or visually impaired and it is a requirement for government-provided services to address this population (Vanderheiden, 2000; Muntz et al., 2003). Traditional accommodations to blind and vision-impaired users include the use of speech synthesizers as screen reader, and/or Braille to convey the text information on the display. However, speech-based approaches are very weak at representing 2D spatial layout in the graphical user interface. This is a challenge for maps as well as most other Information Visualization techniques. For navigation in the real world, GPS-based talking maps have been developed (Golledge et al., 1991a,b). One example product is a talking nationwide digital map consisting of most addresses and street intersections (The Sendero Group, 2003). Users can navigate the map using the arrow keys and listen to speech-synthesized descriptions of the map and directions. Trekker (VisuAide, 2003) is another GPS-based application that helps the blind to navigate. Maps are also important to learn an area beforehand and chose a route. Schneider and Strothotte (2000) have designed a tangible interface using physical building blocks that users manipulate to promote constructive exploration of the map. When using maps to learn Geography or discover spatial patterns in data, people with limited vision can use screen magnifiers (for instance, try the "magnifier" available from the Start menu of MS Windows, in the accessibility accessories), but those with severe vision impairments have to rely on tactile maps or atlases (Imperatore, 1992) as shown in Figure 3.10 or sonification of the maps.
70
C. Plaisant
Figure 3.10. A traditional tactile atlas showing a map of Canada. Reproduced with permission of Natural Resources Canada and the Canadian Government.
Current assistive technology research is exploring a number of promising techniques to help blind users benefit from the spatial awareness provided by maps. The first step is to provide data in both visual and descriptive form. OptiMaps (Corda Technologies Inc., 2003), employs a small "d" character below each choropleth map. This is the standard accessibility mark for image "descriptions" that visually impaired users are able to search for. These provide hyperlinks to a textual version of the data, generated automatically with the dynamic map. Examples are available in AtlasPlus (National Cancer Institute, 2003). New vector graphic file formats, such as SVG, permit embedding text descriptions with the graphic information and should simplify this process as a result. Unfortunately reading the data values still does not give an adequate feeling for the spatial relationships between areas that only spatial techniques can provide. Tactile and haptic techniques (and strategies for signifying data haptically) have been devised (Griffin, 2002). Virtual Touch provides an example of map exploration using a tactile mouse. The mouse is equipped with two Braille character mechanical units, which can be used to indicate tactile patterns (using Braille and other alternatives), as shown in Figure 3.11. As users explore the map, the patterns change over different
Information Visualization and the Challenge of Universal Usability
71
Figure 3.11. The VTPlayer is a mouse equipped with two Braille character units that can also be used to indicate patterns. For example, as users explore a map of the US, patterns change over different states (VirTouch Ltd, 2002). Reproduced with permission of VirTouch Ltd.
states or areas. One of the limitations of the mouse is that it is an input device providing only relative positioning so absolute position on the map has to be provided by audio feedback. Another example developed by TouchGraphics and their research colleagues (Landau and Gourgey, 2001) augments standard printed tactile maps. Maps are secured on top of a touch screen that provides the location of each touch (Figure 3.12). The TouchGraphics Atlas has five operational modes. Users can simply explore by touching a physical tactile map and hear names of places touched. They select a destination from an index, and listen to directions that are updated as a user's finger gets closer to the destination. Distance between two points can be calculated, and descriptions of the areas can be listened to. This work was inspired by Nomad (Parkes, 1994). 3D maps can benefit from the use of haptic devices. For example, the Phantom device provides 6 degrees of freedom input and 3 degrees of freedom output to explore the virtual sound space and augment it with haptic feedback providing the same sensation as moving a single finger over a physical 3D map. Tactile maps can be augmented with abstract audio output (Fisher, 1994; Krygier, 1994). For example, users can hear a series of graduated pitches proportional to elevations above sea level as they explore the map. Sonification alone has been demonstrated to be effective in giving blind users access to graphs (Ramloll et al., 2001) or, in some limited fashion, entire desktops (Mynatt and Edwards, 1992). In the case of maps, Jeong and Gluck (2002) compared the effectiveness of and preferences for using auditory feedback (volume of sound), haptic feedback (extent of vibration) or both in the tasks of identifying the highest or the middle valued state on a static choropleth map of the US. The experiment showed that overall performance is most successful when using haptic feedback alone but users prefer having both haptic and audio feedback. This result may be attributed to the fact that haptic devices provided spatial cues while the standard sound output does not. Unfortunately tactile and haptic techniques require special hardware, from embossed maps to specialized input devices, and hardware is usually
72
C. Plaisant
Figure 3.12. An example of tactile map augmented with audio description when mounted on a touch screen (Touch Graphics, undated). (a) Sample map plate; (b) the sample map plate in use with the talking tactile tablet (TTT). Reproduced with permission of Touch Graphics.
a limiting factor for wide dissemination, as users may not have access to the technology or the financial means to purchase it. Techniques relying solely on sonification have the potential to be used by a larger population as blind users, who often already rely on headphones when using screen readers.
Information Visualization and the Challenge of Universal Usability
73
The BATS project at the University of North Carolina (Parente and Bishop, 2003) uses simple spatial audio (mostly stereo effects) to sonify maps as shown in Figure 3.13. Real world auditory icons are played as the user moves the cursor over the map. For example, car noises are heard over cities and birds and crashing waves are heard over forests and beaches. The sound becomes louder as the user gets close to the source of the sound. Interactivity and immediate feedback provides a pleasant exploration experience and helps blind users explore the map and locate objects. A promising direction for the sonification of spatial information is the use of spatial audio using a head-related transfer function (HRTF) (Shinn-Cunningham et al., 1997). Since humans are usually able to localize sound with amazing precision by using binaural perception, spatial location can be an important aspect of information perception. It is possible that the addition of positional cues will greatly improve the sonification of maps. Stereo effects allow right-left sound separation only, while spatial audio provides up and down cues, and in a less reliable manner front and back information. CAVE-like virtual environments provide such spatial sound with sets of multiple loud-speakers (Pape, 1998; see Bodum, this volume (Chapter 19). But synthesized spatial audio allows users to experience 3D audio with ordinary headsets. Until recently this had been limited to high-performance computing environments but researchers are now able
North Carolina
C ~ ' e m merit ~ n d 1
Ci~' Sprawl (~her S~tes
....................~ u : n b , ~ e r ---
State Border
Figure 3.13. The BATS sonic map helps those without sight to explore spatial information (Parente and Bishop, 2003). Real world audio icons (car noises, wave crashing, etc.) are played as the user navigates the map and appear clearly as coming from your right or your left. Volume increases as users get closer to the source of the sound. Reproduced with permission of the authors.
74
C. Plaisant
Figure 3.14. Spatial audio allows users to hear the location of virtual sound sources. This schematic represents what a user might hear as each US state "plays" its statistics, for example a sound whose pitch is proportional to its African-American population. Proper linearization of sounds allow users to hear patterns of low to high values moving from the northern states to the lower Mississippi area.
to synthesize high-quality 3D sound in real-time on commercial off-the-shelf PCs (Zotkin et al., 2002). We are currently exploring its use for the sonification of maps. Nevertheless, the spatial effects - if noticeable - remain fairly weak and our research at the University of Maryland seeks to combine spatial effects with standard sonification techniques such as those using pitch or tone variations to reinforce the perception of the spatial information. Imagine being in the center of a virtual room with a large US state map hanging from the ceiling and wrapping around you as shown in Figure 3.14. Now imagine that each state on the map "plays" a sound indicating the value of a particular statistics for each state. Because spatial audio is helpful to separate different sounds played together, users can make sense of the location of the states with high, medium and low values. Sonification might also be useful for non-disabled users for monitoring tasks or to represent areas too small to be shown on the map. For example, The District of Columbia is often too small to be visible on US state maps but will be heard in an audio map.
3.4
Conclusions
Information Visualization allows designers to present a large amount of information using abstract representations. Geographic and scientific visualization applications
Information Visualization and the Challenge of Universal Usability
75
usually use representations determined by the nature of the data being displayed. Location within the graphic is usually used to represent location in space. For example in 2D maps, a projection of space structures the representation, whilst in 3D models of the body or of physical processes such as meteorological predictions graphics are constrained by a 3D locational framework. On the other hand, Information Visualization allows designers to choose among a palette of possible representations that fill space in a variety of ways, such as hierarchies, time lines, networks, tabular displays and the like, to produce information abundant displays. Choosing the appropriate representation(s) is challenging and research is needed to evaluate and compare different approaches. User studies are critical to judge the relative merits of different representations. Tightly coupled displays and highly interactive interfaces using the Zoom, Filter and Detail on Demand principles are needed to allow users to rapidly explore alternative views of the data in a matter of seconds to answer instant queries. Advanced interfaces also need to address the longer term process of analysis that may require annotation, history keeping, collaboration with peers, and the dissemination of results and procedures used. Faster rendering algorithms, sophisticated aggregations techniques to deal with large datasets, and novel labeling techniques are also needed, and along with careful studies of users and their needs will lead to successful visualization applications. Many of these, and related issues are addressed in the following chapters by researchers from a number of related fields. Information Visualization is becoming increasingly accessible to the general public and attention should be given to the goal of universal usability by enabling the widest range of users to benefit from the applications we develop. Universal usability remains a formidable challenge as we just begin to address the needs of users with slow modems, small screens, or wireless devices. Translation to other languages, access for novice, low education and low motivation users, children and elders all present special difficulties, and users with visual impairments often remain the "forgotten users" of Information Visualization. Tactile solutions are promising for blind users but sonification might provide wider access to map information, as it does not require specialized hardware.
Acknowledgements The content of this chapter was inspired in part by Ben Shneiderman's many writings on Information Visualization and Universal Usability, and also by papers co-authored with Haixia Zhao, Hyunmo Kang and other members of the Human-Computer Interaction Laboratory. Some of the work presented was funded by the National Science Foundation under Grant No. EIA 0129978 and the US Census Bureau. I also thank the editors of the book for their careful and extensive editing of the chapter
References Ahlberg, C., Williamson, C., and Shneiderman, B., (1992) "Dynamic queries for information exploration: an implementation and evaluation", Proceedings A CM CHI '92: Human Factors in Computing Systems. New York: ACM, pp. 619-626.
76
C. Plaisant
Andrienko and Andrienko (1999a). Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(23/10/03). Andrienko, G., and Andrienko, N., (1999b) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999c) "Knowledge-based visualization to support spatial data mining", Proceedings of Intelligent Data Analysis. Berlin: Springer Verlag, pp. 149-160. Andrienko, G., and Andrienko, N., (1999d) "Making a GIS intelligent: CommonGIS project view", AGILE'99 Conference, Rome, April 15-17, pp. 19-24. Andrienko, G. and Andrienko, N., (1999e) Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(03/10/02). Andrienko, G. L., and Andrienko, N. V., (1999f) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Andrienko, N., Andrienko, G., Voss, H., Bernardo, F., Hipolito, J., and Kretchmer, U., (2002) "Testing the usability of interactive maps in CommonGIS", Cartographic and Geographic Information Science, 29(4), 325-342. Becker, R. A., and Cleveland, W. S., (1987) "Brushing scatterplots", Technometrics, 29(2), 127-142. Bederson, B., (1994) "Pad+ +: advances in multiscale interfaces", Proceedings Human Factors in Computing Systems CHI '94 Conference, Boston, MA, p. 315. Bederson, B., and Shneiderman, B., (2003) The Craft of Information Visualization: Readings and Reflections. San Francisco: Morgan Kaufman. Bederson, B. B., Meyer, J., and Good, L., (2000) "Jazz: an extensible zoomable user interface graphics ToolKit in Java, UIST 2000, ACM symposium on user interface software and technology", CHI Letters, 2(2), 171-180. Bederson, B. B., Shneiderman, B., and Wattenberg, M., (2002) "Ordered and Quantum treemaps: making effective use of 2D space to display hierarchies", A CM Transactions on Graphics (TOG), 21(4), 833-854 (Oct. 2002). Brewer, C. and Harrower, M., (2002) ColorBrewer - Selecting Good Color Schemes for Maps. Online: http://www.personal.psu.edu/faculty/c/a/cab38/ColorBrewerBeta.html (23/10/03). Card, S. K., Mackinlay, J. D., and Shneiderman, B., (eds.), (1999) Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann Publishers. Carlis, J., and Konstan, J., (1998) "Interactive visualization of serial periodic data", Proceedings of User Interface Software and Technology. New York: ACM, pp. 29-38. Chen, C., (1999) Information Visualisation and Virtual Environments. Berlin: Springer Verlag. Chen, C., and Czerwinski, M., (2000) "Introduction to the special issue on empirical evaluation of Information Visualizations", International Journal of HumanComputer Studies, 53(5), 631-635 (Nov. 2000).
Information Visualization and the Challenge of Universal Usability
77
Cleveland, W. S., (1994) The Elements of Graphing Data: Revised Edition. Boca Raton, FL: CRC Press. Cockburn, A., and McKenzie, B., (2002) "Evaluating the effectiveness of spatial memory in 2D and 3D physical and virtual environments", Proceedings ACM CHI 2002 Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 203-210. Corda Technologies Inc., (2003) Corda Products Optimap 5. Online: http ://www.corda.com/products/optimap/(23/10/03). Dang, G., North, C., and Shneiderman, B., (2001) "Dynamic queries and brushing on choropleth maps", Proceedings International Conference on Information Visualization. Piscataway, NJ.: IEEE Press, pp. 757-764. DiBiase, D., MacEachren, A. M., Krygier, J. B., and Reeves, C., (1992) "Animation and the role of map design in scientific visualization", Cartography and Geographic Information Systems, 19(4), 201-214, see also pp. 265-266. Dorling, D., (1992) "Stretching space and splicing time: from cartographic animation to interactive visualization", Cartography and Geographic Information Systems, 19(4), 215-227, see also pp. 267-270. Dougherty, R. and Wade, A., (2002) Vischeck. Online: http://www.vischeck.com/(23/ 10/03). Dumas, J. S., and Redish, J. C., (1999) A Practical Guide to Usability Testing. Exeter: Intellect Books, p. 404. Eick, S., Steffen, J., and Sumner, E., (1992) "SeeSoft~a tool for visualizing line-oriented software statistics", IEEE Transactions on Software Engineering, 18(11), 957-968. Fekete, J.-D., and Plaisant, C., (1999) "Excentric labeling: dynamic neighborhood labeling for data visualization", Proceedings of ACM Conference on Human Factors in Computing Systems, CH! '99. New York: ACM Press, pp. 512-519. Fekete, J., and Plaisant, C., (2002) "Interactive Information Visualization of a million items", Proceedings of IEEE Conference on Information Visualization, Boston, Sept. 2002, pp. 117-124. Fisher, P., (1994) "Hearing the reliability in classified remotely sensed images", Cartography and Geographic Information Systems, 21 (1), 31-36. Friendly, M., (1994) "Mosaic displays for multi-way contingency tables", Journal of the American Statistical Association, 89(425), 190-200. Furnas, G. W., (1986) "Generalized fisheye views", Proceedings CHI'86 Conference: Human Factors in Computing Systems. New York: ACM Press, pp. 16-23. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002a) "GeoVISTA Studio: a geocomputational workbench, Computers", Environment and Urban Systems, 26, 267-292. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002b) "Introducing GeoVISTA Studio: an integrated suite of visualization and computational methods for exploration and knowledge construction in geography", Computers, Environment and Urban Systems, 26(4), 267-292. Golledge, R. G., (1991) "Tactual strip maps as navigational aids", Journal of Blindness and Vision Impairment, 85(7), 296-301.
78
C. Plaisant
Golledge, R. G., Loomis, J. M., Klatzky, R. L., Flury, A., and Yang, X. L., (1991) "Designing a personal guidance system to aid navigation without sight: progress on the GIS component", International Journal of Geographical Information Systems, 5(4), 373-395. Griffin, A., (2002) "Feeling it out: the use of haptic visualization for exploratory geographic analysis", Cartographic Perspectives, 39, 12-29. Grosjean, J., Plaisant, C. and Bederson, B., (2002) SpaceTree. Online: http://www.cs.umd.edu/hcil/spacetree (23/10/03). Imperatore, W., (1992) The Braille World Atlas. Louisville, Kentucky: American Printing House for the Blind, Inc. Inselberg, A., (1985) "The plane with parallel coordinates", The Visual Computer, 1, 69-91. Inselberg, A., (1997) "Multidimensional detective", Proceedings of lEEE Symposium on Information Visualization (InfoVis '97), Phoenix, AZ, Oct. 1997, pp. 100-107. Inxight Software Inc., (2002) Inxight Software - Unstructured Data Management Solutions. Online: http://www.inxight.com (23/10/03). Jeong, W., and Gluck, M., (2002) "Multimodal bivariate thematic maps with auditory and haptic display", Proceedings of 2002 International Conference on Auditory Display, Kyoto, Japan, pp. 1-4. Johnson, B., and Shneiderman, B., (1991) "Tree-maps: a space-filling approach to the visualization of hierarchical information structures", Proceedings IEEE Visualization '91. Piscataway, NJ: IEEE, pp. 284-291. Keim, D. A., (2001) "Visual exploration of large data sets", Communications of the ACM, 44(8), 38-44. Keim, D. A., and Kriegel, H.-P., (1994) "Database exploration using multidimensional visualization", Computer Graphics & Applications, 6, 40-49, (September 1994). Kraak, M. J., (undated) Minard - What has Been Done by ITC? Online: http://www.itc.nl/personal/kraak/1812/minard-itc.htm (23/10/03). Kraak, M. J., (1989) "Computer-assisted cartographic three-dimensional imaging techniques", In: Raper, J., (ed.), Three Dimensional Applications in Geographical Information Systems. London: Taylor & Francis, pp. 99-114. Kraak, M. J., and MacEachren, A. M., (1994) "Visualization of the temporal component of spatial data", In: Waugh, T., and Healey, R., (eds.), Proceedings of the Sixth International Symposium on Spatial Data Handling, Advances in GIS Research, Edinburgh, Scotland, pp. 391-409. Krygier, J., (1994) "Sound and geographic visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Oxford, UK: Pergamon, pp. 146-166. Kuniavsky, M., (2003) Observing the User Experience: A Practitioner's Guide to User Research. San Francisco: Morgan Kaufmann. Kwan, M. P., (2000) "Interactive geovisualization of activity-travel patterns using 3D geographical information systems: a methodological exploration with a large data set", Transportation Research Part C-Emerging Technologies, 8(1-6), 185-203.
Information Visualization and the Challenge of Universal Usability
79
Lamping, J., Rao, R., and Pirolli, P., (1995) "A focus + context technique based on hyperbolic geometry for visualizing large hierarchies", Proceedings of CHI '95 Human Factors in Computing Systems, Denver, CO, May 1995. New York: ACM Press, pp. 401-408. Landau, S., and Gourgey, K., (2001) "Development of a talking tactile tablet", Information Technology and Disabilities, VII(2). Lindwarm, D., Rose, A., Plaisant, C., and Norman, K., (1998) "Viewing personal history records: a comparison of tabular format and graphical presentation using LifeLines", Behaviour & Information Technology, 17(6), 249-262. MacEachren, A. M., Dai, X., Hardisty, F., Guo, D., and Lengerich, G., (2003a) "Exploring high-D spaces with multiform matrices and small multiples", IEEE Symposium on Information Visualization, Seattle, pp. 31-38. MacEachren, A. M., Hardisty, F., Dai, X., and Pickle, L., (2003b) "Geospatial statistics supporting visual analysis of federal GeoSpatial statistics", Digital Government Table of Contents, pp. 59-60. Muntz, R. R., Barclay, T., Dozier, J., Faloutsos, C., MacEachren, A. M., Martin, J. L., Pancake, C. M., and Satyanarayanan, M., (2003) "IT roadmap to a geospatial future", Report of the Committee on Intersections Between Geospatial Information and Information Technology. Washington, DC: National Academy of Sciences. Mynatt, E., and Edwards, K., (1992) "Mapping GUIs to auditory interfaces", Proceedings of the 5th Annual A CM Symposium on User Interface Software and Technology. New York: ACM Press, pp. 61-70. N Space Labs Inc., (2003) incMapTM: Geographic Map of the Market. Online: http://www.nspacelabs.com/(23/10/03). National Cancer Institute, (2003) Cancer Mortality Maps and Graphs. Online: http://www3.cancer.gov/atlasplus/(23/10/03). Nielson, G. M., Hagen, H., and Mueller, H., (1997) Scientific Visualization: Overviews, Methodologies, Techniques. Piscataway, NJ: IEEE Computer Society. North, C., Conklin, N., and Saini, V., (2002) "Visualization schemas for flexible Information Visualization", Proceedings of InfoVis 2002, pp. 15-22. Olson, J., and Brewer, C. A., (1997) "An evaluation of color selections to accommodate maps users with color-vision impairments", Annals of the Association of American Geographers, 87(1), 103-134. OmniViz Inc., (2003) Omniviz. Online: http://www.omniviz.com (23/10/03). Pape, D., (1998), "Crayoland". In: SIGGRAPH Video Review, Issue 127/SIGGRAPH '98 Electronic Art and Animation Catalog CD-ROM. Orlando, FL: ACM. Online: http://www.evl.uic.edu/pape/projects/crayoland. Parente, P., and Bishop, G., (2003) "BATS: the blind audio tactile mapping system", Proceedings of ACM South Eastern Conference, Savannah, GA, http://www.cs.unc.edu/Research/assist/bats/history.shtml (Mar. 2003). Parkes, D., (1994) "Audio tactile systems for designing and learning complex environments as a vision impaired person: static and dynamic spatial information access", In: Steele, J., and Hedberg, J. G., (eds.), Learning Environment Technology: Selected Papers from LETA 94. Canberra, AJET Publications, pp. 219-223.
80
C. Plaisant
Perlin, K., and Fox, D., (1993) "Pad: an alternative approach to the computer interface", In: Kajiya, J. T., (ed.), Proceedings of the 20th Annual A CM Conference on Computer Graphics (SIGGRAPH '93), Aug. 2-6, Anaheim, CA. New York: ACM Press, pp. 57-64. Pickle, L., (2003) "Usability testing of map designs", Proceedings of Interface 2003. Alexandria, VA: ASA. Plaisant, C., (1993) "Facilitating data exploration: dynamic queries on a health statistics map", Proceedings of the Annual Meeting of the American Statistical Association of the Government Statistics Section, San Francisco, CA, Aug 18-23. Alexandria, VA: ASA pp. 18-23. Plaisant, C., Rose, A., Milash, B., Widoff, S., and Shneiderman, B., (1996) "LifeLines: visualizing personal histories", Proceedings of CHI '96 Conference: Human Factors in Computing Systems. New York: ACM, pp. 221-227, 518. Plaisant, C., Grosjean, J., and Bederson, B., (2002) "SpaceTree: supporting exploration in large node link tree, design evolution and empirical evaluation", Proceedings of IEEE Conference on Information Visualization, pp. 57-64. Plaisant, C., Kang, H., and Shneiderman, B., (2003) "Helping users get started with visual interfaces: multi-layered interfaces, integrated initial guidance and video demonstrations", Proceedings of Human-Computer Interaction International 2003, Crete, June 2003, pp. 790-794. Preece, J., Rogers, Y., and Sharp, H., (2002) Interaction Design: Beyond HumanComputer Interaction. New York: Wiley. Ramloll, R., Brewster, S., Yu, Y., and Riedel, B., (2001) "Using non-speech sounds to improve access to 2D tabular numerical information for visually impaired users", Proceedings 15th British HCI Group Annual Conference on Human-Computer Interaction (IHMHCI), Lille, France, Sept. 10-14. Rao, R., and Card, S. K., (1994) "The table lens: merging graphical and symbolic representation in an interactive focus + context visualization for tabular information", Proceedings Human Factors in Computing Systems CHI '94 Conference, Boston, MA. New York: ACM, pp. 318-322. Rosenthal, O., and Phillips, R., (1997) Coping With Color-Blindness. New York: Avery Publishing Group. Schneider, J., and Strothotte, T., (2000) "Constructive exploration of spatial information by blind users", Proceedings of the A CM Conference on Assistive Technologies, ASSETS 2000. New York: ACM, pp. 188-192. Shinn-Cunningham, B. G., Lehnert, H., Kramer, G., Wenzel, E. M., and Durlach, N. I., (1997) "Auditory displays", In: Gilkey, R., and Anderson, T., (eds.), Binaural and Spatial Hearing in Real and Virtual Environments. New York: Erlbaum, pp. 611-663. Shneiderman, B., (1997) Designing the User Interface - Strategies for Effective HumanComputer Interaction. Reading: Addison-Wesley Publishing Company, p. 639. Shneiderman, (1998) Treemaps for Space-Constrained Visualization of Hierarchies. Online: http ://www.cs.umd.edu/hcil/treemaps (23/10/03). Shneiderman, B., (2000) "Universal usability: pushing human-computer interaction research to empower every citizen", CACM, 23(5), 84-91.
Information Visualization and the Challenge of Universal Usability
81
Shneiderman, B., (2001) "Supporting creativity with advanced information-abundant user interfaces", In: Earnshaw, R., Guedj, R., Van Dam, A., and Vince, J., (eds.), Human-Centred Computing, Online Communities, and Virtual Environments. London: Springer-Verlag, pp. 469-480. Shneiderman, B., and Hochheiser, H., (2001) "Universal usability as a stimulus to advanced interface design", Behaviour&Information Technology, 20(5), 367-376, Online: http ://www.otal.umd.edbu/uupracticechen SmartMoney.com, (2003) Smartmoney.com - Investing, Saving and Personal Finance. Online: http://www.smartmoney.com (23/10/03). Spence, R., (2001) Information Visualization. Harlow: Addison Wesley/ACM Press Books, 206 pp. Szego, J., (1987) Human Cartography: Mapping the Worm of Man. Stockholm: Almqvist & Wiksell International, p. 236. The Hive Group Inc., (2003) The Hive Group - Creators of Honeycomb Technology. Online: http://www.hivegroup.com (23/10/03). The Sendero Group, (2003) Sendero Group Homepage - GPS for the blind. Online: http://www.senderogroup.com Theus, M., (2002a) "Geographical Information Systems", In: Kloesgen, W., and Zytkow, J., (eds.), Handbook of Data Mining and Knowledge Discovery. Oxford: Oxford University Press. Theus, M., (2002b) "Interactive data visualization using Mondrian", Journal of Statistical Software, 7(11). Tufte, E. R., (1983) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, p. 197. Tweedie, L., Spence, R., Dawkes, H., and Su, H., (1996) "Externalising abstract mathematical models", In: Bilger, R., Guest, S., and Tauber, M. J., (eds.), CHI '96: Conference on Human Factors in Computing Systems, Vancouver, BC, ACM/ SIGCHI, Online: http://www.acm.org/sigs/sigchi/chi96/proceedings/papers/Tweedie/lt 1txt.htm Vanderheiden, G., (2000) "Fundamental principles and priority setting for universal usability", Proceedings A CM Conference on Universal Usability 2000. New York: ACM, pp. 32-38. VirTouch Ltd., (2002) Innovative Tactile Solutions for PC Users Who Are Blind. Online: http://www.virtouch.com/(23/10/03). VisuAide, (2003) GPS Orientation Solutions - Trekker. Online: http://www.visuaide.com/gpssol.html (23/10/03). Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, p. 384. Weibel, R., and Jones, C. B., (1998) "Computational perspectives on map generalization", Geoinformatica, 2(4), 307-314. Williamson, C., and Shneiderman, B., (1992) "The dynamic home finder: evaluating dynamic queries in a real-estate information exploration system", Proceedings of the ACM SIGIR'92 Conference. New York: ACM, pp. 338-346.
82
C. Plaisant
Wise, T. A., Thomas, J. J., Pennock, K., Lantrip, D., Pottier, M., Schur, A., and Crow, V., (1995) "Visualizing the non-visual: spatial analysis and interaction with information from text documents", Proceedings IEEE Information Visualization '95, Los Alamitos, CA. Piscataway, NJ: IEEE Computer Press, pp. 51-58. Woodruff, A., Landay, J., and Stonebreaker, M., (1998) "Constant information density in zoomable interfaces", Proceedings of the 4th International Working Conference on Advanced Visual Interfaces A VI '98, L'Aquila, Italy, May 24-27, pp. 110-119. Zhao, H., Plaisant, C., and Shneiderman, B., (2003) "Improving accessibility and usability of geo-referenced statistical data", Proceedings 2003 National Conference on Digital Government Research, Online: http://www.dgrc.org/dgo2003 Zotkin, D. N., Duraiswami, R., and Davis, L. S., (2002) "Creation of virtual auditory spaces", Proceedings International Conference on Acoustic Speech and Signal Processing, Orlando, FL, May 2002, pp. 2113-2116.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter 4
Beyond Tools: Visual Support for the Entire Process of GIScience Mark Gahegan, GeoVISTA Center, Department of Geography, The Pennsylvania State University, University Park, PA 16802, USA
Keywords: geovisualization, GIScience, semantics, inference, science, activities, reasoning, visual support
Abstract Progress to date within the geovisualization research community has led to some very useful, innovative tools and systems. However, these tools are typically neither easy to integrate or use together, nor is it always clear what scientific activities they help to facilitate. In order to bring visualization tools to the fore, we must understand clearly what roles they play within the entire process of scientific investigation, including the various tasks that a researcher might undertake and the different kinds of reasoning that we wish visualization to support or enable. This chapter presents an overview of the science process, gives examples of how current approaches to visualization support certain research activities and suggests a framework by which additional aspects of: (i) science activity, (ii) reasoning mode (inference), (iii) outcomes, and (iv) pre-requisite data and knowledge could be used to better define the role or roles that different visualization tools can play, and also in how such tools might be better connected or integrated together. The chapter ends by summarizing the research challenges that follow from a more semantic, process-oriented approach to the use and integration of geovisualization tools.
4.1
Introduction
Consider, if you will, the entirety of the process of "doing" GIScience, perhaps starting by exploring an unfamiliar dataset and discovering emergent patterns - through knowledge construction where these patterns are formed into the building-blocks of analysis: concepts, relationships and instances - then onto analysis where models are constructed from these building blocks to make predictions or descriptions - these models are then evaluated to test their integrity and accuracy - and finally results are reported by way of maps, images, numbers or words. This is just a single example of 83
84
M. Gahegan
Figure 4.1. The nexus of activities comprising the scientific process (Gahegan and Brodaric,
2002a,b).
a path through the nexus 1 of activities that is defined by the geo-scientific process and as later figures show, many more such paths are equally valid. Clearly, the process itself is very complex, with myriad connections and relationships between the various activities. For example, emergent concepts that at first appear useful when exploring the data may turn out to be too difficult to categorize, may fail to provide insightful results during analysis or may be problematic to understand or communicate. So at any stage, they might be abandoned, requiring the analyst to revisit exploration or synthesis. Despite the inherent complexities, there is a mature understanding of the process of science viewed from both the perspective of the philosophy of science (Peirce, 1891; Popper, 1959; Thagard, 1988; Shrager and Langley, 1990; Sowa, 1999) and of the practicing GIScience and earth science community (Schumm, 1991; Baker, 1996; Martin, 1998). The contention presented here is that visualization can, and should, support the entire geoscientific process. But how might this goal be best achieved?
4.2
Current Geovisualization Tools: Some Pros and Cons
Now, ponder for a moment the increasing number of visualization and analysis techniques and tools available and the tasks they enable researchers to carry out. On the surface, many of these appear to have clear purposes or roles, defined by our experiences of using the tool or technique or seeing others use it, or inferred from a theoretical understanding gained from relevant articles and documentation, or even from the name alone (c.f. IBM's Data Explorer). At a finer level of detail, however, it may not be clear exactly what activities (in Figure 4.1) they can or cannot support, how well they can support these activities, and what biases they will impose on the investigation. l Nexus used in the sense described by Alfred North Whitehead in Adventures of Ideas (Whitehead, 1938).
Beyond Tools: Visual Support for the Entire Process of GIScience
85
By contrast, such information is often readily available for many kinds of statistical procedures. There are many excellent examples of tools that address specific activities in the social scientific process (for examples see the lists of resources held by the Center for Spatially Integrated Social Science - CSISS, 2003) and the Teaching Resources and Materials for Social Scientists (Wiggins et al., 2000). Concerning visualization specifically, many tools and systems have been developed that allow the same data to be viewed in a variety of displays, with a high degree of interactivity between the displays, often called "linking and brushing" (Monmonier, 1990a,b; MacDougal, 1992; Cook et al., 1996; Dykes, 1996; Andfienko and Andrienko, 1999; Gahegan et al., 2002a,b). However, the tools we currently have are, by and large, hosted in closed systems that cannot easily be modified or integrated with each other, and specifically where there is little or no thought to the positioning or integrating of tools into the wider scientific process shown above such that the investigator can move seamlessly from one tool to another, back and forth, as activities dictate. From the perspective of the academic community, this is, in part, a consequence of the fact that academics typically build tools to test and present ideas, rather than to become fully operational within a community, and the most effective way to build such tools is to close as many aspects of the system as possible, then there is less to build. From the commercial perspective, open systems carry the threat of a potential loss of market share, and again they are more problematic to develop. Furthermore, to date there is little to guide the investigator in selecting the right tool(s) for the task(s) at hand from those available, nor any way to feed back evidence of success (or failure) when particular tools are used for a specific task. This is nobody's fault, p e r se, but a consequence of many isolated research efforts that focus on pioneering specific techniques. What is required as geovisualization develops is a broader vision by which relevant tools and techniques might be orchestrated into something greater than the sum of their parts. This vision is likely to include perspectives from other disciplines and may continue to incorporate techniques and knowledge from cognate fields with expertise in visualization. Another problem with our current geovisualization tools is that they typically are focused only on the data, and not on conceptual structures such as categories or relationships that operate at a higher level of abstraction. Therefore, while they provide good support for data exploration they are less useful for synthesis activities, where concepts are constructed, or for analysis, where concepts are operationalized. To better support the entire science process, we must provide mechanisms that can visualize the connections between the various stages of analysis, and show how concepts relate to data, how models relate to concepts, and so forth. Returning to Figure 4.1, consider the substance of the links between the various scientific activities, specifically what are the inputs and outcomes that transform one stage to another? Considering just the transitions between exploration and synthesis stages, how might the mapping between data exploration and concept synthesis be visualized? During the early stages of an investigation, an analyst might recognize emergent structures in the data and synthesize these into concepts for use in analytical models. But how might this synthesis be signified so that it too can be explored?
86
M. Gahegan
We need to better understand the interfaces between science activities for two reasons. Firstly, from a theoretical perspective, such understanding will help us to gain a deeper appreciation of the investigative processes used by scientists, and hence increase understanding of how early decisions regarding the data used and the concepts synthesized will affect the outcomes of later analysis models. Secondly, and from a practical perspective, these linkages define interface specifications by which visualization tools can exchange relevant information with each other as the user switches focus between activities. New approaches to visualization are needed that can cross these "bridges of abstraction" that we mentally construct between the science activities in Figure 4.1. Some possible approaches are given by MacEachren et al. (1999a-c), Gahegan and Brodaric (2002b), Lucieer and Kraak (2002), and some examples are provided in w following a more detailed account of GIScience activities.
4.3
A More Detailed Look at the Activities of GIScience
Returning to the process of science, Figure 4.2 adds examples of conceptual structures (light grey) and concretized representations (dark grey) linking the various stages of the nexus shown in Figure 4.1. The figure shows one plausible path (selected from
Figure 4.2. A plausible path around the core scientific activities introduced in Figure 4.1. See text for details.
Beyond Tools: Visual Support for the Entire Process of GIScience
87
many possibilities) linking these core scientific activities, beginning with exploration and ending with the production of a map of categories synthesized from the data. The path might exemplify the building of a map-based representation and understanding of some phenomenon such as the degree of demographic change occurring between two census surveys (where such change is undefined at the outset, but emerges from the analyst's interactions with the data), or of producing a geological or detailed land cover map of an unexplored region. Dark grey ovals represent tangible inputs or outcomes, light grey ovals represent intangible knowledge and expertise provided by the analyst and medium grey rectangles are activities from Figure 4.1. The block arrows show the general directional trend, but the analyst may of course move back and forth between activities as revision takes place. For example, if a researcher is studying land cover mapping and is trying to synthesize useful categories for indicating a range of different species or ground conditions, then current tools can help them understand the general patterns and outliers in remote sensing and ancillary datasets, and via linking and brushing the trends in scatter plots can be explored spatially in a map, and visa versa. But usually, when the categories or indicators are formed, their connection to, and dependence on, the original data are "lost" and cannot be examined in any later stages. Likewise, the classification tools used are also, for the most part, black boxes, so the user cannot explore the cause and effect relationships between the selection of data, their own mental understanding of the study area and the analysis tools they are using. Figure 4.3 clarifies by adding some typical questions that might be asked at each of the conceptual stages, and draws the significant impacts of prior knowledge and experience into the model. The challenge that results from this situation is one of how to engineer visualization systems that explicitly support connections between data, classifiers and categories, and later between categories and prediction models that enable us to answer these kinds of questions as and when required by the process of GIScience. Figure 4.4 shows exploratory visualization tools linked to the visualization of classifiers to gain insight into the process of classification, from data exploration, through synthesis (starting with data and ending with categories in Figures 4.2 and 4.3). A parallel coordinate plot (top left) and scatter plot (right) show several training examples from a land cover mapping exercise. The utility of a maximum-likelihood classifier is shown for each separate attribute dimension across the bottom of the figure and can be visually assessed by comparing the observed points to the theoretical Gaussian distribution (the bell-shaped curves, one for each category). The top right panel shows how the same data points appear in a scatter plot matrix and the middle panel on the left shows the land cover taxonomy used as the conceptual input to the classification process. Interactive linking and brushing is supported not only between the different views of the data, but also between the data and the visual representation of the internal working of the classifiers, allowing the user to explore relationships between the two. Such functionality allows the user to ask direct questions regarding the path from exploration to synthesis (Figure 4.1) such as: Which particular training examples cause the most classification error, and where are they located in geographic space?
Figure 4.3. Questions that might be asked during a land cover change assessment exercise as illustrated by the plausible path around the core scientific activities introduced in Figure 4.2.
c
Beyond Tools: Visual Support for the Entire Process of GIScience
3
89
90
M. Gahegan 2.
3. 4.
Which categories show the most confusion or overlap and therefore might benefit from a different conceptualization or synthesis, for example by generalizing or specializing them further using the taxonomy? Which data attributes (columns in the parallel coordinate plot) only add confusion into the classification process? Which kind of classifier works best for specific categories and why?
Researchers appear very adept at mentally switching between these analysis tasks or stages, and many scientific problems rely on such movement since reaching a solution involves iterating between what the researcher wants to achieve and what can be supported from the data and methods available, and hopefully reaching some middle ground. For example, categories such as those describing land cover introduced in Figure 4.4 are used throughout human and physical geography. As illustrated in Figures 4.2 and 4.3, they are first conceptualized, then synthesized, next applied and subsequently evaluated. There are many ways by which categories can be synthesized (e.g., from theory, directly from observations, from experience) and many theories, methods and datasets by which they may be instantiated and applied. Hence, the process of inference is typically a compromise between data, theory and methods that
Figure 4.5. Other possible paths through the nexus shown in Figure 4.1, beginning with evaluation of results (which proves unsatisfactory in this example) and backtracking through previous states to find and correct problems.
Beyond Tools: Visual Support for the Entire Process of GIScience
91
might take several attempts to converge on a satisfactory outcome (methods used in ways that are consistent with both theory and data). Land cover categories, demographic indicators and eco-regions are all examples where the researcher may go back and forth through exploration, synthesis, analysis and evaluation as their understanding deepens. Of course, there are many more paths than those shown in Figures 4.2 and 4.3 that an investigation might take. Figure 4.5 illustrates one such alternative, starting later in the investigation process with the evaluation of results stage that uncovers some problem, potentially due to inconsistencies in the prior stages. The researcher is now faced with a number of choices to track down and remediate the problem: they could begin with the data, the concepts, the model and so forth.
4.4
Visual Support for Different Kinds of Reasoning
Now let us consider the nature of the reasoning that underlies the tasks that the researcher must undertake, with specific focus on the types of inferential mechanisms that are used in the transition from one state to the next in Figures 4.2, 4.3 and 4.5 (Baker, 1999; Brodaric and Gahegan, 2002). The more we understand about how the researcher is trying to think and what they are attempting to produce, the better we can design suitable visualization tools that support their tasks. Figure 4.6 adds the dominant type of inference undertaken as the researcher engages in each activity (Peirce, 1891). These can be summarized as follows: Deduction: The application of a set of pre-defined rules to data, producing predefined outcomes (e.g., syllogistic logic, rules of the form: if A then B else C). Induction: Generalization from a limited number of examples to a larger population, typically beginning with the construction (learning) of categories followed by their application to a broader collection of data (Gahegan, 2003). For example, classifying the land cover example in w using a small number of training examples. Abduction: The combined act of seeing and explaining, in the sense of observing some specific stimulus and simultaneously finding, or synthesizing, a hypothesis to explain it (Thagard and Shelley, 1997). Sometimes, this is known as the "A-ha!" moment, for example when making sense of an optical illusion. Model-based: Reasoning around and through mental and computationally held models, allowing evaluation to be grounded within structural, causal, and/or functional constraints (Magnani and Nersessian, 2002). An example would be when investigating different climate change scenarios using a global circulation model. Rhetoric: Forms of persuasion such as metaphoric reasoning that are used to justify a stance where formal theory and models cannot fully "prove" correctness - as is the case with many aspects of earth and social science (Covino and Jolliffe, 1995). The arguments in this chapter are largely based on
Smtterplot, grand tour, projection pursuit,
parallel coordinate plot.
Figure 4.6. Scientific activities shown with dominant types of inference undertaken by the analyst (deduction, induction, abduction, rhetoric, modelbased) and examples of current visualization tools (shown in white boxes) and computational tools (shown in black boxes). The question that emerges is “how do we enable the researcher to move between tools that span different activities”?
Beyond Tools: Visual Support for the Entire Process of GIScience
93
rhetorical reasoning (to motivate a more formal approach via model-based reasoning). Figure 4.6 also suggests some possible types of geovisualization tools (white boxes) and statistical-computational tools (black boxes) that "might" facilitate the various activities. These suggestions are made from the author's own experiences and are illustrative, not definitive. Our goal of a more systematic approach to geovisualization might begin by investigating the effectiveness of different existing geovisualization tools and techniques in supporting the different activities shown. Apart from a deeper understanding of the roles that our visualization devices play, this would also suggest tools that need to work directly (interoperate) with each other, that is, coordinate their behavior more closely to help the researcher transition between analysis activities. At a lower level of abstraction, Catherine Plaisant describes a related approach to visualization tool capability via a "data by tasks" taxonomy including seven basic data types and seven basic tasks such as: overview, filter, relate, and extract (Shneiderman, 1998; see Plaisant, this volume (Chapter 3)). This approach provides a framework for measuring the utility of each tool in the context of specific, low-level analysis tasks to be undertaken and the types of data to be analyzed. Figure 4.6 adds to such a framework the high-level analysis activities that the researcher is engaged in, and the types of inference they are attempting to use, the concrete outcomes from these activities and the associated mental knowledge leveraged (Gahegan, 2001). Using these dimensions of: (i) activity, (ii) inference, (iii) outcomes, and (iv) pre-requisite data and knowledge, we can formulate statements that describe the example used above in w such as: A geovisualization tool used for category synthesis must support the inductive formulation of a proposed category, from a multidimensional data-space, based on pre-existing concept(s). This can be contrasted with the current situation we typically face, where our choice of tools is based on more practical considerations such as: tool accessibility, the dimensionality of the data to be visualized and the data importing capabilities. Whereas these are indeed worthwhile considerations, they do not, of themselves, guarantee that the tool we use will help solve the problem we are considering. Such a framework could also support very specific questions that might aid the researcher who does not know which of all the available tools are the most appropriate, such as: What are the individual tasks that comprise inductive category formation and which visual tools can support them? The answer might include tasks such as: (i) inductively defining a category from examples, (ii) removing or dealing with data outliers, (iii) checking for the separability of categories, and so forth.
94
4.5
M. Gahegan
Summary Findings
The discussion presented here can be summarized by the following three major goals, each of which generates a number of significant research challenges as we explore, extend and advance geovisualization. Goal 1. Define a logical framework 2 that provides the conceptual underpinning necessary to "situate" visualization tools and analysis tasks with the scientific activities they empower. a. What is the process, or what are the processes, we are ultimately trying to facilitate? b. What activities within GIScience do we wish to facilitate and how are they interrelated? c. What visual methods and techniques facilitate each of these activities? d. What inferential mechanisms are being employed by the researcher? e. What gaps are evident (where no or few methods are currently available), and where do we have an abundance of methods already?
Goal 2. Adopt an open systems approach to engineering visualization tools, so that systems and platforms do not cause barriers to integration. a. How can the visualization environments and methods be made open and flexible, so that they can be integrated together to support the entire scientific process in an holistic fashion? b. How can systems be open schematically (ensuring that tools can be shared among the research community and that new tools can be added into existing systems as and when they become available)? c. How can systems be open semantically (avoiding or mediating ontological commitments that would otherwise render the tools incommensurate)? Andrienko et al., this volume (Chapter 5) take up this theme of interoperability when considering the issues associated with "Creating Instruments for Ideation" from a number of perspectives.
Goal 3. Devise an infrastructure for coordination between components that draws from the logical framework of Goal 1. a. How can visual tools in a geovisualization environment coordinate their behavior, and specifically support the migration of conceptual understanding (e.g., concepts, models, results) between specific tools? b. What coordination mechanisms are available or can be created to provide the necessary functionality and flexibility? c. What exactly should be coordinated between tools in order to communicate the essence of the science process?
2 Or meta-framework from which alternative frameworks can be synthesized as needed.
Beyond Tools: Visual Support for the Entire Process of GIScience
4.6
95
Concluding Remarks
The GIScience and Information Visualization communities face some difficult challenges regarding the tools and analysis methods that empower science (Bishr, 1998): 9
9
9
Although it is still difficult to share data between researchers and systems, it is even more difficult to share functionality. Many potentially useful tools for spatial analysis, uncertainty modeling, visualization, spatial reasoning, geocomputation and so forth are not portable in the sense that they can be readily shared with others. As a result, a lot of research funding and effort results in outcomes that have reduced practical impact. In our case here, we need to build the necessary standards and infrastructure to better integrate the visual functionality we have separately developed. Perhaps even more importantly, the lack of interoperability means that analyses cannot readily be shared (Goodchild, 2000), and therefore cannot be independently verified or refuted by other scientists. This is a real barrier to scientific progress and credibility. Although we can, and generally do, make our tools available to others, we still lack the means to "wrap" analysis sessions and (say) deploy them on the Internet so others can share (or take issue with) our "Aha!" moments (Takatsuka and Gahegan, 2002). A richer toolbox of methods is only useful if we also can develop ways by which methods can be combined, reconfigured, tested and used that are flexible, interactive and easily deployable (Griss, 2000). Our current diversity of tools is therefore only a potential asset until we work out how to better coordinate their functionality in a meaningful way (such as via the framework suggested here) in order to deploy them effectively.
In short, we have a pressing need to extend beyond classifying visualization tools based on the kinds of data and visual variables they use - to the roles that they play, the inference they enable and hence the analysis activities they facilitate. As GIScience researchers begin to embrace the new paradigm of component engineering, it becomes clear that the advantages on offer could have immense positive impacts on the way in which software is designed, developed and shared (Szyperski, 1997; Wills, 2001). Not least among these is the ability for an entire research community to collaborate in the development of useful functionality, without the need to adopt a common software development environment, or even agree on a specific computational platform. However, component programming and its relation, agent programming, bring into even sharper focus problems related to the ontological commitments held by developers and users and implicitly represented in data and methods (Guarino, 1998a,b; Fonseca and Egenhofer, 1999). The challenge therefore is for us to think more systematically about the science we are trying to support, so that we can better situate the tools we have into the process of science and to orchestrate our efforts more effectively, so that software instruments are developed to support each investigative stage and each style of reasoning. The Open GIS Consortium is now making significant progress towards open GIS functionality and services, developed by
96
M. Gahegan
a community of engaged users (Open GIS Consortium Inc., 2003a-c). Their success is a direct result of carefully designed interface specifications that are made public and adhered to. The geovisualization research community is small by comparison, but this further accentuates the need for the same kind of care and collaboration so that we can leverage the work of others, rather than re-implement it for ourselves.
Acknowledgements Many thanks to the editors, particularly Jason Dykes, who provided much useful feedback, and to GeoVISTA graduate students Xiping Dai and Sachin Oswal who wrote the software shown in Figure 4.4. This research was supported by the National Science Foundation (NSF) under grants BCS-9978052 (HERO), ITR (BCS)-0219025, and ITR (EAR)-0225673 (GEON) and National Institutes of Health (NIH) under Grant # R01 CA95949-01.
References Andrienko, G. L., and Andrienko, N. V., (1999) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Baker, V., (1996) "Hypotheses and geomorphological reasoning", In: Rhoads, B. L., and Thorn, C. E., (eds.), The Scientific Nature of Geomorphology. New York: Wiley, pp. 57-86. Baker, V., (1999) "Geosemiosis", Geological Society of America Bulletin, 111(5), 633-645. Bishr, Y., (1998) "Overcoming the semantic and other barriers to GIS interoperability", International Journal of Geographical Information Science, 12(4), 299-314. Brodaric, B., and Gahegan, M., (2002) "Distinguishing instances and evidence of geographical concepts for geospatial database design", In: Egenhofer, M., and Mark, D., (eds.), Geographical Information Science - 2nd International Conference, GIScience 2002. Berlin: Springer, pp. 22-37. Cook, D., Majure, J. J., Symanzik, J., and Cressie, N., (1996) "Dynamic graphics in a GIS: exploring and analyzing multivariate spatial data using linked software", Computational Statistics: Special Issue on Computeraided Analysis of Spatial Data, 11 (4), 467-480. Covino, W., and Jolliffe, D., (1995) Rhetoric. Boston, MA: Allyn and Bacon. CSISS (2003). Center for Spatially Integrated Social Science - Select Tools. Online: http://www.csiss.org/clearinghouse/select-tools.php3 (23/10/03). Dykes, J. A., (1996) "Dynamic maps for spatial science: a unified approach to cartographic visualization", In: Parker, D., (ed.), Innovations in Geographical Information Systems 3. London: Francis and Taylor, pp. 171-181. Fonseca, F. T., and Egenhofer, M., (1999) "Ontology-driven geographic information systems", In: Medeiros, C. B., (ed.), Proceedings of the 7th ACM Conference on GIS, Kansas City, MO, pp. 14-19, Online: http://www.spatial.maine.edu/(max/RC39.html
Beyond Tools: Visual Support for the Entire Process of GIScience
97
Gahegan, M., (2001) "Visual exploration in geography: analysis with light", In: Miller, H. J., and Han, J., (eds.), Geographic Data Mining and Knowledge Discovery. London: Taylor and Francis, pp. 260-287. Gahegan, M., (2003) "Is inductive inference just another wild goose (or might it lay the golden egg)?", International Journal of Geographical Information Science, 17(1), 69-92. Gahegan, M., and Brodaric, B., (2002a) "Computational and visual support for geographical knowledge construction: filling in the gaps between exploration and explanation", Advances in Spatial Data Handling, l Oth International Symposium on Spatial Data Handling. New York: Springer-Verlag, pp. 11-26. Gahegan, M., and Brodaric, B., (2002b) "Examining uncertainty in the definition and meaning of geographical categories", In: Hunter, G., and Lowell, K., (eds.), 5th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, Melbourne, July 10-12. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002a) "GeoVISTA Studio: a geocomputational workbench", Computers, Environment and Urban Systems, 26, 267-292. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002b) "Introducing GeoVISTA Studio: an integrated suite of visualization and computational methods for exploration and knowledge construction in geography", Computers, Environment and Urban Systems, 26(4), 267-292. Goodchild, M. F., (2000) "Keynotes address", Conference of the Association of American Geographers, Pittsburgh. Griss, M., (2000) My Agent Will Call Your Agent. Online: http://www.sdmagazine.com/articles/2000/0002/0002toc.htm (23/10/03). Guarino, N., (1998a) "Formal ontology and information systems", In: Guarino, N., (ed.), Formal Ontology in Information Systems. Proceedings of FOIS'98, Trento, Italy, June 6-8 1998. Amsterdam: IOS Press, pp. 3-15, Online: http ://www.oceanlaw.net/orgs/maps/ices_map.htm Guarino, N., (1998b) "Formal ontology and information systems", In: Guarino, N., (ed.), Formal Ontology in Information Systems (FOIS'98). Amsterdam: ICES, pp. 3-15, Online: http://www.oceanlaw.net/orgs/maps/ices_map.htm Lucieer, A., and Kraak, M. J., (2002) "Interactive visualization of a fuzzy classification of remotely-sensed imagery using dynamically linked views to explore uncertainty", In: Lowell, G. H. a. K., (ed.), 5th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, Melbourne, July 10-12, pp. 348-356. MacDougal, E. B., (1992) "Exploratory analysis, dynamic statistical visualization and geographical information systems", Cartography and Geographical Information Systems, 19(4), 237-246. MacEachren, A. M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., (1999a) "Exploring the potential of virtual environments for geographic visualization", Annual Meeting of the Association of American
98
M. Gahegan
Geographers, Honolulu, HI, 23-27 March, AAG, p. 371 (full paper: www.geovista.psu.edu/library/aag99vr). MacEachren, A. M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., (1999b) "Virtual environments for geographic visualization: potential and challenges", Proceedings of the A CM Workshop on New Paradigms in Information Visualization and Manipulation, Kansas City, KS, Nov. 6, 1999. MacEachren, A. M., Wachowicz, M., Edsall, R., Haug, D., and Masters, R., (1999c) "Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods", International Journal of Geographical Information Science, 13(4), 311-334. Magnani, L., and Nersessian, N. J. E., (2002) Model Based Reasoning: Science, Technology, Values. New York: Kluwer Academic, p. 418. Martin, R. E., (1998) One Long Experiment. New York: Columbia University Press. Monmonier, M., (1990a) "Strategies for the visualization of geographic time-series data", Cartographica, 27(1), 30-45. Monmonier, M. S., (1990b) "Strategies for the interactive exploration of geographic correlation", In: Brassel, K.,and Kishimoto, H.,(eds.), Proceedings 4th International Symposium on Spatial Data Handling, Dept. of Geography, University of Zurich, Switzerland, pp. 381-389. Open GIS Consortium Inc., (2003a) Geographic Objects Initiative (GO-l). Online: http://ip.opengis.org/go 1/(23/10/03). Open GIS Consortium Inc., (2003b) Geography Markup Language (GML 3.0). Online: http://www.opengis.org/docs/02-023r4.pdf (23/10/03). Open GIS Consortium Inc., (2003c) Open GIS Consortium, Inc. (OGC). Online: http://www.opengis.org (23/10/03). Peirce, C. S., (1891) "The architecture of theories", The Monist, 1(161 - 176), 000-000. Popper, K. R., (1959) The Logic of Scientific Discovery. New York: Basic Books. Schumm, S. A., (1991) To Interpret the Earth: Ten ways to Be Wrong. New York: Cambridge University Press. Shneiderman, (1998) Treemaps for Space-Constrained Visualization of Hierarchies. Online: http://www.cs.umd.edu/hcil/treemaps (23/10/03). Shrager, J., and Langley, P. E., (1990) Computational Models of Scientific Discovery and Theory Formation. San Mateo, CA: Morgan Kaufman. Sowa, J. F., (1999) Knowledge Representation: Logical, Philosophical and Computational Foundations. New York: Brooks/Cole. Szyperski, C., (1997) Component Software: Beyond Object-Oriented Programming. New York: ACM Press. Takatsuka, M., and Gahegan, M., (2002) "GeoVISTA Studio: a codeless visual programming environment for geoscientific data analysis and visualization", Computers and Geosciences, 28(10), 1131-1144. Thagard, P., (1988) Computational Philosophy of Science. Cambridge, MA: MIT Press. Thagard, P., and Shelley, C., (1997) "Abductive reasoning: logic, visual thinking, and coherence", In: Dalla Chiara, M.-L. e. a., (ed.), Logic and Scientific Methods. Dordrecht: Kluwer, pp. 413-427.
Beyond Tools: Visual Support for the Entire Process of GIScience
99
Whitehead, A. N., (1938) Adventures of Ideas. New York: Prentice Hall, p. 308. Wiggins, R. D., Beedham, H., Francis, B., Goldstein, H., Hanavy, M., Harman, J., Leyland, A., Musgrave, S., Rasbash, J., and Smith, A. F., (2000) Teaching Resources and Materials for Social Scientists Home page, Online: http://tramss.data-archive.ac.uk/index.asp (23/10/03). Wills, A. C., (2001) "Components and connectors: catalysis techniques for designing component infrastructures", In: Heinenman, G. T., and Councill, W. T., (eds.), Component-Based Software Engineering: Putting the Pieces Together. New York: Addison-Wesley, pp. 307-319.
This Page Intentionally Left Blank
Section B Creating Instruments for Ideation Software Approaches to Geovisualization Perspectives
5. Creating Instruments for Ideation: Software Approaches to Geovisualization Andrienko et al ........................................................................................................................................... 6. Statistical Data Exploration and Geographical Information Visualization Martin Theus .............................................................................................................................................. 7. Graph Drawing Techniques for Geographic Visualization Peter Rodgers .............................................................................................................................................. 8. Exploratory Visualization with Multiple Linked Views Jonathan C. Roberts .................................................................................................................................... 9. Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour David Mountain .......................................................................................................................................... 10. Impact of Data and Task Characteristics on Design of Spatio-Temporal Data Visualization Tools Natalia Andrienko, Gennady Andrienko & Peter Gatalsky ...................................................................... 11. Using Multi-agent Systems for GKD Process Tracking and Steering: The Land Use Change Explorer Monica Wachowicz, Xu Ying & Arend Ligtenberg ................................................................................. 12. Signature Exploration, a Means to Improve Comprehension and Choice within Complex Visualization Processes: Issues and Opportunities Penny Noy .................................................................................................................................................. 13. Facilitating Interaction for Geovisualization Jason Dykes ................................................................................................................................................
103 127 143 159 181
201 223
243 265
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 5
Creating Instruments for Ideation: Software Approaches to Geovisualization Gennady Andrienko & Natalia Andrienko, Fraunhofer AiS - Institute for Autonomous Intelligent Systems, Schloss Birlinghoven, Sankt-Augustin, D-53754, Germany Jason Dykes, David Mountain & Penny Noy, School of Informatics, City University, London EC1V 0HB, UK Mark Gahegan, GeoVISTA Center, Department of Geography, The Pennsylvania State University, University Park, PA 16802, USA Jonathan C. Roberts & Peter Rodgers, Computing Laboratory, University of Kent, Canterbury, Kent CT2 7NF, UK Martin Theus, Department of Computer-Oriented Statistics and Data Analysis, University of Augsburg, Universit~itsstr. 14, 86135 Augsburg, Germany
Keywords: geovisualization, tools, techniques, instruments, ideation, technology, task, data, user, expertise, interoperability
Abstract New visualization techniques are frequently demonstrated and much academic effort goes into the production of software tools to support visualization. Here, the authors of subsequent chapters in this section identify reasons why they continue to enhance and develop the instruments that they design to support the process of geovisualization, justifying their ongoing work and in doing so offering some perspectives on and solutions to the issues that they address. A number of inter-related themes arise including: advances in technology that create opportunities and generate demands for new geovisualization solutions; increasingly rich data sets and sources that drive design due to the associated potential for revealing new structures and relationships; various and novel tasks to which geovisualization is being applied associated with debate and continuing research concerning the kinds of instrument that are required to best undertake particular tasks in particular conditions; an increasingly diverse set of users who require a variety of tools, environments and systems to support ideation in its numerous forms, including those who participate in simulations of visualization when learning; changes in the available expertise that prompt the development of ideas and instruments that borrow from advances and methods in cognate disciplines such 103
104
G. Andrienko et al.
as Cognitive science, Statistics, Information Visualization, Knowledge Discovery and Datamining (KDD), Human-Computer Interaction and Scientific Visualization. Key issues in developing tools that address these requirements are those of interoperability and re-use. The potential advantages for geovisualizers and tool designers of common software approaches and the sharing of ideas are attractive. They allow us to proceed at speed, designing and adjusting our tools and interacting with revealing views of our data to address our objectives of creating superior instruments and enhancing our knowledge of spatio-temporal phenomena. Developing our geovisualization in this way will enable us to participate effectively in the processes of visualization and ideation and advance our science.
5.1
Introduction
During the last decade, numerous geovisualization tools have been developed by various individuals and organizations to support a variety of purposes, for example, (see Gahegan, this volume (Chapter 4)). Many of these have highly interactive and manipulable interfaces and yet we continue to build new tools using a range of technologies and techniques. Why do we invent new instruments? Or why do we need to invent new instruments? There are many possible motivations: 1. 2.
3.
4. 5. 6.
New technology continues to appear and it often enables us to do things that were not possible before. We may be able to acquire data of a new form or quality that cannot be analyzed with existing tools as the data sets may be so large, dense or contain so many dimensions that no current tool supports interactive investigation effectively. As geovisualization becomes more popular and exploited more widely, we encounter new tasks that cannot be performed using existing tools. Effectively geovisualization may be used to address new societal requirements. The particular needs of specific users (from this growing user base) are likely to vary and tools may serve a new or changing user base. Accessing expertise from cognate disciplines may contribute to what already exists and enhance it further. Collaboration between researchers may improve our ability to visualize geographic information and to develop the various instruments that support this process. The notion of interoperability underlies our efforts to develop ideas and generate knowledge from our data using instruments for ideation.
Ideation relates to the formation of ideas and concepts, the end goal of much geovisualization. Today there are many tools and techniques for creating instruments for ideation - sophisticated hardware, advanced programming languages, graphics libraries, visual programming systems and complex GUIs. In each, the developer or visualizer wishes to generate effective interactive graphic realizations of their data that are useful to them and/or their users. Indeed, these geovisualizers will come across many challenges. We suggest six factors that help explain why new tools and techniques may be developed.
Creating Instruments for Ideation: Software Approaches to Geovisualization
In this chapter, we influence and shape exploratory process. chapters that follow
5.2
105
expand upon these ideas and consider how each of these issues the way we use and develop software instruments that support the Some examples of our software approaches are documented in the in this section.
Technology Issues
Current technology has an important enabling and limiting impact upon the available range of instruments for ideation, which changes significantly over time (Cartwright et al., 2001; Fairbairn et al., 2001). In 1854, Dr. John Snow and colleagues manually plotted the locations of deaths from cholera on a paper map in order to show the outspread of the disease (Tufte, 1997; Brodlie et al., 2000). Today, Snow would most likely load these data into a GIS and look at a map display on a computer screen. In 1967, Bertin (Bertin, 1983) described advanced equipment for manipulation of data represented on paper strips or cards. Current computers obviously offer far more sophisticated opportunities for manipulating data representations and interacting with them. It is natural that builders of tools for ideation, either for their own use or for use by others, aim to utilize the most applicable capabilities of the current technologies. A major benefit of contemporary computer technology is the possibility to rapidly generate various graphical displays from data. This gives us an opportunity to try alternative transient realizations of data, discard those deemed ineffectual but when necessary reproduce them again, and look at several displays simultaneously to provide multiple views of data (McCormick et al., 1987; Becker et al., 1987; Stuetzle, 1988; Roberts, 2000). Such techniques are paramount to the process of geovisualization and enable us to address the data, task and user issues that also drive our pursuit of innovative and effective instruments. They have stimulated much of the work reported in this section. We also use advances in technology to store large amounts of data and access them on demand with minimum effort. Moreover, it is increasingly easy and common to obtain additional data when necessary in an analysis, from a range of sources including the Web or from the wireless "information everywhere" devices. The speed of computation that technology now allows enables us to combine visualization with computationally intensive methods of data analysis such as exploratory statistics or datamining: one can rapidly obtain the results of computation and compare them with what is observed or interpret them through depiction on a map or a graph. Of course, interaction and display manipulation play crucial roles in data exploration (MacEachren, 1994a-c). The speed of data access and display generation also allows the development of dynamic representations capable of changing in real time. Thus, dynamic and animated maps are now widespread and offer exciting opportunities for representing geographic data. 3D graphical representations can be realized with relative ease, and are frequently used, for example, (see Wood et al., this volume (Chapter 14)), yet we are only beginning to determine ways to utilize their potency most effectively for data analysis and ideation by developing and testing tools, for example, (see Kirschenbauer, this volume (Chapter 18) and Coors et al., this volume (Chapter 27)).
106
G. Andrienko et al.
It is not only the increase of computer power that offers us new opportunities to create more sophisticated instruments, but also the progress in software environments, such as the development of programming tools that are high level and/or cross platform and the availability of libraries and reusable software components. Examples include the MacroMedia products, Java, Tcl/Tk, the OpenGL libraries and the OpenSource paradigm. These environments and related methods of development have capabilities for visualization and enable us to produce flexible applications that are quickly extensible and may run on a variety of platforms. As a result, more sophisticated and dynamic graphics can be created and widely used. Besides increasing the opportunities, technical progress raises demands and poses challenges that drive our research and the development and testing of our instruments. New types and volumes of data made possible by technological advances may need handling in innovative ways and so require the development of geovisualization techniques (see w The Web not only enables us to obtain more data when needed but also raises issues relating to data merging, data representation, and the analysis of complex, heterogeneous data sets. Moreover, the Internet and other communications technologies such as mobile devices are changing the way people are working. Indeed, mobile devices do not only enable people to compute anywhere and to access data from everywhere but also create a demand for new instruments that are designed specifically for such devices and which can be conveniently and effectively used. In the design of these instruments, one must creatively manage limitations that once appeared to be issues of the past: small screens with low resolution, memory restrictions, limited computational power and reduced possibilities for interaction. The lucrative games industry is beginning to discover the need and market for device specific applications, and work is in progress to take advantage of opportunities for mobile geovisualization, for example, (see Coors et al., this volume (Chapter 27)). Technical progress enables people situated at different physical locations to work cooperatively, hence, there is a demand for geovisualization tools that support this mode of remote collaboration, for example, (see Brodlie et al., this volume (Chapter 21)), which in itself leads to new geovisualization tasks (see w The availability of computers, data, and the Internet encourages more people to do their own exploration and take part in knowledge construction. This calls for the creation of widely accessible instruments (including those available over the Web) that can be operated by users with different levels of sophistication and computer competence. The availability of such tools confronts designers with further challenges. How are such accessible prompts to geovisualization made understandable to a wide and usually unknown audience? Such issues are further explored in w Whilst being convenient for tool developers, modern software consumes far more computer resources than before. In some cases (for example, using Java), it can be argued that the costs of high-level flexible tools are significantly slower software applications than older alternatives written in Fortran, C or C + +. Whilst this issue can be addressed by improvements in processor speeds it is exacerbated when coupled with the "mobile device considerations" introduced above. Code reuse and exchange of components also poses numerous difficulties, some of which are discussed in w below.
Creating Instruments for Ideation: Software Approaches to Geovisualization
107
Hence, we see that technological progress has tremendous influence on the development of instruments for ideation and explains much of the rationale for our work. This influence is two-sided. On the one hand, progress creates new opportunities that may be explored and utilized. On the other hand, it creates new demands that need to be satisfied. The new demands often come together with particular restrictions and considerations that have to be appreciated. In keeping up with new technologies, geovisualizers should not disregard earlier ideas, approaches, and methods. After all, as we have seen, even the idea of manipulating data displays first appeared in the precomputer era. In the current absence of formal theory to guide or prescribe our use of the new opportunities that technology offers to us, existing knowledge and the methods of the present and past provide important resources on which to build our knowledge and techniques (see also w
5.3
Data
Motivation to develop novel graphic representations and interaction techniques can come from the data itself and our abilities to access and use it (Gahegan et al., 2001). Besides increasing the opportunities, technical progress raises demands and poses challenges that drive our research and the development and testing of our instruments. Continuing technological advances have had a profound effect upon the nature and volumes of data available to the geovisualization community. The current ease with which data are recorded and acquired results in huge data volumes that cannot be effectively explored using standard methods of representation and interaction. New data sets are being generated by such diverse applications as mobile communications technology, digital commercial transactions, Web-logging software, traffic monitoring systems, closed circuit television, various flavours of GPS "tagging" and countless others. These data sets often contain spatial and temporal referencing, whether stored explicitly or implicitly, and may benefit from either the development of new visualization techniques or the adoption of techniques from one domain by another. Consequently, a wide range of new media (such as imagery, animations and audio) is increasingly being used, offering the investigator a variety of data models. Encouraging the user to interact with different media can add a qualitative perspective, for instance by representing a phenomenon in context, that quantitative approaches alone may not be able to offer (Shifter, 1995 a,b). The data structure used to represent the phenomena of interest has implications for which visualization techniques are available and appropriate, for example, (see Keim et al., this volume (Chapter 2)). Many structures are static, representing the situation at a single moment or with no reference to time at all. Relationships can be uncovered between different parameters through techniques such as interactive statistical data exploration, but these cannot reveal dynamic processes. Data structures that incorporate time lend themselves to techniques that can represent these dynamic processes. Animation is one approach to showing time elapsing and works well for presenting information about entities evolving under the influence of long-term trends. The progression of urban form is an example (Acevedo and Masuoka, 1997). A current
108
G. Andrienko et al.
challenge lies in enhancing animation tools that simply present patterns to interactive techniques where user-controlled animation allows variable selection and control over a dynamic display to compare the behaviour of different entities over the same time period (Andrienko and Andrienko, 1999a-e; Rana and Dykes, 2003; see Andrienko et al., this volume (Chapter 10)). Time can also be represented without the need for animation. In particular, spatial plots can be combined with temporal plots to allow simultaneous interaction with both space and time. Such approaches have a long history, and include another much cited graphic that pre-dates computer supported visualization, Minard's depiction of Napoleon's Russian campaign (Tufte, 1983), but have now evolved into interactive approaches allowing selection for comparison and focus (Peuquet and Kraak, 2002; Andrienko et al., 2003). Whether a discrete or continuous data structure has been used also has implications for the visualization techniques that are applicable. Discrete approaches are more appropriate for modelling and analysis of entities, such as the movement of individuals or groups. Continuous structures, such as elevation models and point density surfaces, are more appropriate for representing the wider trends over an area: representation of and interaction with these models is quite distinct to discrete approaches. This continuous sampling strategy can generate very large datasets, often only a small subset of which is relevant to the task being undertaken by a particular user. Abstracting the relevant information from the continuous model at the appropriate scale can reduce data volume and prevent the "information overload" associated with very large datasets (Rana and Dykes, 2001). Moreover, there is a need to store these large structured volumes of data in appropriate databases such that the information is readily available and integrated with appropriate mining and visualization tools (Gahegan et al., 2001). As indicated above, the automated collection of data has become increasingly commonplace leading to a massive increase in the volumes of data that are generated. Such datasets may contain much richer information that can be revealed through most current visualization techniques, requiring the development of new methods and instruments that draw upon them. Various methods have been applied to address this particular problem. Classification by aggregating a collection of samples into a single entity, or by using pruning methods for hierarchical data, can allow large data volumes to be represented more clearly (Kumar et al., 1997). However, these aggregate entities may have different characteristics to the individual entities they represent and a dual approach may have to be adopted if the same interaction technique is to be implemented for source and aggregated data. Under these circumstances, the experience that geo-scientists have in dealing with issues of scale and scale-dependent phenomena, and their visualization could be beneficial to a wider community (for example, see Wood, this volume (Chapter 15)). Classification can also facilitate or even permit interaction with large data volumes. When interacting with a very large number of graphic elements in a display, processing time can otherwise prevent a technique from supporting the real-time interaction required for visualization (see Theus, this volume (Chapter 6)). Here, we see a clear association between the data and technology considerations that influence the instruments that we develop and the techniques that we use.
Creating Instruments for Ideation: Software Approaches to Geovisualization
109
As is the case on paper, on-screen realizations of large data sets can cause particular problems for the viewer in detecting and discerning the symbols used, assembling them into patterned regularities and estimating the magnitudes that they represent (Cleveland, 1993). Interesting trends may be lost through over plotting and the magnitude and distribution of outliers. Various graphic and software techniques can be implemented (Theus, this volume (Chapter 6)). One approach is to summarize key trends: for example GPS track logs representing individual movement can be used to generate point-density surfaces giving a representation of where an individual spends their time and their familiarity with specific locations from which information can be extracted (Dykes and Mountain, 2003; Mountain, this volume (Chapter 9)). arunselvaminThe phenomena that geovisualizers usually study exist in a spacetime framework. Static 2D maps that offer an impression of a complex 3D space at a particular time rely upon dimension reduction. Much cartographic effort has always gone into minimizing the impact of reduction upon the anticipated tasks to which the map is put. However, geovisualization techniques can be applied to data that have no spatial or temporal dimensions. A 2D surface can be generated by principle components analysis and geovisualization interaction techniques applied to it. Similarly, clusters in selforganizing maps are described in terms of distance, however none of the original dimensions that led to the map need relate to spatial metrics (for example, see Koua and Kraak, this volume (Chapter 33)). The results of techniques that use this spatialization (Fabrikant, 2000a,b) aim to utilize human abilities to interpret landform and interacting with non-spatial data in this way may offer new insight (for example, see Fabrikant and Skupin, this volume (Chapter 35)). Thus, a clear opportunity exists to explore the utility of such techniques and develop new instruments of ideation specifically for spatialization. Each graphic depiction of a data model that we develop is one of an infinite number of alternative views. Individuals developing geovisualization techniques are often attempting to display a single specific relationship at the expense of other trends and background noise. Successive transitory screen realizations designed for private "idea chasing" are not intended to communicate a single message about the data to a wide audience as has been the case with many traditional maps. The increasingly rich and potentially revealing data sets becoming available are leading to an increasing number of abstract, novel and interactive graphic realizations of data sets being developed by those creating instruments for ideation.
5.4
Tasks
Whilst technological advances and the availability of new data drive a need for new views, Casner (1991) convincingly demonstrates that the same data need to be represented in different ways using different views in order to effectively serve different information needs. The information needs that require these views are defined by different tasks. Casner considers examples of tasks such as planning a journey from city A to city B with a stopover in city C (where one has an appointment for a particular time) and finding the cheapest flight or most direct travel route. For each task, he proposes a graphical display that allows it to be effectively fulfilled. There is no "ideal" graphic
110
G. Andrienko et al.
realization of the data suited for all purposes. Indeed, the tasks that Casner considers have little to do with exploration, namely there is no attempt to grasp inherent characteristics of unfamiliar data and gain knowledge about the underlying phenomena. However, there may well be generic tasks in data exploration that define geovisualization. Has a data explorer any particular goal in mind while looking at some unfamiliar dataset from different perspectives in a hope to gain serendipitous insights into the information? And if so, is knowledge about possible tasks an important factor when designing instruments for ideation? If Casner's supposition is applicable to exploratory map use it would account for the variety of approaches employed in geovisualization, and as tasks change new instruments need to be developed to support them. Designers and developers of geovisualization tools, including those who have contributed to this chapter and the discussions surrounding its development, may have different approaches to considering tasks. Some believe that having a defined task is not (or not always) necessary in Information Visualization. Others are convinced that tasks always exist, explicitly or implicitly, even when an explorer seems "just to look" at data. Advocates of task-driven tool design argue that usually an explorer does not only look at data but also looks for something "interesting", such as a configuration that may contribute to a better understanding of the data or underlying phenomena. This may be, for instance, a salient pattern of spatial distribution, a local anomaly, some indication of unusual behaviour or an indication of a possible dependency between phenomena or processes. In order to find these "interesting" things, an explorer actually performs a range of exploratory tasks (possibly, without even realizing this): observing a spatial distribution, attempting to detect patterns and anomalies, looking for possible relationships, and so on. These tasks are, of course, very different from those contemplated by Casner. Apart from being less precise and more general, exploratory tasks are often fulfilled in parallel or combination (Gahegan et al., 2001). In the case of Casner's examples, a single (often more straightforward and well specified) task is considered in isolation. While an explorer may not be aware of their tasks or know the specific outcomes, those designing tools that support ideation must consider them explicitly and deliberately design any instrument so that it can assist in the observation of distributions and behaviours, expose patterns and facilitate detection of relationships. Taking into account the concurrency of exploratory tasks it may be inappropriate to follow the approach of building separate graphical realizations for each task. Instead, one should try to design to support a range of tasks, possibly through various methods of interaction. When a single graphical depiction appears to be insufficient, several interlinked complementary realizations of the data may be appropriate. Whichever combination of graphics and exploratory environment is used to support ideation, applying Casner's model to the realm of geovisualization explains the proliferation and development of instruments and suggests that a tool designer needs to know which exploratory tasks exist and to find methods of supporting them in order to promote ideation. Gahegan, this volume (Chapter 4) identifies outcomes of ideation when considering the nature of the research process and suggests a framework for determining the utility of tools and techniques according to a low-level analysis of tasks.
Creating Instruments for Ideation: Software Approaches to Geovisualization
111
Various approaches to defining possible tasks have been developed and numerous alternative task taxonomies suggested (Knapp, 1995; Qian et al., 1997; Shrager and Langley, 1990; Shneiderman, 1998) and Plaisant, this volume (Chapter 3) provides a simple taxonomy of seven "basic tasks". Despite their differences, these taxonomies exhibit some commonality. They are typically built from genetic tasks like "identify", "locate", "compare", or "associate". Such broad task categories cannot help a tool designer much in isolation unless they are explicitly related to data and refined in terms of the nature of the data to which they relate. Thus, it is hardly possible to create a tool supporting the abstract task "compare". Instead, one needs to determine which specific comparison tasks exist for a particular dataset or for a range of datasets with similar structures. This demonstrates another interaction between our identified motivating factors, a clear relationship between data and task. Let us assume, for example, that we need to explore data about objects that move in both time and space. We can instantiate the generic "compare" or "relate" into several more specific tasks: 9 9 9 9 9 9 9 9
compare positions of two or more objects at a particular moment in time; compare positions of an object at different moments in time; compare trajectories of different objects; compare trajectories made by the same object during different time intervals; compare the speed of movement of different objects during the same time interval; compare the speed of movement of the same object during different time intervals; compare distances travelled; ... and so on.
These tasks are obviously different and need to be supported in different ways. Hence, specialization of generic tasks in terms of data components is essential for the successful design and use of tools for ideation. An ideal task taxonomy for a tool designer would be the one that allows apparent and straightforward specialization. Object orientation may be an appropriate methodology for developing tools that utilize such a taxonomy (Boukhelifa et al., 2003). Once we have understood which tasks (potentially) exist, we need to find ways of supporting them. Unfortunately, no appropriate theory or guidelines currently exist, upon which we can rely. Certain empirically derived pieces of knowledge are useful however. For example, choropleth maps may be appropriate for detecting spatial patterns in certain circumstances but are inappropriate for comparing objects (Jung, 1995). Although we primarily deal with geographic information in this chapter, we should not restrict ourselves only to cartographic representations but adapt tools and approaches from different disciplines to our purposes (see w For instance, a scatter plot is good for detecting correlations between phenomena characterized by numeric attributes, and a time-series plot can be effectively used for exploring spatio-temporal data. We should also remember that we have the possibility to enhance our displays by facilities to interact
112
G. Andrienko et al.
with them and manipulate them, and this may radically change their properties so that choropleth maps, as in the example above, can be made more suitable for comparative tasks (Andrienko and Andrienko, 1999d, e). Moreover, interaction can also help us to link together multiple displays serving different tasks. Although we reuse the experience accumulated in the geovisualization area as well as in Information Visualization and statistics, in many cases it is not obvious what kind of instrument is needed for a specific task. In such cases, we may try different tools, either from the existing set or by devising new realizations ad hoc to suit, until we find one that satisfies the particular task in hand. For example, when using geovisualization to explore data about earthquake occurrences using an interactive animated map display, one may find that it does not facilitate the detection of spatio-temporal clusters. In trying to explain this failure, it is possible to hypothesize that the task of cluster detection requires the spatial and temporal dimensions of the data to be viewed simultaneously and in a uniform way. This consideration may lead to the idea of using the "space-time cube" where time is represented by an additional spatial dimension (Mountain, this volume (Chapter 9)). As a result, knowledge about both the process and the nature of the use of graphics is improved. This example demonstrates once again that data and tasks are interlinked factors that shape the design and use of our geovisualization instruments. And the low-level tasks to which geovisualization may be applied are changing as a result of other factors discussed in this chapter such as advances in technology, changes in the amounts and types of data that are available and changes in the type of user who make use of geovisualization.
5.5
Users
Whilst data and tasks shape the nature of geovisualization techniques and tools, the degree to which they are appropriate will ultimately depend upon those who use them. Users have a host of expectations, individual experiences, skills, domain specific knowledge and various capabilities and limitations. Importantly, they are the people who expect to gain knowledge and understanding through their interactions! Creating effective instruments for geovisualization requires us to design tools with which the target user(s) can efficiently and effectively interact (Fuhrmann et al., this volume (Chapter 28)). Thus, it is important to consider who is doing the visualization: not only does the user come with a package of skills, but they also have varying degrees of domain knowledge. For instance, a geologist may have refined skills in interpreting 3D representations, due to their experience with solid models and volumetric concepts (Gahegan, 1998). However, users without these skills may wish to access and interpret the data using alternative realizations. The background and skills of particular users have a considerable influence on the way in which instruments are used and their effectiveness. For example, an individual who foresees that they are going to use a system many times may invest a substantial amount of time and effort in learning it. However, an occasional user may wish to get some results quickly. Moreover, the need to extend our domain knowledge and that of
Creating Instruments for Ideation: Software Approaches to Geovisualization
113
geovisualization may require a geovisualizer to simply create unique graphics rapidly or demonstrate a particular aspect of their work. To minimize development time and expedite the generation of ideas resulting from successive interactive graphics they would usually prefer to build their hand crafted exploratory instruments from pieces of existing functionality that offer flexibility and efficiency. Thus interoperability issues become important (see w Users with specific domain knowledge may drive system development. It is often the case that these domain experts end up developing appropriate tools themselves. Thus, toolkits need to be built that enable the user to quickly build a prototype system from a series of available techniques. This may draw upon many methods of interoperability, from a quick and easy graphical style using a dataflow model of interconnectivity, to a very flexible but complex set of package interactions, relying on knowledge of code and data formats, which is more sophisticated and powerful than a simple model of interconnectivity, but may be harder to use. Some of the various possible approaches towards interoperability are discussed further in w Other users may have different expectations of a particular tool, perhaps depending on their level of expertise, experience, or time available to learn and use the software. One expert user may be more patient with a slow system, waiting for a particular realization to be rendered just to get that high quality and accurate display, while another may wish to generate quick representations that are not necessarily exact. When following Shneiderman's mantra "Overview First Then Details on Demand" (Shneiderman, 1996), the user often wishes to simply provide an overview followed by a process of finding out more about the representation. In w we identified that different tasks in this process require different solutions. But equally defining what an effective overview may consist of is another issue and the answer is likely to be user dependent. Is it, for example, a simple scatter plot showing the statistical relationship between two attributes for all cases in one display (with the resulting problems of interactivity and possible overplotting), or is it a higher level abstraction with symbols depicting the centres of gravity of the major clusters that appear on the plot (and so the problem becomes one of finding an algorithm to effectively abstract the groupings)? Or is it a map, or a set of maps, showing the spatial distribution of one or more of the attributes? Different users may have different answers and each will certainly want to subsequently zoom, pan, filter, and request details on demand in different ways. Personal abilities are one of the characteristics that affect a user's preferences in exploring information. They include physical constraints (e.g., sensory and cognitive limitations) as well as a variety of academic abilities (e.g., levels of reading, writing, arithmetic and interpretive skills). These all may have an effect - to a greater or lesser degree - on the level of interaction required by the user and to the appropriateness of any particular combination of realization and interactivity. The area of unseen disabilities, especially in the case of arithmetic skills, can be considered as one of "difference" (Slocum et al., 2001) and includes diverse styles such as in task-action choices and in general spatial awareness skills. In the field of human computer interaction (HCI), the issue of task-action consistency suggests that it is advisable
114
G. Andrienko et al.
to have multiple task-action methods as different people will regard different tasks as being variously similar or dissimilar (Grudin, 1989). Even though we follow best design practices, individual users are still different and have varying abilities. Indeed, it is these differences that make it difficult to fully understand how users are going to use a system, and often user trials reveal different methods of interaction from those that were expected. As is the case with different tasks, where multiple views may be required, different users are likely to require multiple visualization methods to engender understanding most successfully. Users and tasks go hand in hand. Complex tasks usually require non-trivial interfaces. Confining ourselves to limited modes of interaction and/or representations might restrict developers and investigators. For example, complex multi-variate views such as parallel coordinates and mosaic plots may involve some training to be used effectively by a mass audience. Indeed education is an important issue as we hope that insight is extended as yesterday's new view becomes tomorrow's bread and butter graphic device. Developers should not shy away from producing complex systems that may require users to invest time and effort into learning how to use them effectively - if they are effective. After all in the 1700s, Playfair's scatter plots were unlikely to have been greeted by mass approval and widespread comprehension. Moreover, cooperative work by multiple users on data exploration and analysis requires specific support. A strong impact on tool design is whether the intention is for use by an individual or a group (for example, see Brodlie et al., this volume (Chapter 21)). Developing instruments that provide effective solutions to the latter of these scenarios under particular circumstances is a key objective. Thus, different kinds of users will require and work most effectively with different instruments to support their geovisualization, and exploring the possibilities is a key motivating factor for those involved in tool specification and development. For example, applications software that is to be used widely should be stable, reliable and have a consistent look-and-feel (see Plaisant, this volume (Chapter 3)). Other systems offering additional flexibility (such as those that make use of many linked views and complex graphical representations) may be more appropriate for exploratory work undertaken by experienced and skilled users, rather than tools used in a limited context such as education. To summarize our arguments, we can say that users are diverse and so a variety of tools, environments and systems are appropriate to support ideation in various contexts. When faced with developing software for wide use, the application of multiple linked views allows a variety of user skills and abilities and background systems to be supported. In addition, a non-trivial solution with sophisticated interaction methods that is geared expert use and may offer considerable flexibility has an important role, as do a more limited and less polished or robust tools that are developed in order to demonstrate or test particular techniques. Indeed some tools may offer a variety of levels of complexity of functionality and interface through "multi-layered designs" (see Plaisant, this volume (Chapter 3)) or by developing smooth links between modes of interaction (for example, see Dykes, this volume (Chapter 13)).
Creating Instruments for Ideation: Software Approaches to Geovisualization
115
Expertise from Different Fields
5.6
Finding an appropriate representation technique using suitable methods of interaction is a common challenge in any discipline that makes use of graphics. Geovisualizers can thus draw beneficially from other communities, and many of the techniques and much of the work cited thus far in this chapter does so. This may in turn lead to the need for a wide range of skills. The complexity and specialist nature of the tools and techniques available in other disciplines may prohibit their use in certain circumstances. Collaboration and the delegation of roles may allow more work to be achieved more effectively. Indeed, the 1987 special issue of the Journal of Computer Graphics concerning Visualization in Scientific Computing (McCormick et al., 1987) was enthusiastic about the benefits of having interdisciplinary teams when producing effective visualization tools. There is a good argument for teams composed of a diverse group of investigators with a variety of expertise. The McCormick report describes a group combined of: 1. 2. 3. 4. 5.
specialists with knowledge and skills relating to the target domain; visualization scientists with software, hardware, networking, languages, operating systems and database skills; support personnel with skills to "configure and maintain visualization facilities"; artists with specialist knowledge in composition, lighting and color; cognitive scientists.
Thus, bringing people together often benefits the research (a philosophy of two - or more - heads being better than one). More recently, the US National Research Council on "Geospatial Information and Information Technology" embraces this interdisciplinary perspective (Computer Science and Telecommunications Board, 2003). Many tool development projects do include scientists with geographic and computing expertise, but it is rare that a geovisualization project includes investigators with a more diverse range of skills. Conversely, more diversity may not in fact be better for the project as managerial problems surface: for example, as any software engineer knows - getting more programmers on a project, that is already running late, in fact slows down the project as a whole (Brooks, 1995). This is a difficult equilibrium to resolve and maintain, but one that must be addressed. Experiences of managing large collaborative projects in the open source community may offer some solutions and the opportunity to extend the threshold beyond which the benefits of collaboration between individuals from different domains are offset by the costs of coordination. Technological advances that permit and support collaborative work should prove to be valuable (Brodlie et al., this volume (Chapter 21)). Another challenge with such a diverse group of experts is that the primary focus for each individual might be quite different. Geovisualizers often have a particular graphical representation of a geographic scenario as a primary focus, while Information Visualization focuses on attempts to graphically depict structures that are abstract and have no physical location or space equivalence. Conversely, statisticians may tend to
116
G. Andrienko et al.
depict relationships within sampled data, which can give new insight to the generating process, whereas Information Visualization faces the most general challenge in not being restricted to any specific domain. Despite these differences, increasing overlap exists (as indicated in many of the chapters of this book) and evidently geovisualization can gain much from the diverse knowledge available in cognate disciplines. This may prompt the development of new tools and techniques. Geovisualization and various other related fields are complementary in a number of ways. In the case of geovisualization and statistics, the two disciplines can be considered as relating to different stages of the process of seeking understanding of a dataset. Geovisualization has tended to focus on and support data exploration occurring in the earliest stage of GI scientific endeavour, with an objective of generating plausible hypotheses concerning inherent data characteristics and relationships. Traditionally, statistics becomes particularly appropriate when the hypotheses need to be validated and models built. Gahegan (see Chapter 4) argues strongly that geovisualization is a broader process that supports the entire practice of GIScience and (graphical) exploratory statistics has provided much impetus for geovisualization. Statistical data, here considered to be the characteristics of data that do not describe geographical location or properties, is usually multi-variate. Thus, the structures to be identified are interactions between numerous variables. These configurations are often far too complex to be captured by simple coefficients, and call for graphical exploration or complex statistical models. Linked highlighting and plots for highdimensional data are the building blocks of interactive statistical graphics, and all (human-computer interactions) must be built to support these tasks and plots. "Statistical thinking", that is knowledge about distributions and relationships between them, is very important in geovisualization when participating in exploratory data analysis (EDA). A different relationship exists between geovisualization and knowledge discovery in databases. Like geovisualization, KDD techniques are also meant for revealing significant characteristics and relationships in unfamiliar datasets. KDD is defined as a non-trivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data (Fayyad et al., 1996a,b). This is usually applied to very large datasets. Whilst geovisualization focuses on knowledge extraction undertaken by a human analyst and the interactive visual tools designed to support them, KDD offers techniques for the automatic extraction of knowledge from data. Combination of these two approaches may exploit the strength of each of them and compensate for limitations (Andrienko and Andrienko, 1999b, c; Andrienko et al., 2001; Keim and Kriegel, 1996; MacEachren et al., 1999). Thus, the "human eye" can perceive and process some spatial relationships from a visual representation such as a map while a computer draws upon the relationships encoded in digital spatial information. Yet only limited characteristics of the various aspects of spatio-temporal arrangement that may be potentially relevant can currently be represented in a format suitable for datamining. Examples include, distances between centroids or neighborhood relationships. Results of datamining, in turn, require a human analyst to evaluate and interpret them and, hence, need to be appropriately visualized. On the other hand, computers are superior to humans in processing large volumes of data, and this advantage is exploited in exploratory tools
Creating Instruments for Ideation: Software Approaches to Geovisualization
117
that combine geovisualization with KDD techniques and there is an opportunity for further developments in this area (see Wachowicz et al., this volume (Chapter 11)). Indeed, examples also exist whereby artificial intelligence techniques are incorporated in geovisualization tools (Andrienko and Andrienko, 1999c-e). Such agents may be used to identify items of interest or to help users to select and generate appropriate graphical realizations. Sometimes new representation and interaction forms are brought to geovisualization from other disciplines, for example, direct manipulation from HCI or graph drawing techniques (Rodgers, this volume (Chapter 7)). Scientific visualization is another field that can benefit geovisualization. It is best at representing an object or a single phenomenon or process. Understanding the physical structure of the problem and finding the optimal way to render it are the main tasks. Thus, scientific visualization instruments are often unique to a particular problem. A usual interaction is to navigate through different (virtual) views of the object or phenomenon of interest. Since geographic inquiry tends to involve real world phenomena that have changing locations in 3D space and time, geovisualization can borrow from scientific visualization realization/rendering expertise (Wood et al., this volume (Chapter 14)). This includes the development and testing of hardware-based advances (see D611ner, this volume (Chapter 16); Kirschenbauer, this volume (Chapter 18)) and those that develop software solutions to support interaction, often across multiple linked views (for example, Lopes and Brodlie, this volume (Chapter 14); Roberts, this volume (Chapter 8)). The common theme of all disciplines concerned with visualization is the exploration of a scenario (problem, phenomenon, data or object). By interacting closely with graphical representations, the user may better understand the data, gain information from it and so acquire knowledge of the phenomenon under study. Although there may be differences in focus, skills, ideals or terminology, interdisciplinary projects should be encouraged and the creation of instruments that borrow from advances and methods in a variety of related domains offers a clear rationale for the range of approaches to creating instruments for geovisualization and continued development.
5.7
Interoperability
The increasing requirement to integrate expertise with specialist knowledge and skills, and the call for user, data and task-specific solutions, indicate a need for systems to be interoperable. Indeed, Geography itself can be viewed as the integration of perspectives, which is reflected in our constant requirement to reach beyond ourselves to find new instruments to address emerging problems. Consequently, some consider the whole issue of open systems for geovisualization to be a long-term major concern: that analyses should not be limited to tools developed by individuals (see Gahegan, this volume (Chapter 4)). One way to achieve this goal is to reduce or remove the need for any prior agreement concerning the way in which major parts of a system (sometimes termed "components") interact. Additionally, no single individual has the resources needed to develop the ultimate geovisualization system that can accommodate the various tasks, users, data and other factors that we have considered here. But effective systems could be built
118
G. Andrienko et al.
Figure 5.1. Levels of inter-operability, with the current position in terms of the status of solutions to each of the aspects of interoperability schematically represented.
efficiently, if ways could be found to integrate the wide range of tools available, both within the geovisualization speciality groups and in the wider community of visualization researchers, while maintaining flexibility and extensibility. That is not to say that great things cannot be achieved using existing approaches, but rather this is a call to look forward to the next major challenge - one that must be faced collectively for progress to be made. There are many dimensions to the problem of interoperating visualization tools, and some of these are depicted graphically here. Figure 5.1 illustrates the various levels of abstraction that comprise all facets of the term "interoperability", from the basic connections of machines on a shared network, through the sharing of data, to the fusion of perspectives. At this time, perhaps our most pressing problem is that of consistency of the "world view". This is a concept that covers all assumptions made in relation to semantics, semiotics, user interactions and so on. It is shown as the combination of "interface" and "application and action ontologies" in Figure 5.1. Different user communities and tool developers have different ontological commitments (such as different world views) and different ideas about how one component might interact with another. These can be thought of as types of implied application ontologies (the data models embodied by the tools) and action ontologies (which particular actions are supported, such as linking and brushing). More precisely, the application ontology is the specific way that an application task is conceptualized (in this case by a component or a system). Ideally, an application ontology is shared between components that are collaborating. The action ontology comprises the various actions that are defined by different systems or tools that need to be mapped to other systems and tools, such as linking, brushing, sampling or grouping. While these ontological differences may be small, say within a distinct laboratory (though they need not be) they may be considerable when considering wider
Creating Instruments for Ideation: Software Approaches to Geovisualization
119
communities of researchers discussed in w This will be especially so if their work is motivated by completely different problems or perspectives. This begs the question - to what extent are we able to borrow tools? For example, does the inclusion of spatial data dictate that distinct tools are needed? These questions relate to those that concern some visualization researchers: to what extent can generic techniques and tools meet a variety of general goals? At the moment, we tend to build systems that contain our own ontologies explicitly, hard-coded. We need to either find a way of decoupling graphical functionality from our ontology, and coding it separately, or of re-purposing or wrapping existing ontologies so that we can manipulate them to our own purpose. These issues are studied by a community of researchers under the heading of "problem solving environments", and are well understood, though good solutions are largely elusive (Schuchardt et al., 2001). The sharing of ontologies only solves the conceptual models used and the way in which components interact with various events, by themselves these problems do not guarantee that the combined systems appear to be logically integrated or to have any kind of consistency as far as the user is concerned. Parallel efforts are also required to solve user-interface issues, so that the components assembled appear to be logically consistent in use. For example, it would be desirable for components to use the same look and feel, the same layout guidelines and the same data interfaces. Again this involves decoupling of logical design from user-interface design, and coordination of layout issues among components, preferably at runtime, for the maximum flexibility. Here, interoperability issues overlap with those of usability.
(a)
method of combination
data exchange
(b) level of integration (after Rhyne, 1997)
9
rudimentary
minimal data sharing
(c) ontological agreement of components
data exchange
m, O O
operational
9
consistency of data input/output binding
functional
9
transparent communication
application (data models) action (interaction)
m. (.Q
o 0
o 0 --h
0
libraries of components
merged
9
one comprehensive toolkit
presentation
Figure 5.2. Methods of interaction between components for visualization: (a) method of combination; (b) levels of integration (Rhyne, 1997); (c) level of ontological agreement of components. Note that Rhyne's model was designed specifically for geographic information and scientific visualization systems.
120
G. Andrienko et al.
Rhyne (1997) suggests a plan of interaction that provides a systematic framework to describe this problem - four levels of interaction: rudimentary, operational, functional and merged. These levels are shown in the centre portion of Figure 5.2, alongside representations of the two other integration themes just discussed means of combining tools and level of world view consistency. Rhyne's levels relate specifically to the integration of geographic information and scientific visualization systems, but the concepts are relevant to our more general focus: 1.
2. 3.
4.
The rudimentary interaction level uses the minimum amount of data sharing and exchange. A significant obstacle is the situation where data are the same, but formats different. This requires expansion and adoption of standards for data exchange. Simply encouraging data exchange can however result in advances. A number of the current authors have shared data and applied their own different instruments to address various geovisualization issues and develop unique solutions. The operational level attempts to provide consistency of operation, consistency of data, removal of redundancies and inconsistencies between technologies. The functional level provides transparent communication between the components. Transparent communication implies that they can understand each other, without the use of a third party. The merged level describes toolkits built from the ground up.
Note that the merged level is not necessarily the desired point to be achieved by all systems. It may be inappropriate or infeasible because, for example, the interaction and presentation aspects may be too tightly coupled to be consistent between different systems that it is desired to integrate. So, we might have current systems where we can interoperate across networks, platforms, even share and exchange components, but we cannot make either their ontologies (application and action) or their interfaces agree. Above that we have absolutely no idea about their underlying philosophies since they are not represented explicitly anywhere in the systems. This position is illustrated schematically in Figure 5.1. Whilst various levels of integration of ideas and software components are possible, lessons from attempts to produce common components and standards in Computer Science indicate patchy success. In particular, difficulties arise because of the time taken to decide standards and the increased effort required by developers to meet the requirements. In the worst case, the effectiveness of the component produced can be severely compromised, because the standard does not implement required operations or forces inappropriate behaviour. As technology and perceived requirements change, so may the standards, invalidating legacy components. The evolving open source community offers a developing methodology and considerable experience in managing collaboration from which we can learn. The influence of powerful commercial organizations is also an important issue here. Such organizations can be very reluctant to contribute to coordinated efforts, either maintaining a proprietary system, or driving standards efforts towards their agendas. Yet, good examples exist whereby commercial organizations are successful partners in such projects. The possibility of extending such
Creating Instruments for Ideation: Software Approaches to Geovisualization
121
successes to geovisualization is a motivating challenge and demonstrates that tool developers should not preclude those with commercial interests from engaging in collaboration. The potential rewards of common software approaches are indisputably appealing: the reuse of code avoiding re-inventing the wheel; the effective communication of ideas via the distribution of components; the rapid and efficient development of new tools based on a firm and dependable environment that contains a comprehensive collection of modules representing the current state of the art. And above all interoperability allows us to proceed at speed, to interact with computers rapidly and so quickly design, modify and interact with revealing views of our data to address our need to develop instruments for the reasons cited in this chapter and thus aid the process of ideation.
5.8
Summary
In introducing the variety of approaches and objectives reported in the chapters that follow in this section of Exploring Geovisualization we asked why we invent new instruments and whether we need to do so. The arguments that we have presented identify motivations relating to inter-related changes and variations in the technology and data available, the tasks identified, the skills and experience of users and the availability of associated expertise to fuel our research. These factors demonstrate the wide range of issues that should be considered by those supporting and performing geovisualization and account for the equally varied set of research issues and solutions demonstrated in our work. Our thoughts on interoperability demonstrate an opportunity and identify "re-use" as a key theme that transcends much of our work. At a high level, the re-use of concepts generated by the "ideas chasers" is important, as the advances that they make are adopted, adapted or rejected. Collaboration between people with different skills should be encouraged to foster new ideas. We should not underestimate the importance of ideas re-use as our knowledge of geovisualization improves along with our subject-area expertise. At a more technical level, increasing the various levels of interoperability offers us the opportunity to develop more efficient and flexible instruments to support ideation (see Dykes, this volume (Chapter 13)). And whilst each of the various levels of interoperability that we have identified may be appropriate under certain circumstances each also comes with a set of technical, social, operational and even geovisualization issues that must be resolved. In addition, we believe that tool development has an important pedagogic aspect. When newly exposed to an existing body of tools researchers can sometimes bring novel ideas to the table. Whilst some "wheel re-invention" may take place when developing packages that largely duplicate existing tools, this can be minimized by taking advantage of existing resources and can equip researchers with the capabilities to develop new tools that expand upon and enhance current technology and techniques and address some of the issues raised here.
122
G. Andrienko et al.
In this introduction, we have asked more questions than we have answered. However, further reading will show the "richness" of approaches represented by the chapter authors as each strives to create and use instruments for ideation relevant to the data, questions, tasks, and expertise that they and their users are interested in as technology advances. These solutions will undoubtedly be different, and will change as technologies, expectations and knowledge develop.
Acknowledgements This multi-authored effort requires acknowledgment to our various funding bodies, the delegates at the International Cartographic Association Commission on Visualization and Virtual Environments workshop held at City University, London in 2002, who were the source of much discussion and provided invaluable feedback and the anonymous reviewers whose comments were extremely helpful and enabled us to crystallize and augment many of the arguments and perspectives presented.
References Acevedo, W., and Masuoka, P., (1997) "Time-series animation techniques for visualizing urban growth", Computers & Geosciences, 23(4), 423-436. Andrienko and Andrienko (1999a). Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(23/10/03). Andrienko, G., and Andrienko, N., (1999b) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999c) "Knowledge-based visualization to support spatial data mining", Proceedings of Intelligent Data Analysis. Berlin: Springer Verlag, pp. 149-60. Andrienko, G., and Andrienko, N., (1999d) "Making a GIS intelligent: Common GIS project view", AGILE'99 Conference, Rome, April 15-17, pp. 19-24. Andrienko, G. L., and Andrienko, N. V., (19990 "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Andrienko, N., Andrienko, G., Savinov, A., Voss, H., and Wettschereck, D., (2001) "Exploratory analysis of spatial data using interactive maps and data mining", Cartography and Geographic Information Science, 28(3), 151-165. Andrienko, N., Andrienko, G., and Gatalsky, P., (2003) "Exploratory spatio-temporal visualization: an analytical review", Journal of Visual Languages and Computing, 14(6), 503-514. Becker, R. A., Cleveland, W. S., and Wilks, A. R., (1987) "Dynamic graphics for data analysis", Statistical Science, 2(4), 355-395. Bertin, J., (1983) Semiology of Graphics: Diagrams, Networks, Maps. Madison, Wisconsin: University of Wisconsin Press.
Creating Instruments for Ideation: Software Approaches to Geovisualization
123
Boukhelifa, N., Roberts, J. C., and Rodgers, P. J., (2003) "A coordination model for exploratory multi-view visualization", Proceedings of Coordinated and Multiple Views in Exploratory Visualization (CMV2003), IEEE, (July 2003). Brodlie, K. W., Carpenter, L. A., Earnshaw, R. A., Gallop, J. R., Hubbard, R. J., Mumford, A. M., Osland, C. D., and Quarendon, P., (eds.), (1992) Scientific Visualization, Techniques and Applications. Berlin, Heidelberg: Springer-Verlog. Brooks, F. P. J., (1995) The Mythical Man-Month: Essays on Software Engineering, Addison-Wesley. Cartwright, W., Crampton, J., Dykes, J., Gartner, G., Miller, S., Mitchell, K., Siekierska, E., and Wood, J., (2001a) "User interface issues for spatial Information Visualization", Cartography and Geographic Information Systems, 28(1), 45-60. Casner, S. M., (1991) "A task-analytic approach to the automated design of graphic presentations", A CM Transactions on Graphics, 1O, 111-115. Cleveland, W. S., (1993) Visualizing Data. Summit, NJ: Hobart Press. Computer Science and Telecommunications Board (2003). CSTG Publication: IT Roadmap to a Geospatial Future. Online: http ://www7.nationalacademies.org/c stb/pub_geospatialfuture.html (23/10/03 ). Dykes, J. A. and Mountain, D. M., (2003) "Seeking structure in records of spatiotemporal behaviour: visualization issues, efforts and applications", Computational Statistics and Data Analysis, 43(Data Visualization II Special Edition), 581-603. Online: http://www, soi.city.ac.uk/--j ad7/csda/ Fabrikant, S. I., (2000a) Spatial Metaphors for Browsing Large Data Archives, Department of Geography, University of Colorado-Boulder. Fabrikant, S. I., (2000b) "Spatialized browsing in large data archives", Transactions in GIS, 4(1), 65-78. Fairbairn, D., Andrienko, G., Andrienko, N., Buziek, G., and Dykes, J. A., (2001) "Representation and its relationship with cartographic visualization: a research agenda", Cartography and Geographic Information Sciences, 28(1), 13-28. Fayyad, U., Piatetsky-Shapiro, G., and Smyth, P., (1996a) "The KDD process for extracting useful knowledge from volumes of data", Communications of the A CM, 39(11), 27-34. Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R., (1996b) Advances in Knowledge Discovery and Data Mining. Cambridge, MA: AAAI-Press/ MIT-Press. Gahegan, M., (1998) "Scatterplots and scenes: visualization techniques for exploratory spatial analysis", Computers, Environment and Urban Systems, 22(1), 43-56. Gahegan, M., Harrower, M., Rhyne, T.-M., and Wachowicz, M., (2001) "The integration of geographic visualization with databases, data mining, knowledge construction and geocomputation", Cartography and Geographic Information Science, 28(1), 29-44. Grudin, J., (1989) "The case against user interface consistency", Communications of the ACM, 32(10), 1164-1173. Jung, V., (1995) "Knowledge-based visualization design for geographic information systems", Proceedings 3rd ACM Workshop on Advances in GIS, ACM Press, pp. 101-108.
124
G. Andrienko et al.
Keim, D., and Kriegel, H.-P., (1996) "Visualization techniques for mining large databases: a comparison", IEEE Transactions on Knowledge and Data Engineering (Special Issue on Data Mining), 8(6), 923-938. Knapp, L., (1995) "A task analysis approach to the visualization of geographic data", In: Nyerges, T. L., Mark, D. M., Laurini, R., and Egenhofer, M. J., (eds.), Cognitive Aspects of Human-Computer Interaction for Geographic Information Systems. Dordrecht: Kluwer Academic, pp. 355-371. Kumar, H. P., Plaisant, C., and Shneiderman, B., (1997) "Browsing hierarchical data with multi-level dynamic queries and pruning", International Journal of HumanComputer Studies, 46(1), 105-126. MacEachren, A. M., (1994a) Some Truth with Maps: A Primer on Symbolization and Design. Washington: Association of American Geographers. MacEachren, A. M., (1994b) "Time as a cartographic variable", In: Hearnshaw, H., and Unwin, D., (eds.), Visualization in Geographical Information Systems. Chichester: Wiley, pp. 115-130. MacEachren, A. M., (1994c) "Visualization in modern Cartography: Setting the agenda", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Oxford: Pergamon, pp. 1-12. MacEachren, A. M., Wachowicz, M., Edsall, R., Haug, D., and Masters, R., (1999) "Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods", International Journal of Geographical Information Science, 13(4), 311-334. McCormick, B. H., DeFanti, T. A., and Brown, M. D., (eds.), (1987) "Visualization in scientific computing", Computer Graphics, 21 (6). Peuquet, D. J., and Kraak, M. J., (2002) "Geobrowsing: creative thinking and knowledge discovery using geographic visualization", Information Visualization, 1, 80-91. Qian, L., Wachowicz, M., Peuquet, D., and MacEachren, A. M., (1997) "Delineating operations for visualization and analysis of space-time data in GIS", GIS/LIS'97, Cincinnati, Oct. 28-30, pp. 872-877. Rana, S. S., and Dykes, J. A., (2001) "Augmenting animated maps with morphometric surface derivatives", GeoComputation. Brisbane, Australia: University of South Australia. Rana, S. S., and Dykes, J. A., (2003) "A framework for augmenting the visualization of dynamic raster surfaces", Information Visualization, 2(2), 126-139. Rhyne, T.-M., (1997) "Going virtual with geographic information and scientific visualization", Computers & Geosciences, 23(4), 489-491. Roberts, J. C., (2000) "Multiple-view and multiform visualization", In: Erbacher, R., Pang, A., Wittenbrink, C., and Roberts, J., (eds.), Visual Data Exploration and Analysis VII, Proceedings of SPIE, pp. 176-185, Online: http://www.cs.kent.ac.uk/pubs/2000/963/index.html Schuchardt, K., Myers, J., and Stephan, E., (2001) "Open data management solutions for problem solving environments: application of distributed authoring and versioning to the extensible computational chemistry environment", Proceedings of the Tenth
Creating Instruments for Ideation: Software Approaches to Geovisualization
125
International IEEE Symposium on High Performance Distributed Computing (HPDC 10), San Francisco, CA, August 7-9. Shifter, M. J., (1995a) "Environmental review with hypermedia systems", Environment and Planning B: Planning & Design, 22, 359-372. Shifter, M. J., (1995b) "Multimedia representational aids in urban planning support systems", In: Marchese, F. T., (ed.), Understanding Images - Finding Meaning in Digital Imagery. New York: Springer, pp. 77-90. Shneiderman, B., (1996) "The eyes have it: a task by data type taxonomy for Information Visualizations", Proceedings IEEE Symposium on Visual Languages. Washington: IEEE Computer Society Press, pp. 336-343. Shneiderman, (1998) Treemaps for Space-Constrained Visualization of Hierarchies. Online: http://www.cs.umd.edu/hcil/treemaps (23/10/03). Shrager, J., and Langley, P. E., (1990) Computational Models of Scientific Discovery and Theory Formation. San Mateo, CA: Morgan Kaufman. Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., and Hedley, N. R., (2001) "Cognitive and usability issues in geovisualization", Cartography and Geographic Information Science, 28(1), 61-75. Stuetzle, W., (1988) "Plot windows", In: Cleveland, W. S., and McGill, M. E., (eds.), Dynamic Graphics for Statistics. Belmont, Ca: Wadsworth Inc., pp. 225-245. Tufte, E. R., (1983) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, p. 197. Tufte, E. R., (1997) Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire, CT: Graphics Press, p. 156.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 6
Statistical Data Exploration and Geographical Information Visualization Martin Theus, Department of Computer-Oriented Statistics and Data Analysis, University of Augsburg, Universit~itsstr. 14, 86135, Augsburg, Germany
Keywords: exploratory data analysis (EDA), statistical thinking, selection and linked highlighting, high interaction user interfaces, large data sets
Abstract Geographical Information Systems (GIS) and statistical data analysis systems have a lot in common. Although the geographical reference is the dominant aspect of geographical data, these data can draw upon statistical principles and methodologies to be investigated thoroughly. Much of statistical data analysis has its roots in mathematics whereas most of geovisualization draws most directly from cartography. Today, both fields make heavy use of computing power and computer graphics and take advantage of highly interactive interfaces. Several software packages have been developed to either incorporate geographical information into interactive statistical software (e.g., REGARD, MANET or Mondrian) or vice versa to incorporate statistical graphics into GIS and geovisualization environments (e.g., GeoVISTA Studio, cdv or Decartes). This chapter investigates the overlap and differences between the two areas, and identifies a possible synergy between statisticians and geographers in the light of geovisualization. The current, increasingly common and shared challenges associated with the visualization of very large data sets are also discussed. Cartographic and statistical techniques may well be employed in unison to address these issues.
6.1
Introduction
This chapter is written from the perspective of a statistician, thus focusing on the analysis of the data of a geographical referenced data set and the integration of these analyses within the geographical context. Exploratory data analysis (EDA) has been widely adopted by many disciplines beyond statistics. For these methods to be applied appropriately, the statistical background of EDA should be considered and drawn upon. The roots and aspects of EDA are reviewed in w and the most important interactive techniques developed in EDA are presented in w and w Most of the examples 127
128
M. Theus
and implementations presented derive from research projects the author has been involved in over the last 10 years. These software issues are discussed in a general context in w and w from the perspective of a software developer. An outlook on the current and impending challenges associated with large data sets that draws upon this experience is provided in w
6.2
Exploratory Statistics
EDA was formally introduced by J. W. Tukey with his seminal book, first published in 1977 (Tukey, 1977). The ideas gathered in the book loosened the strict corset of mathematical concepts and introduced a more exploratory view of data analysis. In contrast to mathematical statistical thinking m which needs to set up hypotheses before looking at the d a t a - EDA suggests a more descriptive way of looking at data. Graphic depictions and numeric summaries of data are the key concepts in EDA. The predominance of graphical and descriptive methods as against optimal, mathematical methods has fluctuated throughout the history of statistics as depicted graphically in Figure 6.1. Statistics was dominated by mathematical statistics in the early 20th century, when many breakthroughs were made in the foundations of statistical theory. It took 50 more years and the upcoming availability of computers until EDA methods were introduced. To get a better idea of what is gathered behind the term EDA, one can look at some of the most important headings in Tukey's book. The first chapter, called "Scratching Down Numbers", promotes the idea of actually looking at the raw numbers IL
descriptive graphical large samples
computational/ power / /~,
tables
# graphs
9ED
~ T u k e y
Graunt "~~'~Arbuthnot ~ D u t n ~
~ ~layfair /Gauss ~
K. Pearson ~ \
/
/ /
Wald
Neyman
small samples mathematical F' optimal
mathematical insight
Figure 6.1. Schematicrepresentationof the fluctuating predominanceof the two major paradigms in statistics over the last 200 years.
Statistical Data Exploration and Geographical Information Visualization
~1 Data II
Hypothesis
I~1761 ..I Statistical I
129
Conclusions
LiRfeSri ] dUtlonl Figure 6.2. An illustration of the linear, hypothesis-based workflow of classical statistical reasoning.
of a data set. Stem-and-Leaf plots are maybe the best-known representatives in this class. In the second chapter "Schematic Summaries," Tukey advocates the use of pictures and tables. "Easy Re-expression" is the title of the third chapter, covering various transformation procedures, which either make data more "normal", or reveal new information. Looking at different subgroups or transformations of data is described in Chapter 4 "Effective Comparison". EDA does not ignore classical statistics, but is well aware of the benefits of probability theory, which is reflected in Chapters 19, "Shapes of Distribution" and 20 "Mathematical Distributions". In the Appendix of the book, Tukey dedicates a whole section on "Our relationship to the computer", which might have been a bit peculiar in 1977, given the relative scarcity of powerful computers in the late 70s. Tukey formulated his ideas in times where computers were expensive, non-interactive and non-graphical. This makes him in some sense a visionary, because he was able to imagine tools and techniques that were not available at that time. Today computers are powerful and cheap enough to support all of these exploratory techniques as some of the instruments for exploration described in this chapter demonstrate. The strictly linear thinking typical of classical statistics is shown in Figure 6.2. In contrast, Figure 6.3 represents the iterative work flow embraced by EDA. It is important to understand that there is no fixed set of tools for EDA. Certain graphical explorations might be useful for one data set but less helpful for another data set. Furthermore, we might re-run different methods in the light of newly revealed structures in an iterative process of exploration and confirmation and/or rejection. Exploratory spatial data analysis (ESDA), is the natural extension of EDA thinking and methods towards geographical problems. ESDA dates back as far as the late 1980s, when desktop computers offered the first implementations of EDA, which were soon extended with geographical components (see Unwin and Unwin (1998)). Mark Question to ~
Explorative
Solutions
) ~
I Revealed Structure 1
Figure 6.3. An illustration of the iterative process used in EDA.
130
M. Theus
Table 6.1.
Five stages of implementing a GIS.
Statistics 1. 2. 3. 4. 5.
Data Input Data Manipulation Data M a n a g e m e n t Query and Analysis Visualization
Cartography
Data(base)Management 9
9
o 9
9 9
o
o 9
Monmonier's work describes the possible synergy between EDA and geographical analysis and resulted in example software (Monmonier, 1988, 1989). This was later developed to produce more powerful tools for ESDA.
6.3
Technologies for Data Exploration
When working with a geographic information system, essentially five tasks are performed. Table 6.1 classifies these five tasks according to the use of methods from statistics, cartography and data(base) management (Theus, 2002a). Solid circles indicate a strong impact, open circles a weaker impact of each field on the five activities. Steps 4 and 5 are typical domains where exploratory statistics can be applied. Whereas geo-scientists can learn a lot from computer scientists regarding tasks 1-3 (Gahegan et al., 2001), statisticians could well be the most influential source of ideas for activities 4 and 5. Interactive environments blend steps 4 and 5 together, offering flexible query and analysis methods within the tool for geographical visualization. This results in a much stronger role for cartography on tasks involving "Query and Analysis" in the future (e.g., map algebra might be considered a rather cartographic method contributing to the task of query and analysis).
6.4
Interactive Techniques
Tasks involving both query and analysis and visualization can draw upon a number of forms of interaction. These include the following generic techniques.
6.4.1
Selection and linked highlighting
Selection and linked highlighting are the basic interactive operation required by EDA. The collection of early papers presented in Cleveland and McGill (1988) remains an excellent source. These techniques support dynamic linking of information between various kinds of graphs and/or tables, in order to increase the number of dimensions that we are able to consider concurrently. Any case that is selected in one plot is graphically highlighted in all other plots. Selections can be quite complex, when they include different variables using Boolean operators such as "AND", "OR" or "XOR". Hofmann (1998) and Theus et al. (1998) discuss the benefits of advanced selection mechanisms in interactive data analysis systems.
Statistical Data Exploration and Geographical Information Visualization
131
In a static environment, maps and/or other graphic depictions are typically used for communicating fixed results. Thus, such maps often incorporate information on several variables at a time. In contrast, maps in a dynamic environment will support various linked interactions, allowing the user to focus in on fewer variables at a time. In a geographical context, linking can be used to create a dynamic connection between any kind of information and a graph, or a map. Figure 6.4 shows an example of census data of the Midwest of the USA (Stynes, 1996). In the figure, a parallel coordinate plot (Inselberg, 1985; Wegman, 1990) is linked with the corresponding map by highlighting corresponding cases in each.
6.4.2
Re-ordering
The ordering of graphic objects is an important consideration, whether it is the variables contributing to a parallel coordinate plot (depicted as axes), the orthogonal axes of a scatter plot, or the categories in a barchart. In most visualization tools, this is determined by some initial default. In a spatio-temporal context, a flexible re-ordering of the temporal information is at the core of an interactive analysis. A carefully designed graph might specify the ordering explicitly. In an interactive analysis, we want to be able to reorder objects immediately. All other linked graphs should update to the new ordering immediately where ever the change applies.
6.4.3
Re-expression
When interactively working with maps, operations like zooming and panning are indispensable. As long as the zoom levels do not exceed a factor of about 10, it is possible to work within a single map of a fixed resolution. Higher zoom levels need an implementation of dynamic, on-the-fly maps with different resolutions, leading to the concept of logical zooming (Unwin, 1999). Kraak and Ormeling (1996) discuss zooming in a geographical context as "change of scale" more generally, although changing scale of display and data model should not be confused. There is a need for logical zooming in classical statistical graphics as well, as the amount of data increases either in terms of the number of observed cases or the number of variables recorded. Whereas methods of logical zooming have been well know in Geography for years, addressing this issue is a new challenge for statisticians and data analysts. Plots such as scatter plots and parallel coordinate plots can suffer badly from very large data sets due to heavy over plotting. For scatter plots, one solution is to bin data - i.e., to summarize the data by deriving densities on an imposed 2D grid. When zooming in, the amount of data in the zoomed area can be expected to decrease, until the zoomed view is capable of showing discernible symbols for each measurement once again. Figure 6.5 shows an example of a scatter plot of over 300,000 observations. The data depict financial transactions from an undisclosed financial institution. The scatter plots show the transaction amount and an associated transaction fee in Deutsch Mark. The three leftmost plots show binned data, whereas the rightmost plot shows individual points. The data show extreme self-similarity at almost
132
M. Theus
O
.t-.,, rt ~ "O x_
0 0 o
0 "0 c Q..
E
,n: L_
O'J I,k
Statistical Data Exploration and Geographical Information Visualization
133
Figure 6.5. Zooming into 300,000 points. Plots (a), (b) and (c) use binning to approximate the scatter plot as zooming occurs. Plot (d) shows the 4500 points in the zoomed area, which is 3.9 x 1027 the area of plot (a).
any level of zooming, revealing artifacts (accumulations at thresholds of one variable) of different severity the deeper we zoom in. Logical zooming can be applied to other statistical graphs as well, when the number of cases and/or variables is large. For example, mosaic plots (Hartigan and Kleiner, 1981; Friendly, 1994) benefit from logical zooming, with large numbers of variables and/or categories. Another application of re-expression is to change the color-coding of choropleth maps. Whilst plenty of rules, hints and suggestions are available concerning appropriate color-coding in specific cases (see Brewer (1994)), in an interactive analysis one may need to explore many different codings to reveal patterns and structure and assess their stability. Changing the mapping must be easy and happen instantaneously to help users find structures and pattern in the data efficiently. In contrast to a static choropleth map, which typically uses a specific color scheme determined by the domain problem, interactive choropleth maps for general problems work best with a very simple color scale that does not suggest false interpretations. Unwin and Hofmann (1998) suggest an S-shaped transfer function to map data values
134
M. Theus
Figure 6.6. An example of the transfer-function Equation (6.1) implemented in MANET (left). A collection of four different parameter settings of the transfer-function Equation (6.1) (right).
to (grey-) shades using two parameters a and b. This function also supports the mapping of color scale ranging between two selected hues. ColorBrewer (Brewer and Harrower, 2002; Harrower and Brewer, 2003) can be used as an experimental guide. The S-shaped transfer function is defined as:
f(x) := a(x/a) b
for
x --< a
f(x) := 1 - (1 - a)((1 - x)/(1 - a)) b
for
x>_a
and (6.1)
The parameter a(0 -< a -< 1) determines the centering and b(0 -< b < oo) the form. An example of the proposed transfer-function to map data values to shades of grey is provided in Figure 6.6. The left-hand graphic shows the implementation in the MANET software (Unwin et al., 1996). A map of the US Midwest dataset is shown. The map is shaded according to the absolute population in the counties. The distribution is extremely skewed due to the few big cities - a condition that holds true if population density would be plotted instead due to the relatively minor variation in county area. Thus, parameters have been chosen that show differences between counties of smaller population or population density as well as the extreme outliers associated with the cities. MANET allows the specification of these parameters via sliders and updates the map instantaneously. Furthermore, the bounds of the transformed variable are displayed. The right-hand graph in Figure 6.6 shows four different parameter settings of the transfer function, illustrating its flexibility.
6.4.4
Interrogation
The ability to interrogate is a key feature of interactive systems for E(S)DA. Whenever the analyst identifies structures, anomalies or other interesting features, interrogation allow us to follow up on these structures to prompt thought with immediacy and ease. We can delineate various steps of interrogation: The simplest step is to just give an overview of where the cursor is located within the current coordinate system.
Statistical Data Exploration and Geographical Information Visualization
9
9 9
135
A more detailed query gives the values of the variables actually displayed in the plot (e.g., clicking on a point in a scatter plot gives the exact x- and y-coordinates of that point or object). The same query can also give information on variables not displayed in the graph. Going even further, an interrogation can give us the whole statistics on an object or collection of objects (e.g., clicking on a polygon can result in a local statistic, e.g., z-score.)
Furthermore, we can distinguish between contiguous interrogations and distant information display. Contiguous interrogation displays the information directly next to the location of the object that was queried, i.e., usually at the position of the mouse pointer. Distant interrogations show the information in an extra window preferably a floating window. Figure 6.7 shows two forms of interrogation - an example of an extended contiguous interrogation within Mondrian (Theus, 2002b) and a distant interrogation as implemented in cdv (Dykes, 1995, 1998). Fekete and Plaisant (1999) introduce another taxonomy of interrogations, based on the problem of placing labels of objects most efficiently without over plotting the labels. Whereas the problem described here deals with displaying multiple information components for a single object they try to solve the problem of displaying a single label for multiple nearby objects. This is an issue that has also received much attention from the cartographic community as techniques for automated map-making have taken advantage of sophisticated name placement algorithms. 6.5
Software
Tools
Most of the concepts discussed thus far have been implemented in software. In general, we can categorize these software tools according to three defining characteristics through which they differ: 1.
the range of statistical analysis types available;
Figure 6.7. An example of distant interrogation in cdv - the information is displayed in the upper left (left). Mondrian shows the information directly at the queried object (right).
136
M. Theus 2. 3.
the level of interactivity for an exploratory analysis provided by the system; the degree to which geographical information can be incorporated.
These three characteristics can be used to assess the capabilities of the different tools. At least, three approaches to tool development have been attempted. These relate to development of statistical tools for EDA and geographic analysis, the development of mapping tools to incorporate interactivity and EDA and formal links between commercial GI packages and statistical software, for example, (see Andrienko et al., this volume (Chapter 5)). Most classical statistics software is strong on characteristic 1, and typically offers a wide variety of analysis methods. These are seldom interactive and thus hardly suitable for an exploratory analysis. It is rare that such software offers any means for analyzing geographical data (i.e., the ability to plot, vary and manipulate maps, raster images, flows and other representations of spatial data). Software such as REGARD (Unwin, 1994), MANET or Mondrian are designed for analyzing data interactively and additionally offer basic means of plotting statistical maps, which are fully integrated in the interactive context. cdv and Descartes (Andrienko and Andrienko, 1999) have been developed to offer interactive interfaces for Information Visualization with a geographic focus. Here, the primary display is a map showing the spatial locations of the data. Such software has considerable strengths in terms of the various types of interactive map that are supported, but does not offer the advanced tools for statistical analysis such as that available in EDA software. Another approach to providing EDA capabilities for geographic data is the component-based model implemented in the GeoVISTA Studio software (Gahegan et al., 2001). The result is an extensible and interactive Java-based visual programming environment that allows for the rapid, programming free development of complex data visualization, exploration and knowledge construction applications to support geographic analysis. Finally, classical GIS cover a wide range of aspects of cartography and geographic data management and offer a variety of specialist spatial features. The level of interactivity provided by commercial GIsystems is usually poor and tools for EDA are frequently either absent or of limited scope. Cook et al. (1996) employed a methodology for increasing the EDA capabilities of geographic information systems by incorporating highly interactive statistical graphics into a commercial GIS. Their ArcView/XGobi-Link project resulted in linkages between the XGobi statistical package and the ArcView GIS. This link relies upon the two software tools running in parallel and communicating via remote procedure calls. A review of other similar approaches that employ such links is provided by Symanzik et al. (2000). Additional details on the different types of interactive tools and the benefits of interfaces and various forms of tool integration are offered by Unwin and Hofmann (2000) and Shneiderman (1994).
Statistical Data Exploration and Geographical Information Visualization
6.6
137
Development Environments
When reviewing the systems presented in w it is worth noting that the degree of overlap between tools is relatively minor, despite the inclusion of a variety of statistical functions, high interactivity and a complete set of geographic visualization tools. On the one hand this could just mean a lack of suitable systems, on the other hand it could also mean that the workflow in geographic data visualization is performed in two stages: firstly, exploratory analysis is performed, then secondly once findings are obtained a (static) graphical representation is generated for purposes of communication. When considering the [Cartography] 3 diagram (Figure 6.8) (Kraak and Ormeling, 1996, adapted from MacEachren and Taylor, (1994)), we find high interaction positioned in the opposite corner of the [Cartography] 3 designed for public presentation. This might be an explanation as to why no single tool serves all the needs on the path from the lower left of the cube to the upper right. Thus the development environment required for geovisualization will most probably include more than one "tool" to cover each of the steps in an exploration/visualization/presentation process. Gahegan, this volume (Chapter 4) offers some thoughts on the processes associated with undertaking geovisualization and their (inter) relationships. Different tools are designed for particular types of process and have particular strengths. A brief parable may help illustrate this point. Today we are able to buy an integrated microwave/grill/oven appliance, which would have been three single appliances 20 years ago. Although many of us will be happy with the advantages of the integrated version (space saving, multi-functional and easy to
Figure 6.8. The [Cartography]3 (adapted from MacEachren and Taylor (1994)). The schematic diagram displays the relationship between exploration, analysis and presentation in a map based information system.
138
M. Theus
use), a professional cook still might want to have the single appliances at the highest possible specification due to their concerns with performance. This distinction can also be found in statistics. A highly interactive environment usually does not plot the data in a way that is presentable, whereas presentation graphics usually cannot be used for exploration. On the other hand, if we think of the visualization/presentation side of an interactive environment with user interaction as well, this distinction can no longer be made that strictly. Although this approach would combine all steps into a single system, such an approach demands many more skills from the user than a static solution would do.
6.7
Very Large Data
EDA came into being at times when data set sizes were small ( < 100 cases) or medium ( < 10,000 cases) in size As data sets grew in size and associated challenges became apparent, a new field emerged, known as datamining and knowledge discovery in databases (for example, Fayyad et al., 1996). Although classical mathematical statistics cannot cope well with large data sets, the methods of EDA are still valid here (Carr et al., 1987 provide an early example). The management of very large data sets can only be guaranteed to be efficient, if database systems are used. A domain specific storage strategy is needed, to assure best results. "Large" can mean many cases (> 100,000) and/or many variables (> 50). In both situations, classical methods usually fail to provide an insight into the data. Certainly, not all summaries and graphics used in exploratory analysis scale up to large data sets (for a discussion of scaling problems of statistical graphs, see Theus (2004)). Note that this classification deviates from Huber's first taxonomy, which he set up before the era of datamining and KDD. The classification presented here, reflects the current practice of statisticians and computer scientists working in the area of datamining applications and tool development. When plotting millions of objects, graphs get cluttered and new graphing techniques have to be investigated. One solution to avoid over plotting is to use oL-channel transparency, where over plotting points add their color values, resulting in very saturated areas of high density and light areas of low density (Cook et al., 2004). Figure 6.9 left
Figure 6.9. Example of oL-channel transparency. Each plate has an opacity of 25%.
Statistical Data Exploration and Geographical Information Visualization
139
shows an example of 4 black plates, each having a transparency level of 75% thus an opacity level of 25 %, respectively. As the plates overlap, the intersecting area gets darker, indicating the higher density of objects. Figure 6.9 right shows the same example with two red plates. This second example shows some of the difficulties associated with interpreting combinations of more than one hue when using oL-channel transparency. Other more specific solutions need also to be developed to address this issue. Investigating hundreds of variables can be very cumbersome. A combination of automated selection procedures together with a strong interactive component are likely to be the most productive methods for handling such data. Solutions that address the issues related with very large data sets are likely to apply to geovisualization tools as well as those for EDA. Indeed, issues associated with the visual analysis of very large geographical data sets are even more acute as the need to depict spatial relations means that the graphical representation of geographical phenomena offers fewer degrees of freedom than are available for the graphical depiction of arbitrary statistical data.
6.8
Conclusions
Specialised instruments that provide interactive statistical graphics are the most significant tool for EDA. Most of the interactive functionality can be applied in the context of geovisualization as well. Linked highlighting, re-expressions and interrogation are the fundamentals of interactivity. EDA methods complement the statistical data analysis part of a geographical data exploration. The systems currently available already cover a broad range of functionality and visualization. Nonetheless, it is desirable to add more functionality, by either extending the single systems or by extending the interoperability of different systems in order to make use of various components. Whilst efforts to connect data and resources continue (see Schroeder, this volume (Chapter 24)) the seamless integrate of systems developed for a very informal, exploratory workflow with those that support highquality presentation and communication of final results of an analysis remains problematic. Novel, component-based systems may offer increasingly customizable interactive presentations that guide the user to specific findings in the geographic data, and thus may overcome this separation. The various issues associated with the depiction and interactive analysis of very large data sets constitutes an area of high potential for synergy between those with skills in developing tools for EDA, ESDA and datamining. Such data sets do not only pose new questions for data storage and handling, but point towards new analysis tools and techniques. Only a close integration of expertise in all related fields can lead to effective new solutions.
Acknowledgements The author would like to thank the participants of the ICA workshop at City University London in September 2002 for their interaction and fruitful discussions. Furthermore,
140
M. Theus
thanks go to the reviewer for their thorough work on the chapter, which substantially improved and enriched it. This manuscript was prepared while the author was visiting the Department of Statistics, at Iowa State University, ISU. References Andrienko, G. L., and Andrienko, N. V., (1999) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Brewer, C. A., (1994) "Color use guidelines for mapping and visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography, Vol. 2. Oxford: Elsevier Science Ltd., pp. 123-148. Brewer, C. A., and Harrower, M., (2002) ColorBrewer - Selecting Good Color Schemes for Maps. Online: http://www.personal.psu.edu/faculty/c/a/cab38/ColorBrewerBeta.html (23/10/03). Carr, D. B., Littlefield, R. J., Nicholson, W. L., and Littlefield, J. S., (1987) "Scatterplot matrix techniques for large N", Journal of the American Statistical Association, 82(398), 424-436. Cleveland, W. S., and McGill, M. E., (1988) Dynamic Graphics for Statistics. Belmont, CA: Wadsworth. Cook, D., Majure, J. J., Symanzik, J., and Cressie, N., (1996) "Dynamic graphics in a GIS: exploring and analyzing multivariate spatial data using linked software", Computational Statistics: Special Issue on Computeraided Analysis of Spatial Data, 11 (4), 467-480. Cook, D., Theus, M., and Hofmann, H., "The binned scatterplot", Journal of Computational and Graphical Statistics, (in preparation). Dykes, J. A., (1995) "Cartographic visualization for spatial analysis", In: Proceedings 17th International Cartographic Conference, Barcelona, pp. 1365-1370. Dykes, J. A., (1998) "Cartographic visualization: exploratory spatial data analysis with local indicators of spatial association using TCL/TK and cdr", The Statistician, 47(3), 485-497. Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R., (1996) Advances in Knowledge Discovery and Data Mining. Cambridge, MA: AAAI-Press/MIT-Press. Fekete, J.-D., and Plaisant, C., (1999) "Excentric labeling: dynamic neighborhood labeling for data visualization", In: Proceedings of ACM Conference on Human Factors in Computing Systems, CHI '99. New York: ACM Press, pp. 512-519. Friendly, M., (1994) "Mosaic displays for multi-way contingency tables", Journal of the American Statistical Association, 89(425), 190-200. Gahegan, M., Harrower, M., Rhyne, T.-M., and Wachowicz, M., (2001) "The integration of geographic visualization with databases, data mining, knowledge construction and geocomputation", Cartography and Geographic Information Science, 28(1), 29-44. Harrower, M., and Brewer, C. A., (2003) "ColorBrewer.org: an online tool for selecting colour schemes for maps", The Cartographic Journal, 40(1), 27-37, Online: http://www.colorbrewer.org
Statistical Data Exploration and Geographical Information Visualization
141
Hartigan, J. A., and Kleiner, B., (1981) "Mosaics for contingency tables", In: Eddy, W. F., (ed.), Computer Science and Statistics: Proceedings of the 13th Symposium on the Interface, New York, Springer, pp. 268-273. Hofmann, H., (1998) "Selection sequences in MANET", Computational Statistics, 13( 1), 77-87. Inselberg, A., (1985) "The plane with parallel coordinates", The Visual Computer, 1, 69-91. Kraak, M. J., and Ormeling, F. J., (1996) Cartography - Visualization of Spatial Data, Longman. MacEachren, A. M., and Taylor, D. R. F., (1994) Visualization in Modern Cartography. Oxford: Pergamon. Monmonier, M., (1988) "Geographical representations in statistical graphics: a conceptual framework", 1988 Proceedings of the Section on Statistical Graphics, Alexandria, VA, American Statistical Association, pp. 1-10. Monmonier, M., (1989) "Geographic brushing: enhancing exploratory analysis of the scatterplot matrix", Geographical Analysis, 21 (1), 81-84. Shneiderman, B., (1994) "Dynamic queries for visual information seeking", IEEE Software, 11 (6), 70-77. Stynes, K., (1996) ESDA with LISA Conference: Data for ESDA with LISA. Online: http ://www.mimas.ac.uk/argus/esdalisa/midwest/Data.html (23/10/03). Symanzik, J., Cook, D., Lewin-Koh, N., Majure, J. J., and Megretskaia, I., (2000) "Linking ArcView and XGobi: insight behind the front end", Journal of Computational and Graphical Statistics, 9(3), 470-490. Theus, M., (2002a) "Geographical information systems", In: Kloesgen, W., and Zytkow, J., (eds.), Handbook of Data Mining and Knowledge Discovery, Oxford University Press. Theus, M., (2002b) "Interactive data visualization using Mondrian", Journal of Statistical Software, 7 (11). Theus, M., "Upscaling Statistical Graphics", In: Unwin, A., Theus, M., Hofmann, H., and Wilhelm, A., (eds.), How to Visualize a Million. New York: Springer, (in press). Theus, M., Hofmann, H., and Wilhelm, A., (1998) "Selection sequences - interactive analysis of massive data sets", Proceedings of the 29th Symposium on the Interface: Computing Science and Statistics. Tukey, J. W., (1977) Exploratory Data Analysis. Reading, MA: Addison-Wesley. Unwin, A. R., (1994) "Regarding geographic data", In: Dirschedl, P., and Ostermann, R., (eds.), Computational Statistics. Heidelberg: Physika, pp. 315-326. Unwin, A. R., (1999) "Requirements for interactive graphics software for exploratory data analysis", Computational Statistics, 14, 7-22. Unwin, A. R., and Hofmann, H., (1998) "New interactive graphics tools for exploratory analysis of spatial data", In: St Carver, (ed.), Innovations in GIS 5. London: Taylor & Franics, pp. 46-55. Unwin, A. R., and Hofmann, H., (2000) "GUI and command line - conflict or synergy?", In: Berk, K., and Pourahmadi, M., (eds.), Computing Science and Statistics. Proceedings of the 31st Symposium on the Interface. Unwin, A. R., and Unwin, D., (1998) "Exploratory spatial data analysis with local statistics", The American Statistician, 47(3), 415-421. TM
142
M. Theus
Unwin, A. R., Hawkins, G., Hofmann, H., and Siegl, B., (1996) "Interactive graphics for data sets with missing values - MANET", Journal of Computational and Graphical Statistics, 5(2), 113-122. Wegman, E., (1990) "Hyperdimensional data analysis using parallel coordinates", Journal of the American Statistical Association, 85(411), 664-675.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 7
Graph Drawing Techniques for Geographic Visualization Peter Rodgers, University of Kent, Canterbury, Kent, UK
Keywords: graph drawing, diagram layout, geographic visualization, geovisualization, Information Visualization
Abstract Complex data consisting of related items often occurs in Geography, and is frequently represented as a graph. Examples include travel routes, regional borders, population flow and commerce. The field of graph drawing provides an effective group of techniques to tackle the problem of visualizing such data. This chapter provides a broad background to graph drawing methods and discusses possible future applications of this area to geovisualization.
7.1
Introduction
This chapter overviews the area of graph drawing and discusses its application to the visualization of geographic information. Geovisualizers often need to represent data that consist of items related together. Such data sets can be abstracted to a mathematical structure, the graph. A graph contains nodes and edges where the nodes represent the items or concepts of interest, and the edges connect two nodes together according to some associational scheme. Examples of graph data include: network topologies; maps, where nodes represent towns and edges represent roads; and social diagrams where people are the nodes and edges represent some relationship between people, for instance, that they are friends. A good layout allows users to more easily investigate the data when performing tasks such as following paths, seeing clusters of closely related nodes and discovering general structures in data. Laying out graphs by hand is time consuming; however, there are a number of proven graph drawing methods which can be used to automatically visualize the data. Graph drawing concentrates on the automatic positioning of nodes and edges on screen or paper. Geographic visualization often includes edges that link regions, but often the spatial locations of regions that would be represented by nodes is already known. So why is graph drawing relevant to the visualization of geographic data? 143
P. Rodgers
144
9
9
9
When regions are fixed, often the connections between them are not. Graph drawing includes techniques for distinguishing close edges, and routing them in the most comprehensible manner. For example visualizing train networks (Brandes and Wagner, 2002). In some cases fixing the exact position of locations is not desirable (such as underground railway maps), or even possible (such as global map projections), but some essence of the position is required. For example, the relative positions between regions to the left, right, and below might need to be retained. In these cases, graph drawing can help by nicely laying out the nodes, whilst still maintaining the desired relationships between them. Some data, even if it is connected to geographic position, is not best analysed by retaining physical locations. Datamining of this information might be best achieved by providing visualization based on the topological relationship between items, rather than one based on their location (Monmonier, 1991). For example, economic data might be best seen by spatially grouping countries that have strong links (Figure 7.1). Mapping the internet also typically disregards location, both because of relevance, and because precise location information can be difficult to derive from Web sites (Cheswick et al., 2000; Dodge and Kitchin, 2000).
100 North
Former
583
W.Europe ~
~
136
~157 104 Middle East
Figure 7.1. An example of graph drawing. The edges in the graph represent total merchandise trade in 2001 between regions of the world. Only edges representing amounts over 100 billion dollars are shown. This has been drawn with a force directed graph drawing algorithm that includes an edge length heuristic (Mutton and Rodgers, 2002).
Graph Drawing Techniques for Geographic Visualization
145
The items of interest may not have an obvious physical location (such as data about corporations), hence an automatic graph layout is required to position the items based on how they are related. Familiar geovisualizations may be used effectively with alternate graph visualizations of the same data. Such linked multiple views (Roberts, 2000) would allow an examination of the spatially oriented representation along side the relational representation for greater breadth of understanding. An example of such multiple views is provided in Figure 7.2. Early graph drawing can be traced back to the barycenter drawing method of Tutte (1963). Early computer generated graph drawing included the work of Sugiyama et al. (1981) to address the need to visualize hierarchical structures. Later, Eades (1984) adapted the Spring Embedder from microchip design methods and this became the first widely used technique for laying out general graphs. It is a testament to their effectiveness, and a comment on progress in the field that variations on these methods are still widely used. Further methods rapidly followed and overviews of common techniques can be found in DiBattista et al. (1999); Kaufman and Wagner (2001). Methods to achieve a good drawing often depend on some knowledge of the graph semantics (that is, what the graph represents). If the graph has some known structure then there are often quick specialist algorithms for producing an effective drawing. Such structures include trees and acyclic graphs, which can represent classification hierarchies and organizational charts. Another specialist feature of graphs is planarity. Planar graphs are those which can be drawn without any edge crossings, and have a particular application to geographic data, as graphs that have edges representing region borders are planar. Other algorithms work on more general graphs including force directed methods, which can produce layout reasonably quickly. Slower methods involve quantitatively measuring the aesthetics in a graph and then moving the nodes so as to improve the resulting score. It is common for these methods to be combined in a hybrid manner, so that a fast method can produce the initial drawing, followed by a slower technique for fine tuning. For those wishing to visualize graph based data without coding algorithms there are several effective commercial and academic tools for accessing most of the graph drawing techniques mentioned in this chapter. Graph Layout Toolkit (Tom Sawyer Software, 2003) is one of the most complete and widely used, but is not freeware, whereas AT&T's sophisticated GraphViz (c), containing spring embedder and hierarchical algorithms, is open source. Other systems that implement graph drawing techniques include TouchGraph (TouchGraph, undated), DaVinci (Fr6hlich and Werner, 1994), Graphlet (Himslot, 1996), Jviews (ILOG Inc., 2003), Tulip (Tulip-Software.org, 2003) and VCG (Sander, 1994). Note that this is not a complete list, and there are several general graph algorithm systems available that include graph drawing functionality. The remainder of this chapter is organized as follows: More detail is provided on the graph drawing methods mentioned above in w Dynamic graph drawing, which is the process of laying out a graph that changes over time, is discussed in w The issues involved in drawing graphs in three dimensions are assessed in w followed by a discussion of the rendering of graph drawings in w Finally, a conclusion is
146 P. Rodgers
O O
c r-
111 o 0
N
0
>
if)
(9 .c_
tO -.~ N ...~
O
._~
r0
if)
N
L.. c
E 0 c 0 N
r.-
._m t~
c c _.1
N N
U.
Graph Drawing Techniques for Geographic Visualization
147
provided, offering some idea of future research challenges, particularly those relating to geovisualization.
7.2
Hierarchical Drawing Methods
Early graph drawing methods (Sugiyama et al., 1981) dealt with specialist graph structures, where a notion of a single direction applies. This might be because the data is organized into a hierarchy, with the main concepts at the top, and other concepts below, or because there is actual flow. These might include abstract representation of rivers, data relating to movements of people or money between regions and hierarchical classifications, based upon scale, for example, (see Fabrikant and Skupin, this volume (Chapter 35), figures 7 and 8). The main feature needed for this form of layout is that either the graph has no cycles, or that nodes participating in cycles are ordered. Figures 7.3 and 7.4 show illustrations of hierarchical layout. The nodes in a hierarchical graph can be placed in layers, with the first nodes (called roots) in the top layer, and later nodes further down. The leaves at the bottom are the lowest nodes of all, as they have no following items. Edges that cross layers are often assigned dummy nodes at each layer crossing in order to aid the drawing process. Once nodes are layered their relative positions in the layer can be assigned. Various approaches are taken, but most are designed to reduce the number of edge crossings and edge bend points.
25
Figure 7.3. Top-down hierarchical layout of world dynamics data (Forrester, 1973) drawn with GraphViz.
148
P. Rodgers
Figure 7.4. Radial hierarchical layout (Yee et al., 2001). This allows many more positions for nodes in lower layers, at the expense of a notion of flow that can be seen in the top-down approach.
7.3
Planar Layout Techniques
A planar graph is one that can be drawn with no edge crossing. Many graph theoretic algorithms rely on a graph being planar, and there are quick methods to lay out such a graph. Graphs where edges represent adjacency between regions are planar (because no border can cross another border), as are graphs that represent Delaunay triangulations, a precursor to several analytical techniques in Cartography (Watson and Mees, 1996; Tsai, 1993). A common method for drawing planar graphs is to give the nodes a canonical order, placing the first three nodes at vertices of a triangle and then adding lower ordered nodes in positions within the triangle, as shown in Figure 7.5. This method does not always produce a satisfactory layout, but it is improved if followed by a force directed method, such as the spring embedder. It is also possible to planarize graphs, a process of converting non-planar graphs to planar graphs by removing crossing edges or placing new nodes at edge crossings. Planar drawing algorithms can then be applied to the planarized graph, after which the changes to the graph can be undone, and the modifications integrated into the drawing.
7.4
Force Directed Layout Techniques
Force directed methods are widely applied techniques. Here, repeated applications of a force model result in the graph attaining a minimal energy state, where the graph is considered to be well drawn. These methods have the significant advantage of being applicable to general graphs, with no requirement that the graph has a specific structure or
Graph Drawing Techniques for Geographic Visualization
149
Figure 7.5. A planar graph in a layout based on Schnyder's method (Schnyder, 1990). Planar graphs are those that can be drawn without any of their edges crossing.
that it meets some topological constraints. For example, the spring embedder can be used with either planar or non-planar graphs. A fast approach for general graphs is the barycenter approach (Tutte, 1963) (Figure 7.6). The nodes are drawn at the average distance of their neighbours (which is known as their barycenter). This method is repeated for a set number of iterations, or until nodes stop moving, having reached a minimal energy state. A subset of the nodes must be fixed, to indicate the outer borders of the drawing space, or the nodes will simply collapse to a single point. This method is effective for sparse, symmetrical graphs, where fixed nodes can easily be defined. Eade's spring embedder (Eades, 1984) is perhaps the best known graph drawing technique and is a popular method for visualizing geographic data such as network topologies and the internet (Huang et al., 1998; McCrickard and Kehoe, 1997). A variant that has an additional edge length heuristic is shown applied to a planar graph in Figure 7.7. Nodes are treated as charged particles that repel all other nodes, and edges are treated as
Figure 7.6. A graph after the barycenter drawing approach has been applied. The nodes with black centres are fixed.
150
P. Rodgers
Figure 7.7. A planar graph representing bordering states in mainland Europe. A modified spring embedder has been applied. Such a graph could be attributed with relevant geographic information such as traffic data or population flows. (See Fabrikant and Skupin, this volume (Chapter 35)) for applications in cartographic information spaces.
springs that attract connected nodes. In an iteration of the method, the forces are calculated for all the nodes in a graph and the nodes are then moved in the direction that the forces indicate. Iterations continue until a least energy state is achieved, or a set number of iterations are performed. The result is a graph where nodes have decent minimal separation and connected nodes are reasonably close together. Recent work (Tunkelang, 1998; Walshaw, 2000) has made the application of this method to large graphs possible, with graphs containing thousands of nodes being drawn in a few seconds.
7.5
Optimizing Aesthetic Criteria
The previous methods take implicit judgements about what a good graph drawing is. However, there are approaches that attempt to explicitly improve the aesthetics of a graph by allowing users to choose from a range of aesthetic criteria, weight them, and allow a multiple criteria optimizing technique to automatically lay out the graph based on these preferences. The major disadvantage of these techniques is the time taken to reach a reasonable drawing. Typically, the criteria have to be recalculated on each iteration of the optimizer and many iterations are required. Multi-criteria optimizers that have been used include the (slow) simulated annealing method (Davidson and Harel, 1996) and the (even slower) genetic algorithm approach (Hobbs and Rodgers, 1998). The aesthetic criteria used must be quantifiable. Common criteria include: the number of edge crossings, statistics relating to even node distribution (such as the variation
Graph Drawing Techniques for Geographic Visualization
151
in distance of closest neighbours of each node); the area or aspect ratio of a rectangle containing the graph; the angular resolution of edges emanating from nodes (a small angle between edges means that they are difficult to distinguish); the number of edge bends; evenness of edge length; methods for measuring the symmetry in the graph; and encouraging directed edges in a particular direction. DiBattista et al. (1999) and Kaufmann and Wagner (2001) provide detailed discussions of aesthetic criteria. A feature of such criteria is that empirical studies can be performed to discover user preference. This work is ongoing, but some results indicate that reducing edge crossings is a major factor in making graph layout comprehensible (Purchase et al., 2000). Criteria-based methods may be effective in aiding the visualization of geographic data. In particular, even when the nodes are fixed to definite locations, criteria such as edge angular resolution (Brandes et al., 2000) and the number of edge bends may be still be optimized, producing a better visualization (Figures 7.8 and 7.9). Optimizing on aesthetic criteria is particularly effective where specialist requirements are present. For example, criteria might be developed particularly for drawing public transport routes; (see Fairbairn, this volume (Chapter 26)). Harry Beck's famous tube map provides inspiration, as he straightened out train lines, evened out distances between stations and altered the area of the map to fit within a poster. This is generally accepted as a much clearer way of enabling the public to plan their routes. Generalizing these features might allow the development of a multi-criteria optimizer for public transport schematics. Schematic generators have already been developed for other application areas, as shown in Figures 7.10 and 7.11.
7.6
Dynamic Graph Drawing
Graphs often change over time. This may be due to a user editing a diagram, or more commonly in Information Visualization, there is some alteration in the underlying data structure. For example, a real time map of the Internet will be constantly changing as
Figure 7.8. A graph with no edge crossings, even edge length and a 1:1 aspect ratio, but with poor node distribution and poor angular resolution.
152
P. Rodgers
Figure 7.9. Graph created with Ilog Jviews (Sander and Vasiliu, 2001). It shows links between French towns, using a direct edge routing method. This also shows the application of labelling techniques to graph drawing (Kakoulis and Tollis, 1996; Neyer, 2001).
Web pages and other documents are added, modified and removed. The challenge here is to retain the users' mental map of the graph (Eades et al., 1991). A mental map is the current view the user has of the graph. For complex graphs, a user has to invest considerable time and effort into understanding the current layout, so that any changes should be integrated carefully and with minimal disruption to this mental map. Some iterative techniques, such as the spring embedder can do this naturally to some extent, but this relies on the particular technique being first used to draw the graph. There is a need for research that analyses and defines the user's current mental map, before deciding on the best method for making layout alterations, perhaps drawing on related work in cognitive aspects of Cartography (Lloyd, 2000; Perusich and
Figure 7.10. The diagram on the left (a) shows a spatially exact representation of a cable plan. On the right, (b) shows the same cable plan, but using an automatically generated schematic representation (Lauther and StL~binger, 2001).
Graph Drawing Techniques for Geographic Visualization
Figure 7.11.
153
Examples of dynamic graph drawing - the node "X" has been added.
McNeese, 1997; Raubal et al., 1997). Figure 7.11 shows two possible rearrangements of a graph after the node "X" has been added to the graph. Figure 7.1 l a (left) assumes that a hierarchical algorithm has been used to draw the original graph, and that node "X" is a child of node "a". It attempts to find space on the same level as the other children of "a", without disturbing any other nodes. Figure 7.11b (right) assumes that the spring embedder has been used to draw the original graph and so has produced a layout after moving the node "a" to make space for the additional node. Although the drawing is nicer, the movement of the nodes may have disturbed the user's mental map.
7.7
Drawing Graphs in Three Dimensions
Whilst it is necessary in a document of this sort to display images in two dimensions, there are many methods that draw graphs in three dimensions. These are typically variants of the 2D techniques described above (Munzner and Burchard, 1995; Wills, 1999). For example, force directed approaches require very few modifications to deal with the extra dimension. However, not all graph drawing methods can be applied in three dimensions. In particular, planar drawing methods by their nature are very dependent on being drawn on the plane. Others need considerable adjustment, including those that drawn by measuring aesthetic criteria, as some criteria are not appropriate to three dimensions, such as measuring the number of edge crossings. In addition, new criteria may need to be developed, in particular in relation to viewpoints. Viewpoints (Eades et al., 1997) are a significant difference between 3D and 2D graph drawing. A viewpoint is a 2D projection of the 3D graph. This is of obvious importance, as with current display technology, a 3D layout must be visualized in two dimensions, and the choice of which viewpoint affords the best comprehensibility is a major issue. When presented statically, graphs that occupy 3D techniques have very little advantage, and many disadvantages over those created with methods designed for 2D display (see Ware and Plumlee, this volume (Chapter 29) and Fabrikant and Skupin, this volume (Chapter 35)). However, 3D layouts become more useful when the user can interactively explore the graph (Ware and Franck, 1996). Typically, as users rotate, pan and zoom around a graph, they gather a 3D model of the graph, allowing the structure to become clear. This sort of interactive system can allow complex graphs to be analysed
154
P. Rodgers
Figure 7.12. A map of a web site, drawn with a 3D spring embedder method (Mutton and Rodgers, 2002). The two images are alternative projections of the 3D map onto the 2D page, effectively showing the 3D map from two different viewing positions.
Figure 7.13. A graph rendered with fillets (Lukka et. al., 2002). Fillets are curved joins between edges and nodes, and so help distinguish between edges that connect to nodes, and edges that do not connect to nodes, but where the nodes lie on top of the edges.
Graph Drawing Techniques for Geographic Visualization
155
effectively, whilst using a relatively small amount of screen real estate. Figure 7.12 shows the points of a single 3D graph layout from two different viewpoints.
7.8
Graph Rendering
Frequently in graph drawing research, graphs are treated as simple relational structures, circles connected by lines, with the only challenge relating to the layout of the nodes and routing of the edges. However, the comprehensibility of a graph not only relies on the layout but also how the nodes and edges appear in the final realization. Standard visualization techniques such as colouring nodes and edges, and the use of transparency to deal with occlusion are widely used, and there is some work on distinguishing edge and node occlusion by the use of filleting (Lukka et al., 2002) (Figure 7.13). However, much more research is needed in this area, particularly regarding application-specific rendering, and geographic visualization would be a prime candidate for study.
7.9
Conclusions
This chapter has briefly overviewed the current state of the art in graph drawing and discussed some applications in visualizing geographic data. A great deal of geographic data, both spatial and non-spatial, has a relational element and is likely to benefit from graph drawing methods. There are interesting potential research areas in the application of graph drawing to geographic visualization, such as the development and use of new aesthetic criteria for geographic visualization, the empirical analysis of graph drawing in geovisualization, and the improvement of graph rendering designed specifically for geographic data to support geovisualization.
References Brandes, U., and Wagner, D., (2002) "Using graph layout to visualize train interconnection data", Journal of Graph Algorithms and Applications, 4(3), 135-155. Brandes, U., Shubina, G., and Tamassia, R., (2000) "Improving angular resolution in visualizations of geographic networks", Joint EUROGRAPHICS and IEEE TCVG Symposium on Visualization (VisSym'O0). Amsterdam: Springer, pp. 22-32. Cheswick, W., Burch, H., and Brannigan, S., (2000) "Mapping and visualizing the Internet", 2000 USENIX Annual Technical Conference, San Diego, California, USA. Davidson, R., and Harel, D., (1996) "Drawing graphs nicely using simulated annealing", ACM Transactions on Graphics, 14(4), 301-331. DiBattista, G. D., Eades, P., Tamassia, R., and Tollis, I. G., (1999) Graph Drawing: Algorithms for the Visualisation of Graphs, New Jersey: Prentice Hall. Dodge, M., and Kitchin, R., (2000) Mapping Cyberspace. London, New York: Routledge. Eades, P., (1984) "A heuristic for graph drawing", Congressus Numerantium, 42, 149-160.
156
P. Rodgers
Eades, P., Lai, W., Misue, K., and Sugiyama, K., (1991) "Preserving the mental map of a diagram", Compugraphics, pp. 24-33. Eades, P., Houle, M. E., and Webber, R., (1997) "Finding the best viewpoints for threedimensional graph drawings", Graph Drawing, 97. Rome: Springer, pp. 87-98. Forrester, J. W., (1973). World Dynamics, Pegasus Communications. Fr6hlich, M., and Werner, M., (1994) "Demonstration of the interactive graph visualization system DaVinci", Graph Drawing, 94. Princeton, NJ: Springer, pp. 266-269. Himslot, M., (1996) "The Graphlet System", Graph Drawing, 96. Berkeley, CA: Springer. Hobbs, M. H. W., and Rodgers, P. J., (1998) "Representing space: a hybrid genetic algorithm for aesthetic graph layout", FEA '98 in Proceedings of JCIS'98, pp. 415418. Online: http://www.cs.ukc.ac.uk/pubs/1998/678/index.html Huang, M. L., Eades, P., and Cohen, R. F., (1998) "WebOFDAV - navigating and visualizing the Web on-line with animated context swapping", WWW7: Seventh International World Wide Web Conference. ILOG Inc. (2003) Graph Layout Package - ILOG JViews Component Suite. Online: http://www.ilog.com/products/jviews/graphlayout/(23/10/03). Kakoulis, K. G., and Tollis, I. G., (1996) "On the edge label placement problem", Graph Drawing, 96. Berkeley, CA: Springer, pp. 241-256. Kaufmann, M., and Wagner, D., (2001) Drawing Graphs: Methods and Models, LNCS 2025. Berlin: Springer. Lauther, U., and Stfibinger, A., (2001) "Generating schematic cable plans using spring embedder methods", Graph Drawing. Vienna: Springer, pp. 465-466. Lloyd, R., (2000) "Self organizing cognitive maps", The Professional Geographer, 52(3), 517-531. Lukka, T. J., Kujala, J. V., and Niemel, M., (2002) "Fillets: cues for connections in focus + context views of graph-like diagrams", Information Visualization 2002: IEEE, pp. 557-562. MacEachren, A. M., Brewer, I., and Steiner, E., (2001) "Geovisualization to mediate collaborative work: tools to support different-place knowledge construction and decision-making", Proceedings, 20th International Cartographic Conference, Beijing, China, August 6-10, 2001: ICA, pp. 2533-2539. Online: http://hero.geog.psu.edu/products/ICC 16009amm.pdf McCrickard, D. S., and Kehoe, C. M., (1997) "Visualizing search results using SQWID", In: WWW6: Sixth International World Wide Web Conference, Santa Clara, CA. Monmonier, M., (1991) "Ethics and map design: six strategies for confronting the traditional one-map solution", Cartographic Perspectives, (10), 3-8. Munzner, T., and Burchard, P., (1995) "Visualizing the structure of the World Wide Web in 3D hyperbolic space", Proceedings VRML'95 Symposium, Special Issue of Computer Graphics. San Diego, CA: ACM SIGGRAPH, pp. 33-38. Mutton, P. J., and Rodgers, R. J., (2002) "Spring embedder preprocessing for WWW visualization", Information Visualization 2002: IEEE, pp. 744-749. Online: http://www.cs.ukc.ac.uk/pubs/2002/1372/index.html
Graph Drawing Techniques for Geographic Visualization
157
Neyer, G., (2001) "Map labeling with application to graph drawing", In: Wagner, D., and Kaufmann, M., (eds.), Drawing Graphs: Methods and Models. Berlin: Springer, pp. 247-273. Perusich, K., and McNeese, M. D., (1997) "Using fuzzy cognitive maps to define the search space in problem solving", In: Salvendy, G., Smith, M., and Koubek, R., (eds.), Design of Computing Systems: Cognitive Considerations. Amsterdam: Elsevier Science, pp. 805-809. Purchase, H. C., Allder, J.-A., and Carrington, D., (2000) "User preference of graph layout aesthetics", Graph Drawing 2000, pp. 5-18. Raubal, M., Egenhofer, M. J., Pfoser, D., and Tryfona, N., (1997) "Structuring space with image schemata: wayfinding in airports as a case study", Spatial Information Theory A Theoretical Basis for GIS, International Conference COSIT '97, pp. 85-102. Roberts, J. C., (2000) "Multiple-view and multiform visualization", In: Erbacher, R., Pang, A., Wittenbrink, C., and Roberts, J., (eds.), Visual Data Exploration and Analysis VII, Proceedings of SPIE, pp. 176-185, Online: http://www.cs.kent.ac.uk/pubs/2000/963/index.html Sander, G., (1994) "Graph layout through the VCG tool", Graph Drawing, 94. Princeton, NJ: Springer, pp. 194-205. Sander, G., and Vasiliu, A., (2001) "The ILOG JViews graph layout module", Graph Drawing 2001. Vienna: Springer, pp. 438-439. Schnyder, W., (1990) "Embedding planar graphs on the grid", First ACM-SIAM Symposium on Discrete Algorithms (SODA '90), pp. 138-148. Sugiyama, K., Tagawa, S., and Toda, M., (1981) "Methods for visual understanding of hierarchical system structures", IEEE Transactions on Systems, Man and Cybernetics, 11(2), 109-125. Tom Sawyer Software (2003) Tom Sawyer Software. Online: http://www.tomsawyer.com (23/10/03). Tsai, V., (1993) "Delaunay triangulations in TIN creation: an overview and a linertime algorithm", International Journal of Geographical Information Systems, 7(6), 501-524. Tulip-Software.org (2003) Tulip Software Home Page. Online: http://www.tulip-software.org/(23/10/03). Tunkelang, D., (1998) "JIGGLE: Java interactive graph layout algorithm", Graph Drawing, 98. Montreal: Springer, pp. 413-422. Tutte, W. T., (1963) "How to draw a graph", London Mathematical Society, 3(13), 743-767. Walshaw, C., (2000) "A multilevel algorithm for force-directed graph drawing", Graph Drawing 2000, pp. 171-182. Ware, C., and Franck, G., (1996) "Evaluating stereo and motion cues for visualizing information nets in three dimensions", A CM Transactions on Graphics, 15(2), 121-139. Watson, D. F., and Mees, A. I., (1996) "Natural trees - neighborhood-location in a nutshell", International Journal of Geographical Information Systems, 10(5), 563-572.
158
P. Rodgers
Wills, G. J., (1999) "NicheWorks - Interactive visualization of very large graphs", Journal of Computational and Graphical Statistics, 8(2), 190-212. Yee, K.-P., Fisher, D., Dhamija, R., and Hearst, M., (2001) "Animated exploration of dynamic graphs with radial layout", Proceedings IEEE InfoVis, pp. 43-50.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 8
Exploratory Visualization with Multiple Linked Views Jonathan C. Roberts, Computing Laboratory, University of Kent, Canterbury, Kent CT2 7NF, UK
Keywords: multiple linked views, coupling, linking, coordination, exploratory visualization
Abstract Exploratory visualization enables the user to test scenarios and investigate possibilities. Through an exploration, the user may change various parameter values of a visualization system that in turn alters the appearance of the visual result. For example, the changes made may update what information is being displayed, the quantity or resolution of the information, the type of the display (say) from scatter plot to line-graph. Furthermore, the user may generate additional windows that contain the visual result of the new parameters so they can compare different ideas side-by-side (these multiple views may persist such that the user can compare previous incarnations). Commonly these windows are linked together to allow further investigation and discovery, such as selection by brushing or combined navigation. There are many challenges, such as linking multiple views with different data, initializing the different views, indicating to the user how the different views are linked. This chapter provides a review of current multiple linkedview tools, methodologies and models, discusses related challenges and ideas, and provides some rudiments for coordination within a geovisualization context. The types and uses of coordination for exploratory visualization are varied and diverse, these ideas are underused in geovisualization and exploratory visualization in general. Thus, further research needs to occur to develop specific geovisualization reference models and extensible systems that incorporate the rich variety of possible coordination exploration ideas.
8.1
Introduction
This chapter advocates the use of many lightweight views that are linked together. They are lightweight in that they are: (i) easy to generate by the user, where the user does not spend unnecessary time and effort to explicitly link the new view to existing ones; and (ii) do not take many computer resources (e.g., memory, computation). Such multiple linked 159
160
J.C. Roberts
views (MLVs) enable the user to quickly view a scenario, compare it with previous realizations, examine properties such as dependencies and sizes, put this view to one side and try out another scenario. There are many good principles that can be learned from examining how other systems achieve this MLV exploration. In geovisualization, the explorer often generates many spatial or abstract representations. With such exploratory environments, the user is able (even encouraged) to take a hands-on approach to gain a deeper understanding of the underlying information. They may examine multiple different graphical realizations that reveal different aspects of the data. These principles are applicable to the geovisualization domain (indeed, many MLVs use spatial information databases to demonstrate the techniques). This chapter highlights current trends in MLVs. In order to provide an overview of different multiple-view exploration strategies, we start by placing the MLVs in context, then discuss exploration strategies and expand upon appropriate methods to enable interactive and effective investigation and management techniques that oversee and encourage the user to explore.
8.2
Current Themes in Exploratory (Multiple View) Visualization
When carrying out research, analysts often proceed by using an experimental cycle where the experiment is set up perhaps with some default parameters, the results are noted down, then the parameters are adapted and the results are compared with previous versions. Each new investigation enhances the analyst's knowledge and understanding. When starting the investigative process we may not know anything about the database let alone what questions to ask. DiBiase (1990) focusing on the role of visualization in support of earth science research, summarizes the research process as "a sequence of 4 stages: exploration of data to reveal pertinent questions, confirmation of apparent relationships in the data in light of a formal hypothesis, synthesis or generalization of findings, and presentation of the research at professional conferences and in scholarly publications". Gahegan, this volume (Chapter 4) offers a perspective for "the entire process of GIScience". The need for exploration techniques grows as the data become larger and more complex. In such cases, the important aspects of the data are smaller, in comparison with the whole, and specific details are more likely to be hidden in a swamp of elements. Thus, in general, exploration techniques allow us to sift through volumes of data to find relationships, investigate various quantities and understand dependencies. One method to achieve this exploration, which has been the trend in the recent years, is by "dynamic queries" (Shneiderman, 1994). These are highly interactive systems that enable the visualizations to be manipulated, dissected and interrogated. The user dynamically interacts with the visualization by adjusting sliders, buttons, and menu items that filter and enhance the data and instantly update the display. By doing so the "user formulates a problem concurrently with solving it" (Spence, 2001). For instance, what was once a dark dense black region on a scatter plot can be immediately changed into a colourful and meaningful realization (see Theus, this volume (Chapter 6)). Systems that use this technique include HomeFinder (Williamson and Shneiderman, 1992) and FilmFinder (Ahlberg and Shneiderman, 1994) both now regarded as seminal work on dynamic queries. Ahlberg and Wistrand (1995a,b) developed these techniques into
Exploratory Visualization with Multiple Linked Views
161
the Information Visualization and exploration environment system (IVEE). In one example, they depict an environmental database of heavy metals in Sweden; IVEE was then developed into the commercial Spotfire system (Ahlberg, 1996). Another early example is the "density dail" (Ferreira and Wiggins, 1990), where visual results were chosen dependent on the dial position. More recently, Steiner et al. (2001) provide an exploratory tool for the Web and the Descartes system (Andrienko and Andrienko, 1999a-f) both provide dynamic queries; these systems include map-based views linked to other views. As an alternative to adapting sliders and buttons (as used in dynamic queries), the user may directly manipulate the results; such direct manipulation may be implemented using brushing techniques (Ward, 1994) or methods that select to highlight or filter the information directly. Much of the original work was done on scatter plot matrices (Becker and Cleveland, 1987; Carr et al., 1987). Brushing is used in many multiple-view systems from multi-variate matrix plots, coplot matrices (Brunsdon, 2001) to other geographic exploratory analysis (Monmonier, 1989). One map based visualization toolkit that utilizes multiple views and brushing is cdv (Dykes, 1995). cdv displays the data by methods including choropleth maps, point symbol maps, scatter plot and histogram plots. Statistical and geographic views are linked together, allowing elements to be selected and simultaneously highlighted in each. MANET (Unwin et al., 1996), developed from the earlier tools SPIDER and REGARD, provides direct manipulation facilities such as dragand-drop and selection and control of elements in the display, for example. Moreover, other direct manipulation techniques allow the inclusion of manipulators and widgets; for example the SDM system (Chuah and Roth, 1995) provides the user with handles mounted on visual objects to control the parameters directly. Often the widgets are applied to the objects when they are needed and provide additional functionality. The widgets may be multi-functional, where different adornments provide
Figure 8.1. Diagram taken from the Waltz visualization system (Roberts, 1998a,b), showing the use of the Inventor Jack manipulators.
162
J.C. Roberts
specialized manipulation. Figure 8.1 shows a jack manipulator where the outer cubes allow rotation; both the horizontal plane and vertical tubes allow constrained planar translation. This manipulator is provided by Open Inventor libraries and integrated in the Waltz multiple-view visualization system (Roberts, 1998a,b). In the figure, the manipulator has been attached to an object that has been moved along the XZ plane (using the large horizontal rectangle). Other manipulators exist; for example, selection in Mondrian (Theus, 2002a,b) may be operated through the use of rectangle areas. In this tool, the user may modify the regions by selecting handles on the rectangles, multiple selection areas can be used at once, and the selected items are highlighted in related windows.
8.3
Strategies of Exploration
In any interactive visualization, the decision needs to be made as to where the information goes, that is, when the parameters are changed does the new visualization replace the old, get overlaid, or is it displayed alongside and in separate windows? Roberts et al. (2000) names these strategies replacement, overlay and replication, respectively. This is depicted in Figure 8.2. This fits in well with the design guidelines of Baldonado et al. (2000), who describe the rule of "space/time resource optimization", where the designer must make a decision whether to present the multiple views side-byside or sequentially.
8.3.1
Replacement
The replacement strategy is the most common and has some key advantages, that is, the user knows implicitly where the information is updated and what information has
Figure 8.2. There are three strategies of exploratory visualization that determine where the information is placed: replacement, replication and overlay.
Exploratory Visualization with Multiple Linked Views
163
changed. However, there are some major challenges with this strategy. First, there are problems by using such an ephemeral exploration environment. Information about previous experimentations is usually lost, the user cannot compare different graphical realizations side-by-side, and there is often little guidance as to the sensitivity of different parameters (i.e., whether a small change of a parameter will make a small change in the image, or in fact it makes a large amendment to the visualization). Second, there is a risk of losing navigation context. For example, when a user zooms into a subpart of the display the context of how the zoomed area fits in with the whole is lost. Some visualization systems overcome the transient nature of the display by storing past visualization commands (as data or variable values) in a database, such as Grasparc (Brodlie et al., 1993) and Tioga (Stonebraker et al., 1993). In the case of Grasparc, or HyperScribe (Wright, 1996) as implemented as a module in IRIS Explorer, the user can "roll back" to a predefined state and re-visualize the data with the "old" parameters. As in the case of HyperScribe these states are usually stored in a "history tree" where data arising from the experiment process is modelled in a tree structure and the user can alter parameters and roll back to previous versions (Figure 8.3). As the user explores, it can become unclear how the filtered, extracted and specialized information fits in with the whole. Methods such as animation and distortion help to keep this context. For example, animation is used in ConeTree (Card, 1996); in this instance, a selected node is brought to the foreground by animating the 3D tree (for an explanation and figure, (see Schroeder, this volume (Chapter 24)). The animation occurs long enough for the observer to see a continuation and short enough so that the user still observes the visual momentum. Moreover, there is a current trend towards generating detail-in-context views also known as Context + Focus displays (Lamping et al., 1995). Many implementations are non'linear magnification systems using methods such as those described by Keahey and Robertson (1997). They appear with a linear (and traditional) mapping in the centre or focus of the screen and squashed or distorted mapping outside the focus area. For example, Snyder (1987) generated various magnifying glass projections of the earth. Other people who use distortion to provide a clear field-of-view to an interesting object in three dimensions include Sheelagh (1997).
Figure 8.3. Diagram showing the history tree where data arising from the experiment process are modelled in a tree structure and the user can alter parameters and roll back to previous versions.
164
8.3.2
J.C. Roberts
Replication
Another way of working is to use a replication strategy for information exploration. In this strategy, various parts of the information, parameters or views are copied or duplicated and aspects are displayed in multiple ways and in different windows. Replication refers to the action of the experimenter who wishes to repeat an experiment or procedure more than once. Replication may be used to provide methodical or random repetition of the experiment to confirm or reduce the error of the results (by perhaps averaging the different findings) or to confirm the outcomes. Far too often a user relies upon one display, presenting data by their "favourite" visualization algorithm. However, they may be missing out on the richness of the underlying information. Hence by duplicating and replicating the displays and slightly adapting the parameters for the next incarnation the user is able to observe and compare the result of different scenarios and experiment with the detail of their data. Replication can be divided into two subcategories of usage: (i) the procedure - where the results that are generated by the change of parameters are displayed in separate windows; (ii) the course of action - where the same data may be presented by different mappings. These different forms of the same information are known as multiforms (Roberts et al., 2000). It is useful to display the results of a parameter change in a new window: the user can clearly observe and compare side-by-side the differences and similarities of the results. For example, the user may wish to explore different isosurfaces depicting alternative concentrations of some phenomena. If, as the user changes the threshold value a new window appears displaying the new isosurface then the user can easily observe (and compare) the varying concentrations from the current and previous explorations. As we shall see in w such a dynamic replication could provide a multitude of views. Such a viewexplosion could confuse, rather than support the user in their exploration tasks. Not only can different parameterizations be displayed in multiple windows, but also the same information may be displayed in multiple forms. By doing so the user may be able to see information that was previously obscured, or the different form may abstract the information to provide a clearer and simpler representation, or the different views may represent alternative interpretations on the same information (such as those given by different experts). Indeed, the alternative view may help to illuminate the first. Yagel et al. (1995) advocate the use of "...visualization environments that provide the scientist with a toolbox of renderers, each capable of rendering the same dataset by employing different rendering schemes". Consequently, the user may gain a deeper understanding of their data. For example, our eyes use binocular vision to present two slightly different observations of the same scene, which provides us with a rich depiction of the information. Certainly, we miss out when we look at one picture of something, such as a still photograph of an historic building, and we gain a better understanding of the size, colours, textures and details when we browse through many photographic pictures, fly through a virtual 3D model and view it from multiple viewpoints, and read written explanations from an interactive guidebook. Likewise, it is often beneficial to the data explorer to see the information from different perspectives and in different forms.
Exploratory Visualization with Multiple Linked Views
165
There are many advantages in using replication, for example, the separate views hold a history of the exploration, allow comparisons between images, and the multiforms may emphasize different aspects of the information. Replication should be encouraged (Roberts, 1998a,b). However, not many current systems inherently support many views and the module visualization environments, which can display the data in many representations, leave all the effort of duplication to the user. Indeed, such a replication strategy is possible in the module building visualization environments, such as AVS, IRIS Explorer and IBM Data Explorer (Williams et al., 1992). However, exploring the information in such a way with these tools requires copying and reconnection of multiple modules, and thus the replication strategy is not necessarily encouraged or easy to operate in these module-building environments. It is not a lightweight operation. The system itself should have the functionality to support multiple views, created with little effort from the user, managed appropriately by the system and automatically coupled to other views. Moreover, further understanding may be gained through linking and coupling of information. For example, selections that are made in one view can be reflected in other views, other operations such as zooming and rotation operations can be cordially applied to any associated view - hence the phrase MLVs.
8.3.3
Overlay
A third method of generating the visualization result is to overlay the visualization method in the same display. Overlays allow different visualizations to share the same coordinate space. Such a fan-in method allows different representations of the same information in the same display to be layered together. The advantage of this is that it is easy to understand each view in the context of the other, and the information may be readily compared. Different representation methods may be mixed together in the same view. For example, one view may include 2D pseudo-colour slices, surface representations, legends and useful annotations. However, when too much information is presented in the one view, or layered over a previous version, it may be difficult to select and navigate through or understand specific information. This may be because the presentation is too crowded and complex or that parts of the visualization are occluded. Indeed occlusion may be a problem in 2D visualizations as the objects may lay directly over each other. This may cause a misunderstanding of how many elements are in fact at a particular coordinate. Solutions such as the use of transparency or randomly jittering the points may help to clarify the depictions. Additionally, aggregation followed by different mapping techniques may be useful, as demonstrated by the sunflower plot of Dupont and Plummmer (2003). Obviously, the usefulness and appropriateness of the overlay method depends on the graphical visualization technique and the visualization tasks being used. Related work includes the excellent Toolglass and Magic lenses (Bier et al., 1993) widgets that allow the user to see through and focus on details of the display. Geospace (Lokuge and Ishizaki, 1995) usefully employs translucency between the layers, and Kosara et al. (2002) uses a semantic depth of field to blur layers to keep the context. D611ner, this volume (Chapter 16) uses texture-mapping methods to implement a lens effect that draws
166
J.C. Roberts
upon transparent layers. Moreover, Gahegan (1998) provides an example of an integrated display to achieve more complete integration of geovisualization views. Ongoing work on MANET (Unwin et al., 1996) is focussed on methods to overlay different plots on the same view. The challenge here is to develop effective overlays that enable the user to keep the context information, understand the depth of knowledge and not become overwhelmed by a complex visual representation. Specific challenges include how to effectively operate the overlaid views - does the interaction go through a view or is only the top view active? How is the user made aware that the views may differ in their data? How are the data linked to the data, and can it be coupled?
8.4
Multiple Linked Views
Linking and relating the information in one view to that of other views assists the user in the exploration process and may provide additional insight into the underlying information. Certainly, "multiple views should be coordinated" (Carr, 1999). As the information is explored and placed in separate windows, it is important that the relationships between the views and the context of how one view relates to another are maintained. Indeed, North and Shneiderman (2000b) in their user experiments discover that MLVs are beneficial and state that "the overview and detail-view coordination improved user performance by 30-80% depending on task". Such additional "overview" realizations provide context information that enhances the understanding of the associated view. Many different forms of information may be linked and coordinated. For instance, manipulation operations (such as rotation, translation, zoom, etc.) may be concurrently applied to separate views so as when one view is manipulated the other views respond appropriately to the same manipulation operations; the spatial position of a pointer or probe may be linked between multiple views; filter, query and selection operations may be simultaneously applied. Moreover, these operations need only affect the same information but, more interestingly, to collections of different information. Coordination and abstract views provide a powerful exploratory visualization tool (Roberts, 1998a), for example, in a 3D visualization, a navigation or selection operation may be inhibited by occlusion, but the operation may be easier using an abstract view. Fuhrmann and MacEachren (1999) describe the use of an abstract view to guide navigation in a 3D geospatial representation, ideas that are further developed by Fuhrmann and MacEachren (2001). Thus, a linked abstract view may be used to better control and investigate the information in the coupled view. Accordingly, there are different reasons for coordination. North and Shneiderman (1997) state there are two different reasons for using coupled views, either for selection or for navigation. Although Pattison and Phillips (2001) disagree by saying that there are additional forms of coordination other than selection and navigation, for example, "coordinating the data in preparation for the visualization such as sorting, averaging or clustering". Likewise, Roberts (1999) believes in a broader use of coordination, exemplified by the layered model (Roberts, 1999; Boukhelifa et al., 2003) where the user may link any aspect of the dataflow and exploration process.
Exploratory Visualization with Multiple Linked Views
167
Selection allows the user to highlight one or many items either as a choice of items for a filtering operation or as an exploration in its own right; this is often done by direct manipulation where the user directly draws or wands the mouse over the visualization itself (Cleveland and McGill, 1988; Ward, 1994). Becker and Cleveland (1987) describe this as a brushing operation. Examples, of systems that implement the brushing technique include XmdvTool (Ward, 1994), IVEE (Ahlberg and Wistrand, 1995a,b) and Spotfire tools (Ahlberg, 1996). Joint navigation provides methods to quickly view related information in multiple different windows, thus providing rapid exploration by saving the user from performing the same or similar operations multiple times. Objects, such as pointers, annotations or metainformation, may be coupled. For instance, the developers of the visualization input pipeline (VIP) (Felger and Schr6der, 1992) describe an example that displays several views of the data with the cursors linked together; movement of one pointer causes the others to move correspondingly. Other forms of navigation include data probing, as implemented within both LinkWinds (Jacobson et al., 1994) and KBVision (Amerinex, 1992), and changing the viewport information, as accomplished in SciAn (Pepke and Lyons, 1993) and Visage (Roth et al., 1996), which provide coordinated manipulation of 3D views.
8.4.1
Linking architectures
The study of coordination is interdisciplinary and there is much to learn from other disciplines. Taking the simplistic view of coordination being "sharing things" then we may learn from areas such as sharing hardware devices in a computer system or managing, delegating roles in a human organization or collaborative support, for example, (see Brodlie et al., this volume (Chapter 21)). For an in depth interdisciplinary view of coordination, see Olson et al. (2001). In this particular chapter, we focus on four models: Snap (North and Shneiderman, 2000a), presentation graphics (McDonald et al., 1990) and the View Coordination Architecture (Pattison and Phillips, 2001) and a Layered Model for Coordination (Boukhelifa et al., 2003). Andrieko et al., this volume (Chapter 5) provide an in depth discussion of software issues in geovisualization. The Snap conceptual model (North and Shneiderman, 2000a) takes a datacentric approach to coordination. It uses concepts from database design to provide the required interaction. Relational database components are tightly coupled such that an interaction with one component results in changes to other components. The Snap architecture is designed to construct arbitrary coordinations without the need for programming. However, Snap's user interactions are currently limited to "select" and "load", whereas exploratory visualization permits rich and varied interactions such as representation-oriented coordinations in addition to data-centric coordinations. McDonald et al. (1990) describe a constraint system based on the presentationgraphics programming model (Figure 8.4). In this system, lenses map the subjects (objects) in the database into their visual presentations counterparts, a user interacts with the presentation and the subjects get updated through the input-translator, and finally, a constraint system updates corresponding properties and updates any other related graphical presentations.
168
J.C. Roberts
i/o devices
Links
Scene
I node I
display
~<
I
input
I
I
I presentation I~
Input Translator
Subject
I
Lens
Subject
I
Subject
I
~'
i nod/no
Consistency / constraint
Figure 8.4. Presentation graphics programming model (McDonald et al., 1990).
Figure 8.5. (a) Depicts the traditional MVC pattern. The views reflect the current state of the model; the information held in the model is updated via the controller. (b) Shows the coordination model by Pattison and Phillips (2001) based on an MVC pattern, where the presentation component observes the model for changes and updates its display as necessary, the model component observes both the specification and data model components for change and changes to the specification component are propagated up.
Exploratory Visualization with Multiple Linked Views
169
Pattison and Phillips (2001) developed an architecture based on the model view controller (MVC) design pattern that originated in the Smalltalk architecture (Figure 8.5a). This pattern describes three objects: the model, view and controller, where the model holds the state of the process and publishes notifications to the views when its state changes, the view(s) reflect the state of the data model, and the controller updates the model with requests from external events. The MVC architecture inherently supports multiple views, and Pattison and Phillips (2001) have adapted the model for Information Visualization (Figure 8.5b). Where the presentation component observes the model for changes and updates its display as necessary, the model component observes both the specification and data model components for change modifications to the specification component are propagated up. This architecture fits in with the dataflow paradigm (Haber and McNabb, 1990). Rather than concentrating on the implementation architecture, our work has focussed on a layered approach that is based on the dataflow model (Roberts, 1999; Boukhelifa et al., 2003) and incorporates more layers than that of Pattison and Phillips (2001). In this approach, the coordination may occur between any parameter at any level of the visualization flow (Figure 8.6). Therefore, the user can link a broad range or aspects
Data Space in Layers World
Coupling across layers Real Life Object
.... s a m ~ ~ Data
-s a m ~ ~
Reference Data t
filter
<
~>
J-
......
Reference Data -L
filter
Coupled data changes, new datasets
J ......
Feature
Feature Set
<
~>
Feature Set
Coupled feature set: data alterations
Object
AVO - Abstract Visualization Object
<
~>
AVO - Abstract Visualization Object
Coupled Object and mapping alterations
......
Viewport
Image
<
~> t
Window
Window
<
~>
Coupled image transformations
,,, Image ......
Window
Coupled window management
Figure 8.6. The diagram shows a layer model, where many different forms of information may be linked and coordinated. For instance, manipulation operations (such as rotation, translation, zoom, etc.) may be concurrently applied to multiple views so as one view is manipulated the other views respond appropriately to the same manipulation operations, the spatial position of a pointer or probe may be linked between multiple views.
170
J.C. Roberts
between several windows, for instance, the view projection transformations can be shared (to co-rotate several 3D objects included in separate windows) or characteristics of the objects can be simultaneously changed (such as their appearance, colour, texture or position, etc.), or window-operations can be coordinated (such as moving, deleting or iconizing windows).
8.4.2
The role of MLVs in the exploration process
The exploration process may be described as a history-tree, indeed, even if the views are a result of a set of random thoughts, each view still relates in some way (however tenuous) to former investigations. Often the newest explorations are close to the former; this is the case especially if the user makes minor amendments to a copy of the previous view. Consequently, it is sensible to consider clusters or groups of closely related views. This can occur as "render groups" (Yagel et al., 1995) where different renderers are used to display the same data filtering (at an equivalent the level to the "Data Model" in Figure 8.5). Information within each render group may be straightforwardly related to each other such that default coordinations may be readily defined (Roberts, 1999). Generating multiple views from any part of exploration process may be useful; here the user keeps older versions of their investigations such that they can compare previous incarnations. They provide a context of the whole exploration process. However, linking outside render groups is challenging as some operations may not be generally applicable such as highlighting elements between two disparate data models when each contains a set of disparate non-intersecting elements. It is both possible and often beneficial to coordinate outside the render groups, for instance, multiple 3D worlds may be simultaneously rotated even if they contain dissimilar realizations. There is an advantage in grouping the multiple views together as Kandogan and Shneiderman (1997) discover through their evaluations: the user better understands the relationships in the views, and can more easily find and drill down to the important aspects of the display.
8.5
Linking and Coordination Concepts
All the aforementioned ideas allow many windows to be created and linked with other views, but, rather than arbitrarily creating and linking views there is usually structure in an investigation. Certainly, when developing a coupled visualization system there are many questions to consider about the coupling. What is being coupled? What are their types? What gets changed? How does the information change? It may be that some links do not make sense and in fact may confuse the user, especially in visualization applied to exploration. Therefore, there are many challenges and much research still to be done. We distil these ideas into some rudiments of coordination.
8.5.1
The rudiments of coordination
In essence, the linking of information between views may be described as "information sharing" For example, if two objects in separate windows were projected using the same shared transformation matrix then any change to that matrix would update both views
Exploratory Visualization with Multiple Linked Views
171
simultaneously. Accordingly, coordination may be thought of as in terms of program variables. Thus, using this analogy the links have the following elements: 9
9
9
9
9
9
C o o r d i n a t i o n entities details what is being coordinated. For example, it could be
aspects of the data, record, parameters, process, event, function, aspects of the window or even time. The type expresses the method by which the views are linked. Coordinating parameter values such as coupling binary threshold operations or selecting ranges may be implemented by sharing primitive types (float, integer, etc.) while other operations may use more complex data structures. Some form of translation (or casting) may be required to coordinate entities with different types. In addition to this translation function, it is often useful to allow more intricate functions, such as to allow entities to be related via an offset (or by some other relation). In virtual reality it may be useful to provide two 3D views with one being at ground level and the other tethered above; the tethered view could provide an overview and thus move correspondingly with the ground view, for example, (see D611ner, this volume (Chapter 16)). The types may also determine the directionality of the links whether unidirectional or bidirectional. C h r o n o l o g y details temporal aspects such as the persistence or lifetime of the coupling, that is, how long the coupling exists? For example, it may be that objects in the scene are coupled for a specific task and then uncoupled when the task is over. Incidentally, like program variables, persistence and scope are inherently related. Moreover, the coordination may be synchronous, asynchronous, reactive, and proactive. For example, it may be useful to join the rotation of two views, one from a fast and the other a slow renderer, such that the slower render gets updated at a lesser rate; additionally, the user may make and review a change, then decide whether to commit or cancel this operation. McDonald et al. (1990) describes these capabilities as markup and commit/cancel. S c o p e controls the "area" of the correlation, whether two specific views, many realizations, or all realizations are coupled within an exploration. For example, the render group scenario is equivalent to a local variable and the global variable would be equivalent to coupling every view in the exploratory session. G r a n u l a r i t y expresses how many entities may be connected together. For example, how many entities are coordinated, how many views are connected in one coordination operation. Initialisation indicates who creates a correlation, whether the user or the system. For example, in spreadsheet system it is possible to name particular views for specific operations, or by using a render group method it is possible to automatically correlate aspects of the views. There is a similar issue regarding the creation of the views themselves. Some visualization systems automatically create the visualizations from a database of knowledge (metadata information) and user requirements. The Vista tool (Senay and Ignatius, 1994), for example, creates appropriate visualizations by asking the user to list the variables in order of preference.
J.C. Roberts
172
Updating describes how and when the information within the views and child modules are updated and refreshed, such as lazy update, or greedy update or user initiated. This is similar to the cold/warm/hot-linking concepts mentioned by Unwin (2001). Cold linking allows an adjacent view to be coupled once and ignores any changes to the former view (similar to copying values rather than copying a formulae in a spreadsheet), warm linking allows the user to decide when to update, hot linking provides automatic and dynamic updating of the linked views. Moreover, the interface should reflect the current state, for example by shading out the out-of-date views. However, it may be that views depend on other views and if the user is relying on the data-history it may be prudent to allow the user to force the update when required. Currently, some general-purpose visualization systems do provide some of these rudiments, for instance, IRIS Explorer allows parameters to be coordinated through unidirectional events and more intricate functions may be formed using the p-func editor; however, IRIS Explorer does not provide bidirectional links and disallows simultaneously connecting the reverse linkage to inhibit circular event explosions taking place. In geovisualization, a good example of linking is that of the bi-directional link between ArcView and xGobi (Symanzik et al., 2000). Coordination is used in other geovisualization systems; the GeoVISTA Studio for example (MacEachren et al., 2001) incorporate some coordination features. Many systems provide an overview map to manage the manipulation of the whole (Steiner et al., 2001; Andrienko and Andrienko, 1999a-f). Additionally panoraMap (Dykes, 2000) allows panoramic photographs (georeferenced with GPS positions) and other information to be dynamically linked with an interactive map, other information such as key-points visited and qualitative and quantitative information collected on site are also shown by icons and symbols on the map. It is clear that there are many issues still unanswered regarding each of these rudiments, for example, are there specific rudiments for geovisualization? Or in general: does it make sense to coordinate different types together? And if so: what translators are required? How does the user recognize the scope of the coordination or indeed understand the persistence or recognize whether something is out-of-date? Moreover, many systems do not provide the full rich set of linking strategies that are possible.
8.6
Management of Views and Linkages
In addition to the linking concepts there are some subsidiary issues to consider, such as managing the views and linkages, placement of the views and temporal aspects.
8.6.1
Managing the MLVs
The essence of lightweight MLVs is that they are easy and quick to generate, but by supporting such a strategy the user may generate many views (that will create a view explosion) where many of the representations are only slightly different to the previous This creates two main problems. First, these many representations may easily clutter
Exploratory Visualization with Multiple Linked Views
173
the screen-space (there is a limited "real-estate" in any screen technology), and thus their needs to be either some form of restraint to guard the user from generating too many windows or management strategies to appropriately and automatically place each window (the latter is detailed in w Second, the user may also be confused as to "which image relates to which data-instance". The systems in the literature provide different solutions. One solution is to inhibit the number of views: Baldonado et al. (2000) provides a useful set of guidelines for using multiple views, and include the rule of parsimony - use multiple views sparingly. Another solution is to trade space by time. Spence (2001) discusses this solution and provides the idea of rapid serial visual presentation (RSVP); this allows the user to rifle through a set of objects analogous to flicking through the pages of a book in order to acquire some understanding of its content. This space/time trade off may be described as an overlay methodology. Finally, a good policy would be to use the three strategies (replacement, overlay and replication) together, allowing the user to replace certain instances and replicate when they need to achieve side-by-side comparisons. It is important that the user should clearly understand the relationship of how each view relates to each data model. Many systems display the history tree (on a workpane or canvas) allowing the user to rollback to previous versions (Brodlie et al., 1993; Wexelblat and Maes, 1999). Then the problem becomes how to relate the views with the canvas. This can be achieved using various methods. In the Waltz system (Roberts, 1998a,b), each window is labelled, relating it to its respective module on the work-pane. This is a hierarchical numbering scheme, like the sections of a book, and is used to name each view. The names are then displayed on the history tree. The spiral calendar (Mackinlay et al., 1994) provides a graphical solution by using lines to relate one window to another. There is still much work to be done in developing effective view management strategies for MLVs; whether managing the placement of the views, controlling a possible view explosion, or relating the view information to that of the exploration hierarchy.
8.6.2
View placement strategies
The placement of the many windows can have a significant impact on the usability of the system: it is an important human computer interaction issue. Overlapping windows can cause the user to spend more time arranging the windows rather than doing the task (Kandogan and Shneiderman, 1997), whereas the screen may not be large enough to display each required view simultaneously. There are different placement strategies described as follows. First, the user is given the responsibility to position, iconize and scale the windows. As it is often difficult to select and find occluded windows, the system provides a repository or toolbar to hold a list of the displayed windows. This may take the form of a list of the named views, collection of icons, or thumbnail representation of the current views. Second, the system holds the responsibility for placing the views on the screen. These "intelligent" interfaces tile (or tabulate) the windows such that they appear
174
J.C. Roberts
adjacently without overlap. Elastic views (Kandogan and Shneiderman, 1997) provide a good example; in this methodology, the windows are hierarchically placed on the screen and dynamically scaled to fill the available space. Alternatively, spreadsheet styles are becoming popular (Chi et al., 1998; Jankun-Kelly and Ma, 2001) where the views are positioned in a tabular formation. Furthermore, the strategy may depend on some aspect of the data exploration or some other metric. For example, windows could be scaled smaller if less important, implemented by a zoomable interface such as Pad-+-+ (Perlin and Fox, 1993), or presented in a scatter plot form where the placement of each is dependent on two variables, or hierarchically as in the Flip zoom technique (Holmquist and Ahlberg, 1997). Many of the current multiple view visualization systems hand the responsibility to the user, however, there is much benefit in structuring the position of the views relative to each other. Thus, strategies for positioning the views appropriately should be researched. Many questions remain including: are the requirements of an MLV visualization system very much different to that of a traditional windowing system?
8.6.3
Chronology, animation and timing in MLV
Many datasets are time dependent; their visualization in an MLV environment may be treated in different ways. The simple case is to generate an animation of the data. In the above terminology, each frame would replace the previous. Alternatively, each individual frame (or a sample of frames) may be displayed in a separate view (or stacked and overlaid in a single view). Coupling multiple-view animations would involve synchronizing the two streams. This may be at a fine granularity (e.g., tightly synchronizing each individual frame) or coarse granularity (e.g., synchronizing on specified key-frames). Additionally, it may be that there are objects animated or moving in the scene (such as people, planes or boats). It may be useful to couple one view to the moving object and provide another view of the whole environment. The linked view may be tethered such that it looks down on the object being moved (separated by an appropriate distance). For example, the GeoZui3D of Plumlee and Ware (2003) provide different "frame of reference coupling" methods that describe how the new view moves in relation to the animated objects.
8.7
Current Objectives and Challenges
Recent research has focussed on providing principles for multiple views (Baldonado et al., 2000) and examining linking methods such as Roberts' taxonomy of coordination (Roberts, 1999; Boukhelifa et al., 2003) and North's Snap-together system (North and Shneiderman, 2000a) that allows unforeseen combinations of coordinated visualizations. This research is opening the way for more expressive investigation environments that support the user in their task rather than distracting the user from their task. Currently many multi-view systems only really support a few views where the system determines what and how the information is linked. Thus, further research should focus on developing systems that utilize many lightweight views that are truly quick to
Exploratory Visualization with Multiple Linked Views
175
generate and automatically linked with other information and implicit to operate. Indeed, the system could be designed that would suggest or automatically generate other views that the user had not thought of using. The user may find these non-traditional views unfamiliar, but this unfamiliarity itself may provide a better understanding. There are many issues surrounding MLVs that are lightweight (some have been highlighted in this chapter). To develop an appropriate MLV system that utilizes these aforementioned concepts, it may be that the system needs to automatically generate the visualizations on behalf of the user, such as in the Vista system (Senay and Ignatius, 1994) or at least make it as easy as possible to generate further representations (Roberts, 1998a,b). Furthermore, if the system provides a diverse and functional-rich interface then the user may be overwhelmed by the nature of the system. Overall, a balance needs to be found both to generate the right amount of views for the task (whether they are by replacement, replication or overlay), and to provide an expressive linking mechanism that also restrains the user from performing incomprehensible and unprofitable coupling operations. In addition, more empirical research needs to take place on the different designs to evaluate what is useful. Kandogan and Shneiderman (1997) have evaluated the effectiveness of certain multiple view systems and North and Shneiderman (2000b) have looked at coordinated views. However, more studies are needed. It is well understood that the effectiveness of a particular system or design is highly dependent on the visualization or investigative task and the domain; to this end Baldonado et al. (2000) offers some guidelines, but it still remains unclear when the user should replace, replicate or overlay the information to gain the best understanding. The geovisualization domain poses many challenges (MacEachren and Kraak, 2001). Indeed, highly interactive systems have already been developed such as Descartes (Andrienko and Andrienko, 1999a-f), GeoVIBE (Cai, 2001) and cdv (Dykes, 1997a,b). However, further research is required to put in place the tools and techniques that will allow appropriate multiple-view exploratory geovisualization systems to be easily developed. We propose the following strands of research: 1. Specific geovisualization reference models and toolkits need to be developed that incorporate lightweight MLVs and include the rudiments of coordination. 2. The tools need to support dynamic queries and complex coordination operations enabling highly interactive context + focus navigation. 3. The developed systems need to be easily extensible that will allow the data from the ever increasing and diverse range of data to be suitably visualized. 4. Methods need to be developed that integrate a wide range of different presentation methods, thus, allowing the user to view the information from different perspectives and try out different scenarios.
Acknowledgements This work has been supported by EPSRC (grant reference: GR/R59502/01) entitled coordinated views in exploratory visualization (CVEV), http://www.cvev.org. Moreover, I acknowledge Peter Rodgers and Nadia Boukhelifa and other colleagues in the Kent Visualization Group for their help in this work.
176
J.C. Roberts
References
Ahlberg, C., (1996) "Spotfire: an information exploration environment", SIGMOD Record, 24(4), 25-29. Ahlberg, C., and Shneiderman, B., (1994) "Visual information seeking: tight coupling of dynamic query filters with starfield displays", In: Adelson, B., Dumais, S., and Olson, J. S., (eds.), CHI'94 Human Factors in Computing Systems, Boston, MA, 24-28 April. New York: ACM Press, pp. 313-317. Ahlberg, C., and Wistrand, E., (1995a) "IVEE: an environment for automatic creation of dynamic queries applications", Human Factors in Computing Systems. Proceedings of CHI'95. New York: ACM Press. Ahlberg, C., and Wistrand, E., (1995b) "IVEE: an information visualization and exploration environment", Proceedings International Symposium on Information Visualization, Atlanta, GA, pp. 66-73. A.I. Amerinex Inc., (1992) General Support Tools for Image Understanding. Online: http ://www. aai. com/AAI/KB V/KB V.html (23 / 10/03 ). Andrienko, and Andrienko, (1999a) Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(23/10/03). Andrienko, G., and Andrienko, N., (1999b) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999c) "Knowledge-based visualization to support spatial data mining", Proceedings of Intelligent Data Analysis. Berlin: Springer, pp. 149-160. Andrienko, G., and Andrienko, N., (1999d) "Making a GIS intelligent: CommonGIS project view", AGILE'99 Conference, Rome, April 15 - 17, pp. 19-24. Andrienko, G., and Andrienko, N., (1999e). Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(03/10/02). Andrienko, G. L., and Andrienko, N. V., (1999f) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Baldonado, M. Q. W., Woodruff, A., and Kuchinsky, A., (2000) "Guidelines for using multiple views in information visualization", Proceedings ofA VI '2000, pp. 110-119. Becker, R. A., and Cleveland, W. S., (1987) "Brushing scatterplots", Technometrics, 29(2), 127-142. Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose, T., (1993) "Toolglass and magic lenses: the see-through interface", Proceedings SIGGRAPH'93, Anaheim, CA, pp. 73-80. Boukhelifa, N., Roberts, J. C., and Rodgers, P. J., (2003) "A coordination model for exploratory multi-view visualization", Proceedings of Coordinated and Multiple Views in Exploratory Visualization (CMV2003), July 2003. Piscataway, NJ: IEEE. Brodlie, K., Poon, A., Wright, H., Branklin, L., Banecki, G., Gay, A., and Cai, G. (1993) "GRASPARC - a problem solving environment integrating computation and
Exploratory Visualization with Multiple Linked Views
177
visualization", Proceedings Visualization'93. Los Alamitos, CA: IEEE Computer Society Press, pp. 102-109. Brunsdon, C., (2001) "The comap: exploring spatial pattern via conditional distributions", Computers, Environment and Urban Systems, 25, 53-68. Cai, G., (2001) "GeoVIBE: a visual interface to geographic digital library", JCDL Workshop on Visual Interfaces, ACM/IEEE. Card, S. K., (1996) "Visualizing retrieved information: a survey", IEEE Computer Graphics and Applications, 3(16), 63-67. Carr, D., (1999) "Guidelines for designing information visualization applications", Proceedings of ECUE'99, Stockholm, Sweden, pp. 1-3. Carr, D. B., Littlefield, R. J., Nicholson, W. L., and Littlefield, J. S., (1987) "Scatterplot matrix techniques for large N", Journal of the American Statistical Association, 82(398), 424-436. Chi, E. H. et al., (1998) "Principles for information visualization spreadsheets", IEEE Computer Graphics and Applications, 18(4), 30-38. Chuah, M. C., and Roth, S. F., (1995) "SDM: selective dynamic manipulation of visualizations", Proceedings UIST'95. Cleveland, W. S., and McGill, M. E., (1988) Dynamic Graphics for Statistics. Belmont, CA: Wadsworth. DiBiase, D., (1990) "Visualization in the earth sciences, earth and mineral sciences", Bulletin of the College of Earth and Mineral Sciences, The Pennsylvania State University, 59(2), 13-18. Dupont, W. D., and Plummer, W. D. J., (2003) "Density distribution sunflower plots", Journal of Statistical Software, 8(3). Dykes, J. A., (1995) "Cartographic visualization for spatial analysis", In: Proceedings International Cartographic Conference, Barcelona, pp. 1365-1370. Dykes, J., (1997a) "cdv: a flexible approach to ESDA with free software demonstration", In: Proceedings of the British Cartographic Society 34th Annual Symposium, Leicester, UK, pp. 100-107. Dykes, J. A., (1997b) "Exploring spatial data representation with dynamic graphics", Computers & Geosciences, 23(4), 345-370, Online: http://www.mimas.ac.uk/argus/ICA/J.Dykes/ Dykes, J. A., (2000) "An approach to virtual environments for fieldwork using linked geo-referenced panoramic imagery", Computers, Environment and Urban Systems, 24(2), 127-152, Online: http://www.geog.le.ac.uk/jad7/CE&US/(6/21/1999). Felger, W., and Schr6der, F., (1992) "The visualization in-put pipeline - enabling semantic interaction in scientific visualization", Proceedings of Eurographics 92, Eurographics Association, pp. 39-151. Ferreira, J. J., and Wiggins, L. L., (1990) "The density dial: a visualization tool for thematic mapping", Geo Info Systems, 10, 69-71. Fuhrmann, S., and MacEachren, A. M., (1999) "Navigating desktop GeoVirtual environments", IEEE Information Visualization 99, Late Breaking Hot Topics Proceedings, San Francisco, CA, Oct. 23-28, pp. 11-14. Online: http://www.geovista.psu.edu/publications/amm/ammIV99.pdf
178
J.C. Roberts
Fuhrmann, S., and MacEachren, A. M., (2001) "Navigation in desktop geovirtual environments: usability assessment", 20th International Cartographic Conference Mapping the 21st Century, Beijing, pp. 2444-2453. Gahegan, M., (1998) "Scatterplots and scenes: visualization techniques for exploratory spatial analysis", Computers, Environment and Urban Systems, 22(1), 43-56. Haber, R. B., and McNabb, D. A., (1990) "Visualization idioms: a conceptual model for scientific visualization systems", In: Nielson, G., Shriver, B., and Rosenblum, L., (eds.), Visualization in Scientific Computing. Piscataway, NJ: IEEE Computer Society Press, pp. 74-93. Holmquist, L. E., and Ahlberg, C., (1997) "Flip zooming: a practical focus + context approach to visualizing large information sets", Procedings HCI International'97. Amsterdam: Elsevier, pp. 763-766. Jacobson, A. S., Berkin, A. L., and Orton, M. N., (1994) "LinkWinds: interactive scientific data analysis and visualization", Communications of the ACM, 37(4), 43-52. Jankun-Kelly, T. J., and Ma, K.-L., (2001) "Visualization exploration and encapsulation via a spreadsheet-like interface", IEEE Transactions on Visualization and Computer Graphics, 7(3), 275-287. Kandogan, E., and Shneiderman, B., (1997) "Elastic windows: evaluation of multiwindow operations", A CM Proceedings CHI'97: Human Factors in Computing Systems, Atlanta, GA. New York: ACM, pp. 250-257. Keahey, T. A., and Robertson, E. L., (1997) "Nonlinear magnification fields", Proceedings of the IEEE Symposium on Information Visualization: IEEE Visualization, Oct.1997, pp. 51-58. Kosara, R., Miksch, S., and Hauser, H., (2002) "Focus + context taken literally", IEEE Computer Graphics & Applications, 22(1), 22-29, (Jan~eb 2002). Lamping, J., Rao, R., and Pirolli, P., (1995) "A focus § context technique based on hyperbolic geometry for visualizing large hierarchies", Proceedings of CHI'95 Human Factors in Computing Systems, Denver, CO, May 1995. New York: ACM Press, pp. 401-408. Lokuge, I., and Ishizaki, S., (1995) "Geospace: an interactive visualization system for exploring complex information spaces", CHI'95 Proceedings. New York: ACM, pp. 409-414. MacEachren, A. M., and Kraak, M.-J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Society, 28(1), 3-12. MacEachren, A. M., Hardisty, F., Gahegan, M., Wheeler, M., Dai, X., Guo, D., and Takatsuka, M., (2001) "Supporting visual integration and analysis of geospatiallyreferenced statistics through web-deployable, cross-platform tools", Proceedings, dg.o.2001, National Conference for Digital Government Research, Los Angeles, CA, May 21-23, pp. 17-24. Mackinlay, J. D., Robertson, G. G., and DeLine, R., (1994) "Developing calendar visualizers for the information visualizer", Proceedings ACM UIST, Nov., 109-119. McDonald, J. A., Stuetzle, W., and Buja, A., (1990) "Painting multiple views of complex objects", ECOOP/OOPSLA'90 Proceedings. New York: ACM Press, pp. 245-257.
Exploratory Visualization with Multiple Linked Views
179
Monmonier, M., (1989) "Geographic brushing: enhancing exploratory analysis of the scatterplot matrix", Geographical Analysis, 21 (1), 81-84. North, C. and Shneiderman, B., (1997) A taxonomy of multiple window coordinations, Tecnical Report #CS-TR-3854. University of Maryland, Computer Science Dept. North, C., and Shneiderman, B., (2000a) "Snap-together visualization: a user interface for coordinating visualizations via relational schemata", Proceedings of Advanced Visual Interfaces 2000, Italy, pp. 128-135. North, C., and Shneiderman, B., (2000b) "Snap-together visualization: can users construct and operate coordinated visualizations?", International Journal of Human-Computer Studies, 53(5), 715-739. Olson, G. M., Malone, T. W., and Smith, J. B., (2001) Coordination Theory and Collaboration Technology, Lawrence Erlbaum Associates, Inc. Pattison, T., and Phillips, M., (2001) "View coordination architecture for information visualisation", Australian Symposium on Information Visualisation, Sydney, Australia. Darlinghurst, Australia: Australian Computer Society, Inc. Pepke, E. and Lyons, J., (1993) SciAn: User's Manual. Online: http://www.scri.fsu.edu/---lyons/scian (01/10/02). Perlin, K., and Fox, D., (1993) "Pad: an alternative approach to the computer interface", In: Kajiya, J. T., (ed.), Proceedings of the 20th Annual A CM Conference on Computer Graphics (SIGGRAPH'93), Anaheim, CA, Aug. 2-6. NY: ACM Press, pp. 57-64. Plumlee, M., and Ware, C., (2003) "Integrating multiple 3D views through frame-ofreference interaction", Proceedings of Coordinated and Multiple Views in Exploratory Visualization (CMV2003), July 2003. Piscataway, NJ: IEEE Computer Society. Roberts, J. C., (1998a) "On encouraging multiple views for visualization", Information Visualization IV'98. Piscataway, NJ: IEEE Computer Society, pp. 8-14. Roberts, J. C., (1998b) "Waltz - an exploratory visualization tool for volume data, using multiform abstract displays", Visual Data Exploration and Analysis V, Proceedings of SPIE, Jan. 1998, p. 11. Roberts, J. C., (1999) "On encouraging coupled views for visualization exploration", Visual Data Exploration and Analysis VI, Proceedings of SPIE, Jan. 1999, SPIE, pp. 14-24. Roberts, J. C., Knight, R., Gibbins, M., and Patel, N., (2000) "Multiple window visualization on the web using VRML and the EAI", In: Hollands, R., (ed.), Proceedings of the Seventh UK VR-SIG Conference, Sept. 2000, pp. 149-157. Roth, S. F. et al., (1996) "Visage: a user interface environment for exploring information", Proceedings of Information Visualization, 3-12. Senay, H., and Ignatius, E., (1994) "A knowledge-based system for visualization design", IEEE Computer Graphics and Applications, 6(14), 36-47. Sheelagh, M., (1997) "Extending distortion viewing from 2D to 3D", IEEE Computer Graphics and Applications, 4(17), 42-51. Shneiderman, B., (1994) "Dynamic queries for visual information seeking", IEEE Software, 11 (6), 70-77.
180
J.C. Roberts
Snyder, J. P., (1987) "Magnifying glass azimuthal map projection", The American Cartographer, 14(1), 61-68. Spence, R., (2001) Information Visualization. Harlow: Addison Wesley/ACM Press Books, 206 pp. Steiner, E. B., MacEachren, A. M., and Guo, D., (2001) "Developing and assessing lightweight data-driven exploratory geovisualization tools for the web", 20th International Cartographic Conference, Beijing, China, Aug. 6-10. Stonebraker, M., Chen, J., Nathan, N., Paxon, C., Su, A., and Wu, J., (1993) "Tioga: a database-oriented visualization tool", Proceedings Visualization'93. Piscataway, NJ: IEEE Computer Society Press, pp. 86-93. Symanzik, J., Cook, D., Lewin-Koh, N., Majure, J. J., and Megretskaia, I., (2000) "Linking ArcView and XGobi: insight behind the front end", Journal of Computational and Graphical Statistics, 9(3), 470-490. Theus, M., (2002a) "Geographical information systems", In: Kloesgen, W., and Zytkow, J., (eds.), Handbook of Data Mining and Knowledge Discovery. New York: Oxford University Press, pp. 232-241. Theus, M., (2002b) "Interactive data visualization using Mondrian", Journal of Statistical Software, 7 (11). Unwin, A., (2001) "R objects, two interfaces! (R objects to interfaces?)", Proceedings of the 2nd International Workshop on Distributed Statistical Computing. Unwin, A. R., Hawkins, G., Hofmann, H., and Siegl, B., (1996) "Interactive graphics for data sets with missing values - MANET", Journal of Computational and Graphical Statistics, 5(2), 113-122. Ward, M., (1994) "XmdvTool: integrating multiple methods for visualizing multivariate data", Proceedings Visualization'94. Piscataway, NJ: IEEE Computer Society Press, pp. 326-333. Wexelblat, A., and Maes, P., (1999) "Footprints: history-rich tools for information foraging", Proceedings of CHI 99, Pittsburgh, PA. New York: ACM Press, pp. 270-277. Williams, C., Rasure, J., and Hansen, C., (1992) "The state of the art of visual languages for visualization" Proceedings Visualization'92, Piscataway, NJ: IEEE Computer Society Press, pp. 202-209. Williamson, C., and Shneiderman, B., (1992) "The dynamic home finder: evaluating dynamic queries in a real-estate information exploration system", Proceedings of the ACM SIGIR'92 Conference. New York: ACM Press, pp. 338-346. Wright, H., (1996) The HyperScribe Data Management Facility, NAG Technical Report TRl-~96, NAG. Yagel, R., Ebert, D. S., Scott, J. N., and Kurzion, Y., (1995) "Grouping volume renderers for enhanced visualization in computational fluid dynamics", IEEE Transactions on Visualization and Computer Graphics, 2(1), 117-132. TM
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter 9
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour David Mountain, Geographic Information Science Group, Department of Information Science, City University, Northampton Square, London EC1V 0HB, UK
Keywords: collapsed real-time, geographic relevance, mobile telecommunication, mobile trajectory, spatial behaviour, spatial history explorer, spatio-temporal query, spotlights, time geography
Abstract Recent technological advances, in particular the increased ownership of mobile telecommunication devices, have led to the generation of vast quantities of spatiotemporal data recording the movement of individuals. Concurrently mobile devices are an increasingly important means of accessing information, adding a sense of place to Internet usage that has previously been relatively time and space insensitive. Mobile Internet users often have an information need, either to find a service or answer a query, provoked by some stimulus from the real world that they want to satisfy rapidly to assist decision-making. The interactive visualization of previous spatial behaviour can allow the simultaneous manipulation of space, time and attribute offering insight (at the exploratory stage of work) into underlying processes that may drive or constrain individual behaviour. This in turn assists in the development of algorithms to augment mobile information searches with spatio-temporal information derived from previous behaviour. Several interactive techniques and summaries of spatial behaviour are presented. An interactive tool for visualization of the movement of individuals (or other point entities) across space over time, the spatial history explorer, is described. Approaches for interactive spatio-temporal querying, visualizing mobility attributes and representing the cyclical nature of time are suggested. This approach of linking spatial and temporal representations is then contrasted with a single 3D spatiotemporal representation based upon a time geography approach. Finally, several surface representations that summarize large volumes of point data are described, in particular for identifying interesting locations associated with destinations, transport links and constraints. The insight gained from these techniques has led to several approaches to improving the way in which mobile individuals access digital information; this work is ongoing and described briefly in the conclusions. 181
182
9.1
D. Mountain
Introduction
Recent and continuing advances in mobile telecommunications technology have had a significant impact upon the nature and volume of information stored about individual spatial behaviour (Golledge and Stimson, 1997). Mobile telecommunication devices generate vast quantities of spatio-temporal data from logged network traffic recording when a mobile device, and by inference its owner, was in a particular network cell (Swedberg, 1999; Mountain and Raper, 2001a,b). The spatial accuracy of this information is increasing due to legal (Federal Communications Commission, 2002) and commercial imperatives promising detailed records of individual movement that can be made available with the permission of that individual. These vast, rich data sources offer great utility in numerous disciplines beyond monitoring spatial variation in quality of service (of network connection), the application to which they have thus far been applied. Information about the spatial behaviour of individuals has value theoretically in developing models of individual accessibility (Miller, 1991; Kwan, 2000a,b), in resource management for managers of natural recreation areas (WebPark Project, 2002), and in transportation planning to see (for example) how temporary migration may affect demand upon the transport infrastructure. Advances in the collection and processing of data has allowed the disaggregate approach (Frihida et al., 2002) to be used in transportation and consumer behaviour modelling considering the decisions made by individuals based upon the options available to them, in contrast to aggregate approaches that considered the needs of groups of people based upon distinct regions. Issues of privacy and availability of individual information (Barr, 2000) mean that an individual must consent to the release of this information for such modelling to take place. Individuals must feel that they are getting some service in return for granting this consent. Concurrently, mobile devices are an increasingly important means of accessing information via Web browsers and wireless connections, moving the Internet experience from static to mobile (Vincent, 1999). This is adding a sense of place that has been less obvious when searching for information from a static location such as an immobile desktop machine. Rather than accessing information sources that are relatively time and space insensitive, mobile Internet users often need to access information that is relevant to their physical location at that time, and to do so quickly since the mobile Internet tends to be a less absorbing environment than the static version (attention can be distracted by the external environment). People often have an information need either to find a service or answer a query provoked by some stimulus from the real world, which they want to satisfy rapidly to assist decision-making. Just as static users of the Web require filtering to avoid information overload, so users of mobile communications devices require techniques to ensure that the information they receive is relevant (Dykes and Mountain, 2003). Heuristics based on previous user behaviour could identify geographically relevant information (Raper, 2001) offering an incentive for individuals to overcome their concerns regarding privacy and release their positional data for analysis.
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
183
The nature and availability of this geographic information (GI) has contributed to a recent trend for increasing numbers of collaborations between information scientists and geographers, for example, (see Andrienko et al., this volume (Chapter 5)). The geographic component of information is often used to generate an organizational structure for a wide range of information and spatial metaphors are used to graphically represent a non-spatial information source (Fabrikant, 2000a,b; Fabrikant and Skupin, this volume (Chapter 35)). A highly interactive approach that allows the simultaneous manipulation of space and time can offer insight at the exploratory stage of work and altering the spatial and temporal scales of analysis can reveal different processes that may drive or constrain individual behaviour. This chapter describes efforts to visualize, query and summarize personal spatiotemporal data of this nature in the context of previous research. The spatial history explorer (SHE) has been developed specifically to analyze such datasets in a highly interactive environment that places equal importance upon both the spatial and temporal location of data. A dataset recording the location of a moving point object through time has been termed a mobile trajectory (Smyth, 2000), referring to a path created by that object across space through time. This path has also been called a world line (Raper, 2000), life path (H~igerstand, 1970), space-time path (Miller, 1991) and spatial history. Such mobile trajectories, as well as being recorded by mobile networks (in a way less conducive to retrieving the movement of individuals) can be downloaded from handheld GPS. This has been done by several volunteers at City University who have generated many months of GPS data, some of which will be used as examples in this chapter. Since GPS signals cannot penetrate masonry the dataset cannot be complete and sometimes "no data" can be taken as a proxy for no movement; algorithms have been developed to augment GPS datasets with data for time spent at destinations to provide a less biased mobile trajectory.
9.2
Background
Traditional Glsystems have received criticism for being most suited to an inventory role of objects with a spatial reference at a fixed moment in time (Langran, 1992; Wang and Cheng, 2001). If we accept that geographic space is a combination of space, time and attribute (Parkes and Thrift, 1980), then in order to be more than spatial information systems, Glsystems must include the temporal component and allow the modelling of dynamic processes. Increasingly, the temporal component is also being included in geovisualization processes (Andrienko and Andrienko, 1999a-f; Wang and Cheng, 2001; Dykes and Mountain, 2003) although the idea pre-dates the computing era with examples from the nineteenth century on paper (Tufte, 1983) and on the cusp of the digital age using film (Moellering, 1976). In order to develop effective on-screen realizations and interact with the temporal component, the nature of the spatio-temporal processes we aim to represent must be understood. Within the GI community, spatial data structures have been the focus of more previous research than temporal databases (A1-Taha et al., 1994), however more recent efforts to define a truly spatio-temporal Glsystem has forced researchers to consider the dynamic characteristics of the real world entities that they wish to represent.
184
D. Mountain
Claramunt and Th6riault (1996) define three main classes of spatio-temporal processes related to the evolution of an individual entity, dynamic relationships between multiple entities and the restructuring of complex entities. The classes are hierarchical, for example, evolution of individual entities comprises changes in existence, location and physical transformation (of shape or size). Sixteen subclasses are used in all to represent all possible spatio-temporal processes. Much parallel research is being conducted leading to representations of phenomena being developed that may only be relevant within one of the classes or subclasses of spatio-temporal process identified above. For example, land-use change can be represented by discrete events where complex entities are restructured internally (Peuquet and Duan, 1995) and dynamic geomorphological systems can be represented by succession of or transmission between multiple entities (Raper and Livingstone, 1995). These two examples are dependent upon an area representation whereas modelling individual mobility, the focus of this chapter, can be better represented by the movement of a point abstraction (Erwig et al., 1999) since the space occupied by individuals is negligible for most scales of analysis. A current focus for modelling entities as mobile point abstractions is in fleet monitoring for haulage firms and transportation systems (Cleary, 2000). Individual mobility is most suited to representation by stepwise change (defined by Wang and Cheng as an object that is sometimes moving and sometimes static) as opposed the discrete change (static with occasional, instantaneous changes in location) associated with changes in land-use or continuous change associated with many natural physical processes (Wang and Cheng, 2001). The use of a space-time or time geography framework (Miller, 1991) to analyze individual mobility can be traced back to Torsten H~igerstand of the Lund School who identified the importance of spatial and temporal constraints on individual behaviour (H~igerstrand, 1970). An individual's movement was realized as the trajectory of a point through a 3D space-time framework (or aquarium), where two horizontal planes represent spatial dimensions and the vertical axis represents time elapsing (Golledge and Stimson, 1997). This work was motivated in part by individual accessibility for transport planning, other locations that are accessible from any point for a given period of time can also be delimited based on the user's limiting maximum speed, resulting in the activity volume or "prism" (Lenntorp, 1976) defining the region of space-time accessible to them. Early paper-based realizations often took a graphics team several days to produce and offered no opportunity for user interaction. Similarly and in conjunction with the quantitative revolution in data collection and processing (Barnes, 2001), more studies in transportation modelling were moving from an aggregate approach (considering the individuals in a single traffic zone as a single entity) to a disaggregate approach, considering the transportation choices open to individuals (Wang and Cheng, 2001). Miller (1991) brought the time geography approach to a digital GIsystem, considering the suitability of GI technology for creating and querying space-time prisms and the approach has also been adapted to identifying differences in space-time constraints based on gender (Kwan, 2000a,b). An alternative approach to modelling individuals as mobile point objects was the continuous field methodology adopted
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
185
by Forer (1998) where space-time is represented as a 3D array with individuals occupying taxels (3D pixels where the horizontal plane represents spatial dimensions and the vertical axis the temporal dimension). A number of new ideas have been suggested in the literature and implemented in visualization software. For example, geographic traces allow users to define sequences of positions for which local statistics are generated and analysed in geographic data sets (Wills et al., 1989). Traces can be used to detect the geographic distributions of various attributes according to physical features such as rivers (Wilhelm and Sander, 1998). The sequences used in traces can be defined by sections of a spatial history as well as geographic features. Combining spatio-temporal activity with measurements of independent geographic phenomena in this way enables analysts to identify the dominant attributes of any routes taken through time and space. Relationships can be considered, requirements assessed and models generated from any insights that are achieved. Graphical representations of such statistics can be implemented through local parallel coordinates plots (Dykes, 1998). Each of these techniques could be used to analyse the variation in attributes across time and space as defined by a spatio-temporal user-profile. A recent, novel representation that combines user-centred and spatially distributed GI is the radial distance function (RDF) plot (Imfeld, 2000). The temporal RDF plots generated from such functions are explicitly designed to focus on sequences of positions in time and space by representing the variation in measured phenomena (such as land-use) at a series of distances from successive locations. The 2D plots use one axis to represent time and another to represent distance depicting the way in which the environment around a mobile object varies over time. This approach, developed for the study of animal behaviour, is an effective integration of the temporal characteristics of GI moving away from a purely space-centred method of analysis. Considerable success is reported in identifying patterns at a range of scales using temporal RDF plots (Imfeld, 2000), suggesting that they may be useful for a series of applications outside those relating to the study of wildlife and habitats such as associating them with mobile trajectories. Present implementations can take several hours to generate, hence such techniques cannot yet be integrated into real-time visualization software.
9.3
Interactive Visualization of Mobile Trajectories
SHE is an application that has been developed to interpret spatio-temporal point data representing the movement of individuals or entities across space through time. It has been designed to be used at the exploration and confirmation stage of analysis (DiBiase, 1990), thus to promote private visual thinking (MacEachren, 1994a-c). Initial objectives of the software include: 9 9 9 9
simultaneous on-screen realization of spatial, temporal and attribute data; representation of alternative temporal views to consider cyclical trends on a range of scales; facilitatation of interactive spatio-temporal querying; promotion of visual thinking about the mobile trajectory in order to conceptualise the types of information that might be generated from it
186
Figure 9.1. (see caption on page 187)
D. Mountain
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
187
The software provides a traditional planimetric "map view" (Figure 9.1) of the spatial information contained in a spatio-temporal log. In isolation, this approach is inherently static in nature (Wang and Cheng, 2001). A reciprocal "time view" (Figure 9.1b) displays absolute time plotted along the x-axis, to retain order for the data set, whilst the temporal y-axis can display the frequency of data collection or different temporal attributes representing cyclical trends for a range of temporal scales. A third panel, the "attribute view", has axes configurable by the user and hence can display a combination of spatial, temporal and attribute parameters. Figure 9.1c shows the relationship between heading and speed.
9.3.1
Representing mobility attributes
The shading of points in the linked spatial, temporal and attribute views can represent either spatial and temporal variables or other attributes that can be associated with mobility. Speed can offer an immediate overview of an individual's mobility at different locations and times as shown in Figure 9.1. Locations associated with low speeds or no movement highlight destinations, such as places of home, work or leisure pursuits, whereas sections of the mobile trajectory with higher speeds show the routes between those destinations. Shading by acceleration offers an opportunity to represent natural breakpoints in an individual's mobile trajectory, since the start and end of journeys are associated with acceleration and deceleration respectively; acceleration can also display the characteristics of journeys, whether they are smooth or involve a series of stops and starts. Shading by direction (using a colour wheel) helps identify "to" and "from" routes and asynchronous journeys. For example, routes to and from work may be different depending upon traffic restrictions, individual preferences and the constraints dictated by
Figure 9.1. The SHE map view, time view and attribute view. The data set shown is focused spatially upon South-West tip of Portugal and extends in time over two days in March, 2003. The points are not shaded by any attribute but low speed points (less than 2 m per second) have been highlighted in the attribute view; the views are linked and the corresponding points are also selected in the spatial and temporal views. (a) Map view (spatial distribution). Although no contextual background data are shown, the data set extends from Monchique in the North, Sagres in the SouthWest and Lagos (the temporary home location) in the South-East; the view is about 30 km wide. Notable hubs of activity can be detected at the aforementioned locations and also at Aljezur North of Sagres. These hubs tend to be associated with low-speed behaviour (as indicated by the lighter highlighted shade); the links between them are associated with high-speed behaviour. (b) Time view (temporal distribution). The time view shows absolute time (from 10:32 25 March to 16:23 26 March) on the x-axis and time of day on the y-axis. Each day elapses from midnight (at the bottom of the y-axis) to midnight (at the top of the y-axis); hence the daily trajectory from the bottom of the graph to the top is repeated every 24 h. Blank areas represent a lack of data collection associated with a lack of positional fix on the GPS receiver. It can be seen that most time is spent at low speed (the lighter highlighted shade) rather than at higher speeds travelling between destinations. (c) Attribute view (attribute distribution). This view can display a range of attributes on the axes; however, in this case the circular attribute heading (in degrees from North) is plotted on the orbital axis, against speed in metres per second on the radial axis. Heading is represented by the angle from North for each point relative to the centre of the plot. Speed increases with distance from the centre of the plot from 0 to 30 metres per second. The dragged circular section in the lighter shade indicates that the user has highlighted all points where speed is less than 4 metres per second.
188
Figure 9.2. (see caption on page 189)
D. Mountain
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
189
personal activity schedules at different times of the day. Direction can help establish whether there is a pattern to the differences between travel from A to B and from B to A. Shading points by point density allows magnitude to be added to the display that may be lost from over-plotting; SHE also allows the generation of point density surfaces for the same purpose (as described in w Point density gives an indication of an individual's familiarity with specific areas; whether it is their first time in a specific location, or a well-frequented route or destination (for some insights into interactive statistical techniques for large data sets, see Theus, this volume (Chapter 6)). Points can also be shaded by distance from a selected location to show patterns of return periods for that location in the temporal panel. Further development should include interactive techniques for changing the shading of points according to the distance from a clicked or dragged cursor in the spatial or temporal window. Such a technique would identify how "near" points are to a single point moving through space or time at a speed and direction determined interactively by the user. A range of further attributes associated with mobility and interactive techniques have been implemented and more still earmarked for development.
9.3.2
Emphasizing the temporal component
The inescapably linear nature of our experience of time (Hazelton, 1998) is an additional factor that must be considered for representations of individual spatio-temporal behaviour. When places were visited is important, as prior knowledge affects our behaviour considerably. By plotting absolute time along the x-axis of the temporal view (Figure 9.1), the SHE allows us to assess the way in which a user's behaviour changes as their experience of a new region increases. If detected, such information on changing behaviour could be used to predict future responses to visiting new areas (which are likely to be locations in which users have high information needs) and to determine likely information requirements. When tailoring location-based information services (such as tourist information systems) it is likely that the user will require introductory information and details of the main destinations on their first visit to an area. During subsequent visits more specific and detailed information may be required, such as points of interest that are "off the beaten track". A tool targeting these needs could establish information in a user profile that allows identification of the "beaten track". Beyond representing the ordering of the mobile trajectory by absolute time of collection, further temporal attributes can be symbolised to explore cyclical temporal patterns (Hazelton, 1998) that occur on a range of scales. As mentioned previously, the xaxis of the temporal view plots absolute time, however the y-axis can represent additional
Figure 9.2. The "temporal view" with the y-axis representing different temporal measures. Various temporal measures can be displayed on the vertical axis of this view. When brushing with linked views, this can give insight into differences in behaviour at different times of day, variation on a weekly temporal scale and seasonal patterns. (a) Daily distribution. Start of day (midnight) at the bottom to end of day (midnight) at the top. (b) Weekly distribution. Seven discrete categories from Monday (top) to Sunday (bottom). (c) Monthly distribution. Twelve discrete categories from January (top) to December (bottom).
190
D. Mountain
temporal variables such as time of day (Figure 9.2a), day of the week (Figure 9.2b) or the month (Figure 9.2c) to look for cyclical patterns at different temporal resolutions. When visualizing time of day, a complete dataset will show patterns of movement and static periods; the repetition of these patterns suggests that individual mobility is (to a degree) predictable for single individuals. The weekly cycle is a human construct rather than a natural phenomenon but can place greater constraints than any naturally occurring cycle on individual accessibility. A comparison of weekdays and weekends reveals that weekdays exhibit a far more restricted distribution associated with the schedule restrictions enforced by employment (focused upon the home and work locations and routes between them). Interactive highlighting of the weekend demonstrates far less restricted movement and more frequent longer distance travel away from the home location. Present data sets are not sufficiently detailed to look for cyclical behaviour on a monthly basis but this option has been implemented and trends dictated by the seasonal cycle can be uncovered for a suitable dataset. 9.3.3
Spatio-temporal querying
Interactive techniques can be extremely useful in reducing the complete dataset to a meaningful subset (Becker et al., 1987). In SHE, linked brushing between views is supported and views can be re-configured so that a subset can be derived by interactively selecting from particular spatial, temporal limits (Hornsby, 2001) or attribute limits. In each case, the selection is updated for all views of the data. This familiar concept of spatial focus (from a click and drag operation) can be extended for temporal focusing. Beyond selecting all data between two absolute start and end times, the y-axis of the temporal view can also be used to develop a more complex spatio-temporal query that restricts the selection according to the time of day, day of the week or for a range of months. For example a simple interactive click and drag function return and display the result of the query SUBSET BETWEEN "12:52 29/06/2001" AND "14:43 08/09/2001" FOR dayOfWeek -- {"Fri", "Sat", "Sun"} This interactive selection, taking seconds to perform, will return weekend results for the specified time period. By altering the y-axis parameter, the above query can be modified for selections according to time of day (e.g., restricting the subset to morning or afternoon behaviour) or month (e.g., seasonal distributions). Dealing with data interactively in this graphical and user-centred manner allows the analyst to gain an appreciation of any periodicity or pattern in the data and the temporal and spatial scales at which these forms occur through a process of visualization. Several subsetting operations of each type may be required to reduce a large data set in order to extract significant episodes and higher level information such as familiar and unfamiliar locations, ranges of movement, repeat and periodic behaviour. A subset operation has the disadvantage of not displaying the selected data in the context of the entire dataset, however the same interactive technique of brushing can be
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
191
used to highlight selected points. The views are linked so a simple enclosing rectangle selecting a (highlighted) set of points in the spatial view will also highlight these points in the temporal view. Being interactive this technique can select and unselect points by dragging with the mouse in real time, so can answer queries such as: 9
"what was the direction o f travel along this route?" (by advancing the selection
through time); 9 9
"when did this individual visit this area?" (by making a spatial selection); "in what areas does this individual spend their evenings?" (by restricted a
temporal selection to a specific time of day). This technique promotes data exploration and visual thinking in an interactive, user friendly way and can aid hypothesis formulation.
9.3.4
Space-time point clouds
The on-screen realizations produced by SHE display the spatial and temporal dimensions separately, each on a 2D panel. The same data sets can be displayed in 3D, where the horizontal axes (x, y) represent space and the vertical axis (z) represents time, akin to the aquaria in time geography described previously (Forer, 1998). Plotting an absolute linear time measure (such as milliseconds since 1970 or seconds since 1900) would result in a series of trajectories with vertical bars showing time spent at destinations and diagonal lines showing journeys between those destinations, analogous to the "collapsed real-time" animated sequence generated by Moellering (1976). Plotting a circular time measure such as the day of the week (from 1 to 7) will result in a point cloud where repeated patterns of behaviour may be displayed, analogous to Moellering's "composite" animation. An on-screen realization of a 10-month mobile trajectory for one individual created using Earthvision software (Dynamic Graphics Inc, 2002) is shown in Figure 9.3; the dataset has been clipped to only show data collected in central London, specifically the home and work locations. Time of day (as seconds past midnight) is plotted on the vertical axis and distinctive patterns are revealed at different times of the day, such as night-time behaviour, the journeys to and from work and evening behaviour. Plotting these circular trends as point clouds offers an effective summary of a typical day, week or summary for another user defined temporal scale of analysis. An interactive 3D approach where the temporal axis parameter could be manipulated, the views rotated, spatial and temporal subsets derived easily and multiple windows created would be a more integrated spatio-temporal approach over a 2D approach where the spatial and temporal dimensions are separated on different panels.
9.4
Data Synopsis with Continuous Surfaces
In w the SHE software was demonstrated in which patterns of spatio-temporal behaviour and cyclical temporal trends can be detected through a process of interactive brushing to perform spatio-temporal queries through data focus and highlighting. There are, however, problems associated with visualizing this raw point data including overplotting and perceived difficulties in detecting structure and form in large point clouds
192
Figure 9.3. (see caption on page 193)
D. Mountain
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
193
(individual data sets can contain up to 80,000 points). When plotted in a relatively small space it is both impossible to infer the magnitudes of the densities and difficult to visually synthesize this information with ancillary graphical data.
9.4.1
The spotlight metaphor and representations
One solution that addresses these problems is the use of density surfaces to provide estimates of point density at each location. The approach is relatively standard and has been applied in an exploratory data analysis (EDA) context to show point density in scatter plots (Haslett et al., 1990). A graphical advantage of the technique is that point density can be represented with a single visual variable, such as colour lightness, allowing other colour variables to be used for additional graphical information. In Figure 9.4a colour lightness is used to represent density. Figure 9.4b also uses this visual variable, but varies hue and saturation to reveal underlying exogenous information that adds context. We refer to these views as 'spotlights' (Dykes and Mountain, 2003) because of the way that they focus attention on the contextual information at locations with the greatest spatial familiarity. The dark areas represent locations of relative information paucity as inferred by the spatio-temporal log.
9.4.2
Surface feature networks
An opportunity exists for generating feature networks from continuous fields such as those modelled using density surfaces (Pfaltz, 1976; Wolf, 1984). These describe the topology of a surface and can provide useful summaries of form for visualization (Bajaj et al., 1998) that cannot be derived so easily from the raw, unevenly distributed point data itself. Such networks consist of peaks, pits, channels, ridges and saddles that determine the topology of continuous surfaces at specified scales. Detected peaks are likely to relate to locations that are visited often and pits to areas of relative local knowledge dearth. Ridges may relate to preferred or regularly used routes between peaks and channels to locations that are avoided. These features can be mapped and analysis of the relationship between these measured characteristics and the spatio-temporal record may reveal that the peaks, pits, channels and ridges of activity relate to some of the constraints imposed
Figure 9.3. A space-time point cloud: 10 months for Central London. In the images below, the x-axis plots longitude, y latitude and z the time elapsed since midnight (running from midnight to midnight, bottom to top); the points are shaded by speed from 0 to 80 kmph. The circular nature of the time value sees each day plotted in the same space rather than in a sequential linear fashion. (a) Complete data set, clipped to Central London, specifically showing the home location (solid vertical bar, foreground right) and work location (broken vertical bar, background left). (b) 00:00-06:30 Night behaviour. This period is usually spent at the home location; an anomalous late journey home from close to the work can be seen. (c) 06:30-11:00 Journey to work. The journey to work stands out during this, usually around 9am but with deviations on either side. Weekend behaviour is usually outside of this region. (d) 16:30-20:30 Journey from work. The return journey is seen within these temporal bounds but is less distinct. The deviations suggest the return is more likely to be part of a multi-task journey. (e) 20:30-24:00 Evening behaviour. The home location stands out here as do time spent at other locations within the work-home domain.
194
D. Mountain
Figure 9.4. Spotlights. Varying colour lightness to show density allows us to use colour hue and saturation to show exogenous information. This can help explain distributions by "throwing light" on the nature of the most frequently visited locations. Animated sequences of most frequently visited locations showing behaviour over different periods (see Figure 9.3) would result in shifting and panning foci. The overall contrast and brightness have been increased in this communicative example to show detail. (a) Spotlight showing weekly spatio-temporal log. (b) Spotlight with superimposed exogenous information. Crown Copyright Ordnance Survey. An EDINA Digimap/JISC supplied service.
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
195
upon our spatio-temporal behaviour introduced earlier. This knowledge may be usefully applied to address some of the key issues relating to the analysis and use of spatiotemporal logs. As the measures are dimensionless, basic graphical representations of the vectors generated from surface feature networks simply indicate the existence and location of these topological features, not their magnitude. Figure 9.5 demonstrates. Surface feature networks have great potential for emphasizing the information content of continuous fields, an important consideration when a user is presented with a large number of complex surfaces (Bajaj et al., 1998; Rana and Dykes, 2001). This is likely to be the case when performing the iterative process of visualization, which includes "selection". The networks can be generated at a range of spatial scales, a feature that can be used to identify different types of events in spatio-temporal data surfaces as illustrated in Figure 9.6. The kind of visualization generated has indicated that a series of surface feature networks covering a range of scales may summarize the information contained in large logs of spatio-temporal behaviour. Efforts to substantiate these findings and apply them are ongoing. 9.5
Conclusions
This chapter has provided an overview of visualization and interactive techniques for analyzing individual spatio-temporal information in the context of previous research. Mobile trajectories recording the movement of an individual across space through time are becoming more readily available through the proliferation of mobile communications and GPS devices. There is an opportunity to develop digital implementations of time geography (H~igerstrand, 1970) and accessibility modelling (Miller, 1991) techniques for the analysis of these data; researchers have often relied previously upon travel diaries or other secondary sources of individual mobility data. Spatio-temporal representations, such as dual or integrated realizations of spatial and temporal dimensions support our efforts to understand the underlying dynamic processes as opposed to querying a dataset that is "frozen in time" with no temporal referencing. Further abstractions of the dataset such as point density surfaces or surface feature networks allow the preservation of what is "interesting" about the data (Rana and Dykes, 2001; Mountain et al., 2003) but reduction of data volume. New views, advances in Information Visualization and interactive techniques that support geovisualization offer the opportunity to understand these rich data sources and use them effectively with a view to exploration, explanation, prediction and the provision of information according to need. The approaches to visualizing spatial, temporal and attribute information in an interactive environment described here allow spatio-temporal querying and promote visual thinking about the data to assist the development of data summaries and algorithms for knowledge extraction. Planned future research in this area will focus upon the development of automated algorithms to predict potential future destinations of mobile users and use this as another variable for ranking the results of mobile queries, and to identify times and locations when a particular individual is receptive to having information "pushed" to them; this work is progressing with an interface to agents that will utilize rules sets and knowledge extracted from these data sets (Mountain et al., 2003).
196
D. Mountain
Figure 9.5. Surface feature networks and spotlights derived from northern section of weekly log. Feature categories are symbolized with colour hue: peaks in red, pits in grey, ridges in yellow, channels in blue, passes or saddles in green. The densities (surface value) use colour lightness in "spotlight" fashion, highlighting the densest areas. The width of the border at the edge of the surface reveals the scale of kernel used to calculate the network of features. (a) Surface feature network derived from density surface. (b) Surface feature network and magnitude (spotlight).
Figure 9.6. Surface feature networks and spotlights. Surface feature networks and spotlights generated from northern section of weekly log. Deriving networks at a variety of spatial scales reveals different structural components in the data. Calculating the networks at a range of spatial scales results in the identification of features that relate to events at different spatio-temporal scales in this instance. The 500 m kernel reveals peaks at stopping points on a series of recreational walks recorded in the log. The peaks identified in the 900 m kernel highlight areas of repeat activity such as "home", quays and favourite locations for lunch. The channels define "no go areas" that are not navigable by boat due to shoals, or foot due to vegetation and access restrictions imposed following an outbreak of disease. The surface network derived from the 1500 m kernel identifies the locations at which most time was spent: "home" (the centre of the peak) and the main tracks along the island, describing the activity extremely succinctly. (a) 5 cell window (500 m). (b) 7 cell window (700 m). (c) 9 cell window (900 m). (d) 15 cell window (1500 m).
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
197
Such algorithms and summaries of extracted knowledge can be seen as an end product from the hypothesis formulation and testing period facilitated by the techniques described here. The development and integration of novel techniques in an interactive environment for the analysis of mobile trajectories remains a challenging and exciting prospect.
Acknowledgements I would like to express gratitude to the GIScience group at City University (especially Jason Dykes, Jonathan Raper and Jo Wood) for ideas and advice to the volunteers who have collected GPS data used for this research (notably Pete Boyd and Emma Greatbatch) and to members of the ICA CoVVEs (particularly Jason Dykes again, Menno-Jan Kraak, Alan MacEachren and anonymous reviewers) for comprehensive comments at various stages of drafting.
References A1-Taha, K. K., Snodgrass, R. T., and Soo, M. D., (1994) "Bibliography on spatiotemporal databases", International Journal of Geographical Information Systems, 8(1), 95. Andrienko and Andrienko (1999a). Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(23/10/03). Andrienko, G., and Andrienko, N., (1999b) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999c) "Knowledge-based visualization to support spatial data mining", Proceedings of Intelligent Data Analysis. Berlin: Springer, pp. 149-160. Andrienko, G., and Andrienko, N., (1999d) "Making a GIS intelligent: CommonGIS project view", A GILE'99 Conference, Rome, April 15-17, pp. 19-24. Andrienko, G., and Andrienko, N., (1999e). Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(03/10/02). Andrienko, G. L., and Andrienko, N. V., (1999f) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Bajaj, C. L., Pascucci, V., and Schikore, D., (1998) "Visualization of scalar topology for structural enhancement", In: Ebert, D., Rushmeier, H., and Hagen, H., (eds.), Visualization'98 (Oct). Research Triangle Park; NC: IEEE, pp. 51-58. Barnes, T. J., (2001) "Lives lived and lives told: biographies of geography's quantitative revolution", Environment and Planning D, 19(4), 409-430. Barr, B., (2000) GeoEurope. Online: http://www.geoplace.com/ge/2000/0700/0700ds.asp (July 2000). Becker, R. A., Cleveland, W. S., and Wilks, A. R., (1987) "Dynamic graphics for data analysis", Statistical Science, 2(4), 355-395.
198
D. Mountain
Claramunt, C. and Th6riault, M., (1996) "Toward semantics for modelling spatio-temporal processes within GIS", In: Kraak, M. J., and Molenaar, M., (eds.), Advances in GIS H, Delft, The Netherlands: Taylor and Francis, pp. 47-64. Cleary, B., (2000) "Satellite communications and fleet management-an integrated approach to the road transport industry", Journal of Navigation, 53(1), 20-24. DiBiase, D. (1990) "Visualization in the Earth Sciences, Earth and Mineral Sciences", Bulletin of the College of Earth and Mineral Sciences, The Pennsylvania State University, 59(2), 13-18. Online: http://www.geovista.psu.edu/publications/others/dibiase90/swoopy.html Dykes, J. A., (1998) "Cartographic visualization: exploratory spatial data analysis with local indicators of spatial association using Tcl/Tk and cdv", The Statistician, 47(3), 485-497. Dykes, J. A., and Mountain, D. M., (2003) "Seeking structure in records of spatiotemporal behaviour: visualization issues, efforts and applications", Computational Statistics and Data Analysis, 43 (Data Visualization II Special Edition), 581-603. Dynamic Graphics Inc., (2002) Welcome to Dynamic Graphics ... Home of EarthVision Software. Online: http://www.dgi.com (23/10/03). Erwig, M., Hartmut Gueting, R., Schneider, M., and Vazirgiannis, M., (1999) "Spatiotemporal data types: an approach to modeling and querying moving objects in databases", Geoinformatica, 3, 269-296, (Numb 3). Fabrikant, S. I., (2000a) Spatial Metaphors for Browsing Large Data Archives, University of Colorado-Boulder, Department of Geography. Fabrikant, S. I., (2000b) "Spatialized browsing in large data archives", Transactions in GIS, 4(1), 65-78. Federal Communications Commission, (2002) Enhanced 911 Initiative. Online: http://www.fcc.gov/911/enhanced (01/10/02). Forer, P., (1998) "Geometric approaches to the nexus of time, space and microprocess: implementing a practical model for mundane socio-spatial systems", In: Egenhofer, R. G., and Golledge, M. J., (eds.), Spatial and Temporal Reasoning in Geographic Information Systems. New York: Oxford University Press, pp. 171-190. Frihida, A., Marceau, D., and Theriault, M., (2002) "Spatio-temporal object-oriented data model for disaggregate travel behaviour", Transaction in GIS, 6(3), 277-294. Golledge, R. G., and Stimson, R. J., (1997) Spatial Behaviour: A Geographic Perspective. New York: The Guildford Press. H~igerstrand, T., (1970) "What about people in regional science", Papers, Regional Science Association, 24, 1-21. Haslett, J., Wills, G. J., and Unwin, A. R., (1990) "SPIDER: An interactive statistical tool for the analysis of spatially distributed data", International Journal of Geographical Information Systems, 4(3), 285-296. Hazelton, N. W. J., (1998) "Some operational requirements for a multi-temporal 4D GIS", In: Egenhofer, M. J., and Golledge, R. G., (eds.), Spatial and Temporal Reasoning in Geographic Information Systems. New York: Oxford University Press, pp. 63-73. Hornsby, K., (2001) "Temporal zooming", Transactions in GIS, 5(3), 255-272.
Visualizing, Querying and Summarizing Individual Spatio-Temporal Behaviour
199
Imfeld, S., (2000) Time, Points and Space - Towards a Better Analysis of Wildlife Data in GIS, Department of Geography, Universit~it Zfirich. Kwan, M. P., (2000a) "Gender differences in space-time constraints", Area - Institute of British Geographers, 32(2), 145-156. Kwan, M. P., (2000b) "Interactive geovisualization of activity-travel patterns using three-dimensional geographical information systems: a methodological exploration with a large data set", Transportation Research Part C - Emerging Technologies, 8(1-6), 185-203. Langran, G., (1992) Time in Geographic Information Systems. London: Taylor and Francis. Lenntorp, B., (1976) Paths in Space-Time Environments: A Time Geographic Study of the Movement Possibilities of Individuals, Department of Geography, The Royal University of Lund. MacEachren, A. M., (1994a) Some Truth with Maps: A Primer on Symbolization and Design. Washington: Association of American Geographers. MacEachren, A. M., (1994b) "Time as a cartographic variable", In: Hearnshaw, H., and Unwin, D., (eds.), Visualization in Geographical Information Systems. Chichester: Wiley, pp. 115-130. MacEachren, A. M., (1994c) "Visualization in modern cartography: setting the agenda", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Pergamon: Oxford, pp. 1-12. Miller, H., (1991) "Modelling accessibility using space-time prism concepts within geographic information systems", International Journal of Geographical Information Systems, 5, 287-303. Moellering, H., (1976) "The potential uses of a computer animated film in the analysis of geographical patterns of traffic crashes", Accident Analysis & Prevention, 8, 215-227. Mountain, D., and Raper, J., (2001a) "Modelling human spatio-temporal behaviour: a challenge for location-based services", Geocomputation'2001 - 6th International Conference on Geocomputation, University of Queensland, Australia, September 2001. Mountain, D., and Raper, J., (200 lb) "Positioning techniques for location-based services: characteristics and limitations of proposed solutions", Aslib Proceedings, 53(10), 404-412. Mountain, D. M., Goncalves, A., and Rodrigues, A., (2003) "Geographic knowledge discovery and spatial agent interfaces for location-based services", GIS Research UK, 9-11 April 2003. London: City University, pp. 206-209. Parkes, D., and Thrift, N., (1980) Times, Spaces and Places. New York: John Wiley. Peuquet, D. J., and Duan, N., (1995) "An event-based spatiotemporal data model (ESTDM) for temporal analysis of geographical data", International Journal of Geographical Information Systems, 9(1), 7. Pfaltz, J. L., (1976)"Surface networks", Geographical Analysis, 8(1), 77-93. Rana, S. S., and Dykes, J. A., (2001) "Augmenting animated maps with morphometric surface derivatives", GeoComputation. Brisbane, Australia: University of South Australia.
200
D. Mountain
Raper, J., (2000) Multidimensional Geographic Information Science. London: Taylor and Francis. Raper, J. F., (2001) "Geographic relevance", Meeting on Fundamental Questions in GIScience, Palace Hotel, Manchester, Online: http://www.spatial.maine.edu/(mduckham/GIScience/raper.html (1-2 July 2001). Raper, J., and Livingstone, D., (1995) "Development of a geomorphological spatial model using object-oriented design", International Journal of Geographical Information Systems, 9(4), 359. Smyth, S., (2000) "Mining mobile trajectories", In: Han, J., and Miller, H., (eds.), Geographic Data Mining. London: Taylor and Francis, pp. 337-361. Swedberg, G., (1999) "Ericsson's mobile location solution", Ericsson Reviews, 4, 214-221. Tufte, E. R., (1983) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, p. 197. Vincent, G., (1999) "Mobilising the net", lEE Review, 241-245, (November 1999). Wang, D., and Cheng, T., (2001) "A spatio-temporal data model for activity-based transport demand modelling", International Journal of Geographical Information Science, 15(6), 561-585. WebPark Project, (2002) webPark - Geographically Relevant Information for Mobile Users in Protected Areas. Online: http://www.webparkservices.info (23/10/03). Wilhelm, A., and Sander, M., (1998) "Interactive statistical analysis of dialect features", Statistician, 47(3), 445-456. Wills, G., Haslett, J., Unwin, A. R., and Craig, P., (1989) "Dynamic interactive graphics for spatially referenced data", In: Faulbaum, F., (ed.), Fortschritte der StatistikSoftware 2. Stuttgart: Gustav Fischer Verlag, pp. 278-287. Wolf, G. W., (1984) "A mathematical model of cartographic generalization", GeoProcessing, 2, 271-286.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 10
Impact of Data and Task Characteristics on Design of Spatio-Temporal Data Visualization Tools Natalia Andrienko, Gennady Andrienko & Peter Gatalsky, Fraunhofer AiS - Institute for Autonomous Intelligent Systems, Schloss Birlinghoven, Sankt-Augustin, D-53754, Germany
Keywords: exploratory data analysis, spatio-temporal data, geographic visualization, interactive data displays, animated maps
Abstract It is widely recognized that data visualization may be a powerful methodology for exploratory analysis. In order to fulfill this claim, visualization software must be carefully designed taking into account two principal aspects: characteristics of the data to be visualized and the exploratory tasks to be supported. The tasks that may potentially arise in data exploration are, in their turn, dependent on the data. In the chapter, we present visualization software tools for three different types of spatio-temporal data developed using a task-driven approach to design. We demonstrate that different exploratory tasks may be anticipated in these three cases and that different techniques are required to properly support exploration of the data. Prior to the consideration of the examples, we briefly describe the typologies of data and tasks we use in our work.
10.1
Scope and Perspective
This chapter offers a view on geovisualization from the perspective of computer scientists with an extensive experience in developing software tools for exploratory analysis of spatial data. Our tools are mostly based on combination of familiar techniques from various disciplines: Cartography, Statistical Graphics, Information Visualization, and Human-Computer Interaction. Traditional mapping and graphing techniques are enhanced with interactivity and manipulability. Typically, the ideas concerning useful technique combinations and enhancements come to us when we examine some specific datasets received from people interested in exploring these data. It is commonly recognized that techniques used for graphical representation of data must correspond to characteristics of the data (Bertin, 1983), and the same applies to software tools for visual data exploration. However, as we have learned from our 201
202
N. Andrienko, G. Andrienko & P. Gatalsky
experience, the route from data characteristics to the development of appropriate tools consists of two parts: first, data characteristics determine the potential questions (tasks) that may emerge in the process of the data exploration; second, the tasks make requirements of the tools and thereby define the space of possible design options. In this chapter, we advocate the task - analytical approach to the selection of appropriate visualization techniques and design of tools for the exploratory analysis of geographically referenced data. For this purpose, we offer three examples of geovisualization tool design for different types of spatio-temporal data. Prior to the consideration of the examples, we introduce the typological framework we use for revealing the set of potential tasks from the characteristics of datasets to analyze. We hope this material will be useful both for designers of geovisualization tools and for analysts applying existing tools to their data.
10.2
Related Work
The large number of recently published papers and books combining in their titles the words "Geography" and "time" or their derivatives indicates the importance of temporal issues for contemporary geographic information science. It can be noted that most publications refer in this or that way to the same cardinal problem "How to make computers (or, more specifically, GIS) understand temporality and handle time-related information?" Various formal theories have been suggested that attempt to simulate human's understanding of time and (spatio-) temporal reasoning (Allen, 1984; Galton, 1987; Egenhofer and A1-Taha, 1992; Cohn et al., 1998; Frank, 1998). On this basis, different frameworks and methods for internal representation and operation of spatiotemporal data in databases and GIS are devised (Langran, 1992; Peuquet, 1994; 2002; Worboys, 1998; Wachowicz, 1999). As developers of software tools for geographic data visualization in the sense defined in MacEachren (1994a-c); MacEachren and Kraak (1997), we focus primarily on another problem related to space, time, and computers: "How to make computers support a human analyst in visual exploration of spatio-temporal information?" While internal representation of spatio-temporal data is an important issue in implementation of tools, our main research interest is how the data should be displayed and accessible to a user. Our work on developing visualization-based exploratory tools was initially actuated by practical needs: we participated in several projects where different types of spatio-temporal data had to be sensibly presented to users. We studied the existing examples of geovisualization tools described in the literature (Kraak et al., 1997; Bloc et al., 1999; Fredrikson et al., 1999; Harrower et al., 2000; Oberholzer and Hurni, 2000; Slocum et al., 2000; and many others). Our attention was first attracted to the technique of map animation that can be found in almost all software systems for visualization of spatio-temporal data. However, we found soon that this technique in its "pure" form (i.e., playing a sequence of "snapshots" representing states of a phenomenon at successive moments of time) does not work as well for arbitrary data as it does in demonstrating evident trends like urban growth (Tobler, 1970). The main problem is that
Impact of Data and Task Characteristics
203
animation does not give an analyst an opportunity to compare directly states at different moments in time. In order to detect and evaluate changes, she/he has to compare the state viewed at the current moment with mental images of earlier states. Investigation of temporal trends would require memorizing a large number of consecutive states. Hence, comparison and trend detection must be supported by other techniques, possibly, combined with animation. We have failed to find a ready-to-use methodology for the design of geovisualization tools in the literature, i.e., a methodology that would link possible types of (spatio-temporal) data, possible types of questions about these data, or analytical tasks, and techniques that could support finding answers to these questions, although some systematization efforts have been undertaken by a number of researchers. Thus, one can find classifications of temporal maps on the basis of their contents (Muehrcke, 1978) or the methods of representing time (Kraak and MacEachren, 1994), consideration of "traditional" visual variables (Bertin, 1983), such as location, size, and color, from the perspective of their appropriateness for the representation of time (Kraak and MacEachren, 1994), and descriptions of new, "dynamic" variables, such as display date, frequency, and synchronization (DiBiase et al., 1992; MacEachren, 1995). Peuquet (2002) notes that this research is conducted within the paradigm of map semiotics, the original goal of which was to provide a set of rules for effective representation and communication of an intended "message". In data exploration, there is no message to be communicated, and hence, semiotic principles cannot be directly applied - they need to be extended or recast (Peuquet, 2002, p. 289-291). From our perspective, it is necessary to establish explicit links between visual and dynamic variables, on the one hand, and types of exploratory tasks on the other. Another shortcoming of this research is that it deals solely with representational forms and not interaction techniques. Some work exists that considers particular types of interactive tools. Kraak et al. (1997) present a theoretical framework for the design of temporal legends for animated maps, and Edsall et al. (1997) make an attempt to relate three types of temporal legends (linear, cyclic, and textual) to different types of data (attribute changes vs. location changes) and temporal queries ("when?", "how long?", and "how fast?"). All these are valuable nuggets of a general design methodology, which is yet to be assembled. Meanwhile, the development of spatio-temporal geovisualization tools is largely an empirical endeavor supported by evaluation, for example, (see Fuhrmann et al., this volume (Chapter 28)). In this chapter, we present a range of tools that we have designed for different types of data. The main purpose of the chapter is to demonstrate the interaction between data characteristics and the tasks that can emerge in data exploration, and the impact of these two factors on the requirements for visual data analysis tools that need to be developed. Although characteristics of the intended users are also one of the important factors affecting our work on tool development, a detailed consideration of this topic is beyond the scope of this chapter. After the description of each design example, we provide references to the literature describing other approaches to the visualization of similar data. Prior to the description of the examples, we introduce the typologies of spatiotemporal data and analytical tasks we use in our work.
204
N. Andrienko, G. Andrienko & P. Gatalsky
10.3
Typologies of Data and Tasks
Spatio-temporal data involve three major components: space (where), time (when), and objects (what) (Peuquet, 1994). Each component consists of specific elements that may have some attributes and be linked by various relationships. Existing data typologies refer to these components. Thus, it is conventional to classify spatial objects and phenomena according to their spatial distributional form into discrete and continuous (MacEachren, 1995). A continuous phenomenon is defined everywhere over the territory (e.g., population density or air temperature) whereas a discrete one occurs at distinct spatial locations or within restricted areas (e.g., deposits of resources). Discrete objects are usually further subdivided into point, line, and area objects. Non-spatial, or thematic properties of spatial objects are expressed through attributes, the latter being most often classified according to "level of measurement" into nominal, ordinal, and numeric (Bertin, 1983). Sometimes numeric attributes are further subdivided into interval and ratio measurements. Spatio-temporal phenomena are also classified according to their temporal properties, in particular, according to the type of changes that occur to them over time (Bloc, 2000): 9 9 9
existential changes: appearing, disappearing, reviving of objects or/and relationships; changes of spatial properties of objects (location, size, shape); changes of thematic properties, i.e., values of attributes.
Sometimes only one type of change takes place or is of interest to an analyst, but in many cases one needs to consider several types simultaneously. In existential changes, further diversity is possible depending on whether the duration of events is significant or not. An analyst may treat events (e.g., earthquakes) as instant when duration of an event is negligibly short in comparison to the length of the time interval under analysis or when time of event appearance is important but not how long it lasts. According to the three components comprising spatio-temporal data, Peuquet (1994) defines three basic types of possible questions about such data: 9
when + where ~ what: Describe the objects or set of objects that are present at
9
when + what ---* where: Describe the location or set of locations occupied by a
9
where + what ~ when: Describe the times or set of times that a given object or
a given location or set of locations at a given time or set of times. given object or set of objects at a given time or set of times. set of objects occupied a given location or set of locations. This classification parallels the notion of question types introduced by Bertin: "There are as many types of questions as components in the information" (Bertin, 1983, p. 10), the type being defined by what component is unknown. Such a component will be further referred to as "search target". Here we use the word "search" in
Impact of Data and Task Characteristics
205
an abstract sense, which is different from the traditional usage of this term in Cartography or psychology for denoting only search in space. A complementary division of questions proposed by Bertin is according to what he terms levels of reading: elementary, intermediate, and overall. Elementary questions refer to individual elements of data (e.g., individual places, time moments, and objects) while questions of the intermediate and overall levels address more general characteristics of a phenomenon (e.g., how it is distributed in space, how it behaves in time, or how characteristics are distributed over a set of objects). From our perspective, there is no principal difference between the intermediate and overall levels, as defined by Bertin. Both levels involve consideration of sets rather than individual elements. The difference is whether the whole set or its subsets are considered. It should be noted that Bertin considered data in general, not specifically spatio-temporal data. Koussoulakou and Kraak (1992) demonstrate that in the specific case of spatio-temporal data, the distinction according to the reading levels can be independently applied to the spatial and to the temporal dimensions of the data. For example, the question "What is the trend of changing values at location 1?" belongs to the elementary level in relation to the spatial component and to the overall level with respect to the temporal component. An analogous observation can be also made for the object dimension. Hence, each of the Peuquet's general question schemes of the form A § B---, X (where A and B denote known, or given, data components and X stands for unknown information) can be further subdivided according to the level on which the known information is specified: elementary A and B, elementary A and overall/intermediate B, overall/intermediate A and elementary B, and overall/ intermediate A and B. MacEachren (1995) and Kraak et al. (1997) classify possible questions concerning spatio-temporal data into seven query types addressing the existence of an entity (if?), its location in time (when?), its duration (how long?), its temporal texture (how often?), its rate of change (how fast?), sequence of entities (what order?), and synchronization (do entities occur together?). These types can be viewed as an elaboration of a more general task "describe the times or set of times that a given object or set of objects occupied a given location or set of locations" (where § what ---, when) in the classification suggested by Peuquet. While Bertin and Peuquet define possible information-seeking tasks in terms of components present in data, many researchers take another perspective by associating tasks with cognitive operations performed by a user. Different researchers consider quite diverse sets of cognitive operations, for example, Wehrend and Lewis (1990) distinguish eleven operations while Knapp (1995) suggests a set of four tasks: "identify", "locate", "compare", and "associate". It is not our goal to discuss and compare here all existing task typologies. We would only like to note that according to our observation, most of the "cognitive" task classifications implicitly involve differentiation of tasks on the basis of the search target and reading level. For example, identify differs from locate in the search target: in identify, the target is thematic characteristics of objects while in locate it is positions of objects in space and time; the task "distribution" from the classification of Wehrend and Lewis may be treated as identify or locate on the overall reading level.
N. Andrienko, G. Andrienko & P. Gatalsky
206
For our purposes, we found it useful to distinguish between two basic cognitive operations, identify and compare, where compare means establishing relationships of various kinds (including temporal relationships) rather than just examining similarities and differences. In particular, compare subsumes such temporal queries as "in what order do entities appear?" (sequence) and "do entities occur together?" (synchronization) (MacEachren, 1995). We regard the distinction identify vs. compare as one more independent dimension of the task space in addition to the search target and reading level. It should be further borne in mind that any general scheme acquires different shades of meaning when being applied to different types of spatio-temporal data. For example, when data about movement of discrete objects are analyzed, the three types of questions considered by Peuquet can be formulated as follows: 9
when + where ~
9
when + what ~
where:
what:
9
where + what ~
when:
What objects were present at the time t at the location 1? What was the location of the object o at the time t? When did the object o visit the location 1?
For data about changes of attributes of static spatial objects or locations (e.g., changes in population number and structure by municipalities of a country), the same schemes would be instantiated differently: 9
when + where ~
what:
What was the value of the given attribute at the time
t at the location 1? 9
when + what ~
where:
At what locations was the value v of the attribute
attained at the time t? 9
where + what ~
when:
When was the value v of the attribute attained at the
location l? Hence, potential information-seeking tasks substantially interact with characteristics of data. As a result, a tool designer needs to define the set of possible tasks individually for each particular type of data, possibly by adopting and adapting some general typology. In the remaining part of the chapter, we are going to demonstrate through a few specific examples of different spatio-temporal data, ways in which a designer analyzes data characteristics in order to determine possible types of questions that may arise and how this guides the choice of appropriate exploratory techniques to suggest to the users.
10.4
Time Controls and Dynamic Map Display
Despite the variety of spatio-temporal data, there are some general techniques applicable to all data types. Specifically, the spatial aspect of data is typically visualized with the use of maps, as they are well suited to conveying spatial information through human vision. Therefore, all our exploratory tools involve interactive map displays. For dealing with the temporal dimension of data, we have developed the time manager - an assembly of interactive widgets connected to a map display. These widgets are similar to those available in most of the systems supporting map animation. In particular, the time manager includes VCR-style buttons for starting and stopping animation and "stepping"
Impact of Data and Task Characteristics
207
through time. In addition to the standard functions, however, the time manager allows the user to choose between the instant view (the map represents the state of the world at a selected moment) and the interval view (the map represents events, movements, etc., that occurred during a selected interval). Both views can be combined with animation. In the interval view, the interval of the user-selected length will be shifted along the time axis by the specified number of time units on each step of the animation. While the time controls and the dynamic map display are common for all types of spatio-temporal data, the content of the map and the visualization methods used vary depending on the data type. For some data types, the dynamic map is combined with other exploratory tools, as will be shown below.
10.5
Visualization of Instant Events
Within the project "NaturDetektive" (NaturDetektive, 2003), schoolchildren from all over Germany registered through the Internet their observations of nature, specifically, when and where they had noticed a certain plant or bird. For some plants, the children had to distinguish different stages of development: appearance of first leaves, beginning of blossoming, or appearance of fruits. Our task in the project was to design and implement methods for visualization of the collected data that could be used, on the one hand in schools for educational purposes, and on the other hand, by project managers and interested public to examine children's involvement in the project. The observation data can be treated as instant events. The events differ in their qualitative characteristics: the species observed and, possibly, the stage of its development. Taking into account the peculiarities of the data, we anticipated the following types of questions: 1.
Elementary level (with respect to time): 9
What species were observed, and in which states, at the moment t around the location 1/in the area a? What species and states were predominantly observed over the whole territory at the moment t? Are there any differences in the variety of species observed in the north and in the south? etc. 9 Where was the species s (in the state s~) observedat the moment t? What was the spatial distribution of observations of the species s at the moment t? 9 When was the species s (in the state s~) observed around the location 1/in the area a? When did the largest number of observations of the species s occur? 9 What are attributes of a particular observation, e.g., who made it, when, in what environment, etc.?
2.
Intermediate/overall level (with respect to time): 9
How did the variety of species observed at the location 1/in the area a/over the whole territory change over time? 9 How did the occurrences of the species s (in the state s~) at the location 1/in the area a vary over the time? How did the spatial distribution of observations of the species s change over the time?
208
N. Andrienko, G. Andrienko & P. Gatalsky
9
What is the spatio-temporal behavior of the species s (i.e., when does it/its different stages appear in different parts of Germany, how long are the intervals between appearing of different stages, etc.)?
From the analysis of the possible questions we saw, first of all, the necessity of visual discrimination of observations of different species. Therefore, we chose to visualize the data on the map using iconic symbols with the shapes resembling the appearance of the species (Figure 10.1a). Variation of icon colors was used for representing different stages of plant development. Furthermore, seeking answers to the questions requires the following operations: 9 9 9
9
9
Selection of specific time moments (supported by the time manager). Focusing on particular locations or areas (supported by map zooming and panning facilities). Selection of a particular species. For this purpose we implemented a "species toolbar" in which the users can either choose one of the species (in this case the icons of the other species are hidden) or all the species at once. The second mode allows the users to study the total variety of species and its development over time. Access to information about a particular observation. To support this operation, we implemented a lookup interface that requires the user just to point with the mouse at the corresponding icon. All the information about this observation (date, species, state of development, and who made the observation) will be shown in a popup window (Figure 10.1 a). An observation record may have a reference to a URL where additional information about the species or/and the observer is given. This URL is opened when the user clicks on the icon. Observing changes over time. This operation is done using map animation controlled through the time manager. The animation allows the user to investigate, on the one hand, evolution of the variety of species or the spatiotemporal behavior of a particular species, on the other hand, how participation of the schoolchildren in the project developed over time.
Taking into account that the tool was intended for schoolchildren, we implemented a simplified version of the time manager, which can be seen in Figure 10.1 a on the right of the map below the legend. With this version, it is possible to select time moments (by clicking on the slider bar or entering dates in the text field) as well as start (resume) and stop the animation. However, if the user presses the button "Erweitert" ("Extended"), the time manager turns to its normal form providing more sophisticated operations (Figure 10. lb). The user can return back to the simple version by pressing the button "Simplifiziert" ("Simplified"). Although the resulting tool allows comparisons of different areas and distributions of observations of different species to be made, it does not support comparison of different time moments. For the latter purpose, one could suggest using two (or even more) parallel map displays. Another opportunity is to represent "older" events on the map in a specific way, for example, by "dimmed" icons. However, in this
m
! -'
,
. <
--' - 1- I -
n
.-
Figure 10.1. Visualization of nature observations. The time manager initially appears in its simplified form shown in the lower right corner of Figure 10.la. Figure 10.lb shows the appearance of the time manager after pressing the button "Erweitert >>" (Extended).
210
N. Andrienko, G. Andrienko & P. Gatalsky
particular case, we preferred to avoid further complication of the tool since it was intended first of all for children. For the same reason, we did not initially enable selection of arbitrary combinations of species through the "species bar". However, as the Naturdetektive project continued, we received a request from the organizers to add this opportunity and implemented it in a later version of the tool. Another improvement concerns dealing with overlapping symbols. In the latest version, when several symbols have close locations, they are replaced by a special "stack" symbol. Clicking on this symbol displays the list of observations it represents, and the user can select any of them for viewing the details. Unfortunately, the project organizers insisted on changing the method of depicting observations to using differently colored rectangular frames with numbers inside them instead of the iconic symbols. As a result, it has become hard to distinguish different species and states on the map. Therefore, we preferred to illustrate here with screenshots from the older version of the software. A useful addition to the described tools for exploration of instant events would be calculation and visual representation of various statistics: the total number of events that occurred at each moment/interval, the number of events of each kind (e.g., observations of each species), the average characteristics of events (in a case of numeric data), etc. Various other approaches to the visualization of events exist. The SpaTemp system (Stojanovic et al., 1999) combines computer-oriented techniques for visualizing events with traditional cartographic representation methods. In particular, the system can show the time that an event commences or the period of its existence through labels. The "age" of events may be represented by variation of colors. Fredrikson et al. (1999) describe, by example of traffic incidents, how data about events can be explored using various forms of data aggregation: spatial, temporal, and categorical (i.e., according to types of the events). The software displays summary characteristics of the aggregates, such as the total number of events or their average duration, and allows the user to "drill down" into each aggregate in order to see data about the individual events. Thus, summary data about spatially aggregated events (e.g., by road sections) are shown on an interactive map by symbols the size of which is proportional to the number of events. In our system CommonGIS, we apply the space-time cube technique (MacEachren, 1995), with the horizontal dimensions representing the geographical space and the vertical dimension representing time. Events are positioned in the 3D space according to their geographical locations and the times of appearance, for example, (see Mountain, this volume (Chapter 9)). Changes in the observed variety of species over time summarized over the whole territory or over selected regions could be represented using the "Theme River" technique (Havre et al., 2002).
10.6
Visualization of Object Movement
The most important types of questions that could be expected to arise in investigating movement of objects in space are the following: 9 9
Where was each object at a selected moment t? When did a particular object o visit the location 1?
Impact of Data and Task Characteristics
9 9 9 9 9
211
How long did it stay at this location? How did the positions of objects change from moment tl to moment t2? What were the trajectories of the objects during the interval (tl, t2)? What was the speed of movement during the interval (tl, t2)? How did the speed of movement change over time or with respect to spatial position?
The telemetric observations recording the migration of white storks to Africa in autumn and back to Europe in spring are example of data recording moving objects. For identifying and comparing trajectories, it is convenient to have the paths represented on a map by lines or arrows. However, static representation of trajectories is not appropriate for exploration of the speed of movement. Besides, when routes of several objects cross, it may be hard to determine whether the objects really met at the crossing point or just visited it at different time moments. Map animation may help to overcome the drawbacks of the static representation. There are three different variants of animated representation of object movement: 1. 2.
3.
Snapshot in time: at each display moment the map shows only the positions of the objects at the corresponding real-world moment. Movement history: the map shows the routes of the objects from the starting moment of the movement up to the currently represented moment. Hence, at the end of animation, the entire routes are visible. "Time window": the map shows the fragments of the routes made during the time interval of a specified length.
Our tool enables all three variants of animation. In our experiments, we found that the variant "snapshot in time" is suitable for exploring movement of a single object. With several objects, however, it is difficult to keep all the objects in the focus of attention. The variant "movement history", in which the current position of every object is graphically linked to its previous position, may prevent the analyst from losing track. However, after several steps of animation, the "tails" representing past movements often become very long or/and very complex (e.g., self-crossing) and distract the analyst from perceiving current movements. The "time window" animation mode cuts the "tails" and shows only a few movements preceding the currently represented moment. Thereby, the advantages of the movement history mode are preserved while the shortcomings can be reduced. The time window mode proved to be the most convenient for exploration and comparison of object behaviors in terms of the speed of movement. In this mode (Figure 10.2), arrow chains moving over the map represent path fragments made during time intervals of a constant length. The lengths of the chains thus show the distances passed during this extent of time and, hence, allow the analyst to estimate the speed of movement. Shrinkage of a chain in the course of animation signalizes that the movement of the corresponding object slows down, and expansion means that the movement becomes faster. When an object is motionless, staying for some time in the same place, the corresponding chain reduces to a single dot. By varying the length of the time window, it is possible to explore complex trajectories.
212
N. Andrienko, G. Andrienko & P. Gatalsky
Figure 10.2. The time window technique for animating object movement. The screenshots represent the appearance of a fragment of a map at six consecutive animation moments. The length of the time window is 5 days, that is, each screenshot shows route fragments passed by the moving objects (white storks) during 5 days. In the second and subsequent screenshots, the time window is shifted by one day forward relative to the preceding image. Note that movements of a particular bird did not necessarily occur every day.
We invite the readers to explore the movement of the storks by running the Java applet that is available online (Andrienko and Andrienko, 2003). A more detailed description and color illustrations can be found in Andrienko et al. (2000a,b). A useful enhancement of this visualization method would be a tool to synchronize presentation of movements made during different time intervals. This would allow an analyst to detect similarities in asynchronous behaviors and periodicity in movements. It would also be appropriate to calculate for each object the distance traveled during a selected interval or from the beginning of the observation to the current display moment and represent this for the analyst. A graph showing dependence of the traveled distance on the time passed could help in studying variation of the speed of movement and comparison of speeds of different objects. When we built the tool, migration data for just one year were available. Since then, more data have been collected, and now there is an opportunity to compare migrations in different years. Therefore, we are currently working on extending the tool so that such comparisons become possible. Various other approaches to the visualization of object movement exist. MacEachren (1995) and Peuquet and Kraak (2002) suggest that trajectories of object movement can be represented using the technique of the space-time cube. According to this technique, points in 3D space, where the vertical dimension corresponds to time, represent the positions of an object at different time moments. Lines connect the points corresponding to consecutive moments. A demonstrator can be seen online (Kraak, Undated). This technique, however, seems unsuitable for exploring the movement of multiple objects. Mountain and Raper (2001a,b) and Mountain and Dykes (2002) describe a software program called location trends extractor, or LTE, for analysis of routes. LTE represents a trajectory on a spatial display as a sequence of points colored according to the time when the corresponding locations were visited. Simultaneously, the data are summarized on a temporal histogram, which shows the number of visited locations by time intervals. The user can focus the analysis on interactively selected data subsets by applying spatial, temporal, or attribute selection criteria. LTE includes various computational procedures for analyzing route data, e.g., for breaking the movement history into periods of homogeneous behavior, revealing rapid changes in direction or speed, identifying places of interest, etc. The software was designed for analyzing
Impact of Data and Task Characteristics
213
the movement of single objects through time and space, rather than comparing multiple trajectories. Wilkinson (1999) describes how data about butterfly migration can be represented on a map using the SYSTAT graphics package.
10.7
Visualization of Changing Thematic Data
We now describe the tools we developed for visualization of temporal variation in thematic data associated with spatial objects, more specifically, values of a numeric attribute referring to areas of territory division. Examples of such data are demographic or economic indices referring to units of administrative division of a territory. By their nature, the data correspond to a continuous spatial phenomenon but have been discretized by means of dividing the territory into pieces. In contrast to events or moving objects, these pieces can be considered as stable objects that typically do not disappear and do not change their location. In the two applications described above, it was possible to show data for several different time moments on a single map. This created good opportunities for comparisons, detecting changes, and estimating the degrees of changes. With this particular data type, such combination is rather difficult since at each moment the objects cover the whole territory. A possible technique for representing data related to several moments on the same map would be drawing a bar chart (sequence of bars) in each area with heights of the bars proportional to the values of the attribute at these moments in this area. Such a map is convenient for examining changes of the attribute values in each particular area. However, it is unsuitable for considering the distribution of attribute values over the territory at any particular moment and for observing changes of the distribution over time. An overall view of a territory is well supported by the cartographic presentation method called the choropleth map. According to this method, the contour of each geographic object is filled with some color, the intensity of which encodes the magnitude of the value of the attribute. We combined this representation method with time controls and received a dynamic choropleth map display (Figure 10.3). This display provides a good overall view of the spatial distribution of attribute values at a selected time moment and dynamically changes when another moment is chosen, in particular, in the course of animation. However, the choropleth map is poorly suited for the tasks of estimating changes and trends occurring in each particular area and is even less appropriate for comparing changes and trends occurring in different areas. Therefore, we found it necessary to complement the dynamic map with an additional non-cartographic display, a time plot, showing the temporal variation of attribute values for each area (shown at the bottom of Figure 10.3). The technique of time plot is widely used in statistics. The X-axis of the plot represents the time, and the Y-axis - the value range of the attribute. The lines connect positions corresponding to values of the attribute for the same area at successive time moments. Like a map with bar charts, the time plot is good for examining dynamics of values for each individual object. In addition, it supports comparison of value variations for two or more objects and the detection of objects with outstanding behaviors (in terms of the variation of attribute values) much more successfully than a chart map.
214
N. Andrienko, G. Andrienko & P. Gatalsky
Figure 10.3. An interactive time-series plot dynamically linked to a map. Highlighted in the graph is the line corresponding to the municipality of Genoa, which is pointed to on the map by the cursor.
The earliest combination of a map with a time plot was Minard's famous presentation of Napoleon' s campaign in Russia (described, for example, in Tufte (1983)). In order to refer locations on the map to the marks on the graph showing the temperatures at the time moments when these locations were visited, Minard connected them with lines. In computer displays, other linking techniques are typically used. Thus, in our implementation, the time plot is sensitive to mouse movement: it highlights the line or bundle of lines pointed to with the mouse. Simultaneously, the corresponding objects are highlighted in the map. The link works also in the opposite direction: pointing to any object in the map results in the corresponding line being highlighted in the time plot (Figure 10.3). So, the map serves as a "visual index" to the plot making it easy for the user to focus on any particular object for considering dynamics of its characteristics with the use of the plot. And vice versa, it is easy to determine which object corresponds to any particular behavior that attracted the analyst's attention. Due to its interactivity and dynamic link to the map, the time plot is a useful analysis tool even despite the lines
Impact of Data and Task Characteristics
215
overlapping and being cluttered: highlighted lines are clearly visible, and the remaining lines provide a useful context for comparing the object(s) in focus with the other objects. Besides, it is possible to switch the display to the mode when only selected lines are visible. Furthermore, the time plot may be zoomed in the horizontal and vertical dimensions, also reducing line overlap. However, even when enhanced with the time plot, the map display still does not adequately enable the comparison of different moments and observation of changes on the overall level. To support such analytical tasks, we have devised a number of data transformation techniques supporting the comparison of the values for each moment... 9 9 9 9
with with with with
the values for the previous time moment; the values for a selected fixed moment; the value for a selected object; a constant reference value.
In the comparison mode, the map represents changes (computed absolute or relative differences between values) rather than the initial attribute values. This technique is known from Cartography as "change map". We represent the results of computations using a bi-directional color scheme (Brewer, 1994): shades of two different colors represent values higher and lower than the current reference value. The degree of darkness shows the amount of difference between the represented value and the reference value. White coloring is used for objects with values exactly equal to the reference value. The concept of the reference value has different meanings depending on the comparison operation selected. In the comparison with the previous time moment, the reference value for each object is the value of the attribute for the same object at the previous moment. So, it is easy to distinguish visually the areas with growth of values from those where the values decreased. Comparison with a fixed moment is achieved in a similar manner; the reference values in this case do not change with the change of the currently displayed time moment. When comparison with a selected object is chosen, the reference value for all the objects is the value for the selected object at the current moment, i.e., it is the same for all objects. The user sees which objects have lower values than this object, and which are higher. When the currently represented time moment changes, the new reference value comes into play, unlike the case with a constant reference value that may be only explicitly changed by the user. Visualization of differences can be combined with animation. On each step of the animation, the differences are recalculated and shown in the map. Two possible approaches exist to encoding time-variant data by degrees of darkness. We can assign the maximum degree of darkness to the maximum attribute value in the whole data set or to the maximum of the subset of data referring to the currently represented time moment. Each approach has its advantages: the first allows consistent interpretation of colors in successive images while the second shows more expressively value distribution at each time moment and makes changes in the distribution more noticeable. Therefore, we have implemented both approaches, and the user can switch from one of them to the other.
216
N. Andrienko, G. Andrienko & P. Gatalsky
As the user chooses the comparison mode and applies various variants of comparison, the time plot changes in accord with the map. Initially the time plot shows the source data, i.e., values of the explored attribute at each time moment. In the comparison mode it switches to displaying the results of subtraction of the reference values from the source values (absolute difference) or division of source values by the reference values (relative difference). Figure 10.4 demonstrates how the time plot from Figure 10.3 looks in different comparison modes. All four screenshots represent absolute differences. We continue to develop the tool described here. In particular, we are implementing temporal aggregation of numeric attribute data. The system will calculate and visualize on the map and on the time plot various summary statistics of data on intervals of a user-specified length: mean, median, minimum, maximum values, etc. In particular, averaging values on intervals will result in smoothing the time plot, which can help in the trend analysis. In the comparison mode, the aggregation will be applied to the calculated values. Various other approaches to visualizing thematic changes exist. Change maps, i.e., maps representing for each location or area the absolute or relative amount of change between two time moments, are known in traditional Cartography and used in other software systems, for example, Atlas of Switzerland (Oberholzer and Hurni, 2000) and MapTime (Slocum et al., 2000). In these two systems, it is also possible to analyze the variation of attribute values by comparing several maps corresponding to different time moments. The same technique is used in the Cancer Mortality Maps & Graphs of the USA National Cancer Institute (National Cancer Institute, 2003); however, here the data are aggregated by time intervals with the minimum length of 5 years. It is also possible to view the aggregated data on an animated map display. In addition to displaying data on maps, some software packages, for example, TEMPEST (Edsall and Peuquet, 1997) and STEM (Morris et al., 2000), can show temporal variation of numeric attribute values at selected locations on a time-series graph; however, these examples offer no dynamic link between the maps and graphs. Hochheiser and Shneiderman (2001) suggest sophisticated interactive tools for data exploration with an interactive time-series graph that being combined with a map display, would also be very useful for exploring spatio-temporal data. Analysis of temporal variation of numeric variables (attributes) is one of the most important research topics in statistics, and it is not surprising that statisticians are undertaking significant research work on visualization and analysis of spatio-temporal data. Thus, Unwin and Wills (1998) show how time-series data can be analyzed using interactive, transformable time plots (although not linked to maps). Carr et al. (1998) suggest the visualization method called Linked Micromaps. The method is based on dividing the set of geographical objects into small groups. Each group is represented on a separate small map accompanied by a statistical display, which may contain, in particular, time-series plots or box plots summarizing the value distribution over the whole time period. Mockus (1998) describes a tool consisting of a map view and a so-called Aggregation Eye, which allows the user to interactively select time intervals and ranges of attribute values. As a result, the data satisfying the query are aggregated and shown in the map using symbols varying in size. Alternatively, it is possible to display attribute
Impact of Data and Task Characteristics
217
Figure 10.4. Transformations of the time plot from Figure 10.3 in different comparison modes: (a) with the previous moment; (b) with a fixed moment, specifically, with the year 1983; (c) with a selected object, specifically, Genoa; (d) with a selected value, specifically, 10%. In all the screenshots, the line for Genoa is highlighted. In Figure 10.4d, the highlighted straight line corresponds to the selected value of 10%.
variation by line plots drawn at the corresponding locations on the map. Eddy and Mockus (1994) apply spatial and temporal data smoothing for building animated map visualization.
10.8
Conclusion
Visual exploration of spatio-temporal data requires tools that can help users find answers to different types of questions that may arise in relation to such data. In this
218
N. Andrienko, G. Andrienko & P. Gatalsky
chapter, we have shown that the questions an analyst is likely to be concerned with are closely related to the characteristics of data under analysis. Therefore, different sets of exploratory techniques are needed for different types of spatio-temporal data. We have considered three different types of data: instant events, object movement, and changing thematic characteristics of static spatial objects. We have described how we designed visualization tools to support exploratory analysis of each type of data. Thus, in analysis of movement the time window technique showing trajectory fragments made during intervals of a constant length is particularly productive. Various methods of data transformation and combination of a dynamic map display with a time-series plot are useful for analysis of thematic data associated with area objects. The time plot can also be recommended for representing calculated statistics for other types of spatio-temporal data, such as numbers of events or traveled distances. The types of data we considered here do not cover all possible variety of spatio-temporal data. Also, our intention was not to enumerate all possible techniques that can support exploration of each type of data. The main objective here is to demonstrate that development of interactive visualization tools must be data- and task-driven. It should be noted, however, that currently there is no appropriate theoretical and methodological background for such work, and this needs to be built. It can be noted that the tools we implement are mostly based on combination of familiar mapping and graphing techniques, which are enhanced with interactivity and manipulability. A tool may consist of several complementary data displays that are tightly linked in such a way that one display immediately reacts to user's actions in another display. A great flexibility in creating various technique combinations for different purposes (depending on data, tasks, and user requirements) can be achieved if each visualization technique is implemented as a component that can be included or removed without requiring any changes in the other parts of the system. Our system CommonGIS, which includes numerous visualization tools, is built according to this principle. The system has a centralized architecture: the tight linking of components is provided by the system's core, which propagates various events among the components attached to it. Hence, each component does not need to "know" about the other components. Another approach is implemented in GeoVISTA Studio (Takatsuka and Gahegan, 2001), where the components are autonomous and can be combined by explicitly linking their inputs and outputs using a graphical configuration tool. An alternative and comprehensive Java-based software development kit for data visualization called nViZn is now commercially available from SPSS Inc. The underlying ideas are described by Wilkinson et al. (2000, 2001). Such developments offer clear opportunities for creating flexible and usable visualization tools. The argument presented here is that data and task-driven approaches to designing and developing tools that draw upon these technologies will result in the most effective instruments for those investigating spatio-temporal phenomena. Doing so will enable us to develop an appropriate theoretical and methodological context for geovisualization.
Impact of Data and Task Characteristics
219
Acknowledgements The development of the techniques and tools for visual exploration of time-variant spatial data was partly undertaken within the project EuroFigures ("Digital Portrait of the EU's General Statistics", Eurostat SUPCOM 97, Project 13648, Subcontract to JRC 150891999-06) and is currently a part of the project SPIN ("Spatial Mining for Data of Public Interest", IST Program, project No. IST-1999-10536). We are grateful to Prof. Alan MacEachren for valuable comments on the early versions of the chapter.
References Allen, J. F., (1984) "Toward a general theory of action and time", Artificial Intelligence, 23(2), 123-154. Andrienko, G., and Andrienko, N., (2003) Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(03/10/02). Andrienko, N., Andrienko, G., and Gatalsky, P., (2000a) "Supporting visual exploration of object movement", Proceedings of the Working Conference on Advanced Visual Interfaces A VI 2000, Palermo, Italy, pp. 217-220. Andrienko, N., Andrienko, G., and Gatalsky, P., (2000b) "Towards exploratory visualization of spatio-temporal data", Proceedings of the 3rd Agile Conference on Geographic Information Science, Helsinki. Bertin, J., (1983) Semiology of Graphics: Diagrams, Networks, Maps. Madison, Wisconsin: University of Wisconsin Press. Bloc, C., Koebben, B., Cheng, T., and Kuterema, A. A., (1999) "Visualization of relationships between spatial patterns in time by cartographic animation", Cartography and Geographic Information Science, 26(2), 139-151. Bloc, C., (2000) "Monitoring Change: characteristics of dynamic geo-spatial phenomena for visual exploration", In: Freska, Ch., et al., (eds.), Spatial Cognition, LNAI 1849. Berlin, Heidelberg: Springer-Verlag, pp. 16-30. Brewer, C. A., (1994) "Color use guidelines for mapping and visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Vol. 2. Oxford: Elsevier Science Ltd, pp. 123-148. Carr, D. B., Olsen, A. R., Courbois, J. P., Pierson, S. M., and Carr, D. A., (1998) "Linked micromap plots: named and described", Statistical Computing and Statistical Graphics, Newsletter, 9(1), 24-32. Cohn, A. G., Gotts, N. M., Cul, Z., Randell, D. A., Bennett, B., and Gooday, J. M., (1998) "Exploiting temporal continuing qualitative spatial calculi", In: Egenhofer, M. J., and Golledge, R. G., (eds.), Spatial and Temporal Reasoning in Geographic Information Systems. New York, Oxford: Oxford University Press, pp. 5-24. DiBiase, D., MacEachren, A. M., Krygier, J. B., and Reeves, C., (1992) "Animation and the role of map design in scientific visualization", Cartography and Geographic Information Systems, 19(4), 201-214, see also pp. 265-266.
220
N. Andrienko, G. Andrienko & P. Gatalsky
Eddy, W. F., and Mockus, A., (1994) "An example of the estimation and display of a smoothly varying function of time and space - the incidence of mumps disease", Journal of the American Society for Information Science, 45(9), 686-693. Edsall, R., and Peuquet, D., (1997) "A graphical user interface for the integration of time into GIS", Proceedings of the 1997 American Congress of Surveying and Mapping Annual Convention and Exhibition, Seattle, WA, pp. 182-189. Edsall, R. M., Kraak, M. J., MacEachren, A. M., and Peuquet, D. J., (1997) "Assessing the effectiveness of temporal legends in environmental visualization", GIS/LIS'97, Cincinnati, Oct. 28-30, pp. 677-685. Egenhofer, M. J., and A1-Taha, K. K., (1992) "Reasoning about gradual changes of topological relationships", In: Frank, A. U., Campari, I., and Formentini, U., (eds.), Theories and Methods of Spatio-Temporal Reasoning in Geographic Space, Lecture Notes in Computer Science, Vol. 693. Berlin: Springer-Verlag, pp. 196-219. Frank, A. U., (1998) "Different types of "Times" in GIS", In: Egenhofer, M. J., and Golledge, R. G., (eds.), Spatial and Temporal Reasoning in Geographic Information Systems. New York: Oxford University Press, pp. 40-62. Fredrikson, A., North, C., Plaisant, C., and Shneiderman, B., (1999) "Temporal, geographic and categorical aggregations viewed through coordinated displays: a case study with highway incident data", Proceedings of Workshop on New Paradigms in Information Visualization and Manipulation (NPIVM'99). New York: ACM Press, pp. 26-34. Galton, A. P., (1987) Temporal Logics and Their Applications. San Diego: Academic Press. Harrower, M., MacEachren, A. M., and Griffin, A. L., (2000) "Developing a geographic visualization tool to support earth science learning", Cartography and Geographic Information Science, 27(4), 279-293. Havre, S., Hetzler, E., Whitney, P., and Nowell, L., (2002) "ThemeRiver: visualizing thematic changes in large document collections", IEEE Transactions on Visualization and Computer Graphics, 8(1), 9-20. Hochheiser, H. and Shneiderman, B., (2001) Visual Specification of Queries for Finding Patterns in Time-Series Data, Technical Report #CS-TR-4326. UMIACS-TR-200125, University of Maryland, Computer Science Dept. Knapp, L., (1995) "A task analysis approach to the visualization of geographic data", In: Nyerges, T. L., Mark, D. M., Laurini, R., and Egenhofer, M. J., (eds.), Cognitive Aspects of Human-Computer Interaction for Geographic Information Systems. Dordrecht: Kluwer Academic, pp. 355-371. Koussoulakou, A., and Kraak, M. J., (1992) "Spatio-temporal maps and cartographic communication", The Cartographic Journal, 29, 101-108. Kraak, M. J., (undated) MinardmWhat Has Been Done by ITC?. Online: http://www.itc.nl/personal/kraak/1812/minard-itc.htm (23/10/03). Kraak, M. J., and MacEachren, A. M., (1994) "Visualization of the temporal component of spatial data", In: Waugh, T., and Healey, R., (eds.), Proceedings of the Sixth International Symposium on Spatial Data Handling, Advances in GIS Research, Edinburgh, Scotland, pp. 391-409.
Impact of Data and Task Characteristics
221
Kraak, M. J., Edsall, R. M., and MacEachren, A. M., (1997) "Cartographic animation and legends for temporal maps: exploration and or interaction", Proceedings 18th International Cartographic Conference, Stockholm, June 23-27, 1997, IGU, pp. 253-262. Langran, G., (1992) Time in Geographic Information Systems. London: Taylor and Francis. MacEachren, A. M., (1994a) Some Truth with Maps: A Primer on Symbolization and Design. Washington: Association of American Geographers. MacEachren, A. M., (1994b) "Time as a cartographic variable", In: Hearnshaw, H., and Unwin, D., (eds.), Visualization in Geographical Information Systems. Chichester: Wiley, pp. 115-130. MacEachren, A. M., (1994c) "Visualization in modern cartography: Setting the agenda", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Oxford: Pergamon, pp. 1-12. MacEachren, A. M., (1995) How Maps Work: Representation, Visualization, and Design. New York: The Guildford Press, p. 513. MacEachren, A. M., and Kraak, M. J., (1997) "Exploratory cartographic visualization: advancing the agenda", Computers & Geosciences, 23(4), 335-344. Mockus, A., (1998) "Navigating aggregation spaces", IEEE Information Visualization Symposium 1998 Late Breaking Hot Topics Proceedings. Research Triangle Park, NC: IEEE Computer Society Press, pp. 29-32. Morris, K., Hill, D., and Moore, T., (2000) "Integrated data management and visualization through space and time", In: Fullerton, K., (ed.), Proceedings of 5th EC-GIS Workshop, 28-30 June 1999. Stresa, Italy: EUR 19018 EN, European Communities, pp. 460-473. Mountain, D., and Dykes, J., (2002) "What I did on my vacation: spatio-temporal log analysis with interactive graphics and morphometric surface derivatives", Proceedings of GISRUK'2002, University of Sheffield, April 2003. Mountain, D., and Raper, J., (2001a) "Modelling human spatio-temporal behaviour: a challenge for location-based services", Geocomputation'2001 - 6th International Conference on Geocomputation, University of Queensland, Australia, September 2001. Mountain, D., and Raper, J., (200 lb) "Positioning techniques for location-based services: characteristics and limitations of proposed solutions", Aslib Proceedings, 53(10), 404-412. National Cancer Institute, (2003) Cancer Mortality Maps and Graphs. Online: http://www3.cancer.gov/atlasplus/(23/10/03). NaturDetektive (2003) Naturdetektive 2003. Online: http://www.naturdetektive.de (23/10/03). Oberholzer, C., and Hurni, L., (2000) "Visualization of change in the interactive multimedia atlas of Switzerland", Computers & Geosciences, 26(1), 423-435. Peuquet, D. J., (1994) "Its about time: a conceptual framework for the representation of temporal dynamics in geographic information systems", Annals of the Association of American Geographers, 84(3), 441-461.
222
N. Andrienko, G. Andrienko & P. Gatalsky
Peuquet, D., (2002) Representations of Space and Time. New York: Guilford, p. 379. Peuquet, D. J., and Kraak, M. J., (2002) "Geobrowsing: creative thinking and knowledge discovery using geographic visualization", Information Visualization, 1, 80-91. Seasonal Migration of White Storks, (2003). Seasonal Migration of White Storks, 1998-1999. http://www.ais.fraunhofer.de/and/java/birds Slocum, T., Yoder, S., Kessler, F., and Sluter, R., (2000) "MapTime: software for exploring spatiotemporal data associated with point locations", Cartographica, 37(1), 15-31. Stojanovic, D., Djordjevic-Kajan, S., Mitrovic, A., and Stojanovic, Z., (1999) "Cartographic visualization and animation of the dynamic geographic processes and phenomena", Proceedings of 19th International Cartographic Conference, pp. 739-746. Takatsuka, M., and Gahegan, M., (2001) "Sharing exploratory geospatial analysis and decision making using GeoVISTA Studio: from a desktop to the web", Journal of Geographic Information and Decision Analysis, 5(2), 129-139. Tobler, W., (1970) "A computer movie simulating urban growth in the Detroit region", Economic Geography, 46(2), 234-240. Tufte, E. R., (1983) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, p. 197. Unwin, A., and Wills, G., (1998) "Exploring time series graphically", Statistical Computing and Graphics Newsletter, 2, 13-15. Wachowicz, M., (1999) Object-Oriented Design for Temporal GIS. London: Taylor and Francis. Wehrend, S., and Lewis, C., (1990) "A problem-oriented classification of visualization techniques", Proceedings of the First IEEE Conference on Visualization: Visualization'90, October 1990. Los Alamitos, CA: IEEE, pp. 139-143. Wilkinson, L., (1999) The Grammar of Graphics. New York: Springer-Verlag. Wilkinson, L., Rope, D. J., Carr, D. B., and Rubin, M. A., (2000) "The language of graphics", Journal of Computational and Graphical Statistics, 9(3), 530-543. Wilkinson, L., Rope, D. J., Rubin, M. A., and Norton, A., (2001) "nVizn: an algebrabased visualization system", Proceedings of the 1st International Symposium on Smart Graphics, March 21-23, 2001. New York: Hawthorne, pp. 76-82. Worboys, M. F., (1998) "A generic model for spatio-bitemporal geographic information", In: Egenhofer, M. J., and Golledge, R. G., (eds.), Spatial and Temporal Reasoning in Geographic Information Systems. New York: Oxford University Press, pp. 25-39.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 11
Using Multi-agent Systems for GKD Process Tracking and Steering: The Land Use Change Explorer Monica Wachowicz, Xu Ying & Arend Ligtenberg, Wageningen UR, Centre for Geo-Information, Droevendaalsesteeg 3, PO BOX 47, 6700 AA Wageningen, The Netherlands
Keywords: geographic knowledge discovery, geovisualization, datamining, collaborative environment, multi-agent systems, abductive reasoning, reactive agent architecture
Abstract Autonomous agents are a new paradigm for modelling a geographic knowledge discovery (GKD) process. Agents are usually capable of having their own actions and internal states (autonomy), reacting to system changes or other agents' actions (responsiveness) until they complete their tasks (social ability). This chapter presents an approach to the process of GKD based on a reactive architecture in which an agent' s behaviour is implemented in some form of direct mapping from situation to action. The situation is characterized by a set of system states. At any given point of time, the system is assumed to be in one of these states. The agents' actions are realized through a set of tasks. Each action can be seen as a decision-making function that continually takes perceptual input and maps it into an action to perform a task. There are many potential decision-making functions in a GKD process and they will have a different effect on the behaviour of an individual agent and the performance of the overall system. An approach, termed "GeoInsight" is proposed for characterizing decisionmaking functions in a multi-agent system (MAS). This approach is illustrated using spatial planning as a decision-making process, a context that requires geo-information about past, present, and future land use practices and other related aspects of a landscape. The approach is implemented, and empirically evaluated within the Land Use Change Explorer prototype, using an MAS (Beegent System), a geographic information system (ArcInfo ODE), and a graphical user interface (Java API).
11.1
Introduction
Traditional exploratory spatial analysis tools are inadequate for handling the increasing volumes of data and the complexity associated with discovering spatio-temporal patterns 223
224
M. Wachowicz, X. Ying & A. Ligtenberg
within a geographic knowledge discovery (GKD) process. Some attempts to develop methods and associated tools for a GKD process have already illustrated how difficult it is to make use of appropriate interaction forms, visual representations and mining tasks in order to allow users to dynamically construct geographic knowledge. Andrienko and Andrienko (1999) have shown the use of dynamic and interactive cartographic representations by users who are allowed to generate on-the-fly maps according to their individual cognitive abilities and understanding of the problem domain. The main purpose is to allow users without cartographic knowledge to produce maps by incorporating their geographic knowledge about spatially referenced data such as demographic and economical data sets. Complementary work involves the development of a taxonomy of geovisualization and datamining operations to facilitate a GKD process for clustering time-series data (MacEachren et al., 1999; Wachowicz, 2001). This latter work has demonstrated the potential role of these user-friendly operations in "process tracking" (using visual operations that display key aspects of the GKD process as it unfolds) and "process steering" (adapting mining operations as the GKD process unfolds, thus changing outcomes on the fly). The development of methods and tools for supporting GKD as a user-centred process has introduced new problems resulting from the complexity of this process. The process begins with data exploration, leading to the development of a working hypothesis by users (e.g., about change in geographic phenomena over time), which in turn leads to the constructions of knowledge (objects, rules, conceptual hierarchies, patterns) around which analyses can be built, and results produced and evaluated (Koperski et al., 1999; Roddick and Spiliopoulou, 1999; for an additional perspective, see Gahegan, this volume (Chapter 4)). The major challenge here is to empirically demonstrate that a combination of geovisualization and datamining methods can improve a GKD process and to build collaborative environments that take advantage of the best that each has to offer. From a system implementation perspective, the integration of geovisualization and datamining methods is becoming more straightforward. Similarly, multi-agent systems (MASs) can provide platform independence, Internet access, and simplify the exchange of spatio-temporal data and functionalities that are necessary to support a GKD process. However, a GKD process is inherently spatio-temporal data intensive, making considerable demands on graphics hardware and database structures. If a GKD process is to be interactive, these demands are even higher. Therefore, the issues are related to effectively supporting user-data interactions in both spatial and temporal domains, as well as the development of useful interface metaphors that can support interactive visual representations and datamining tasks in order to amplify user cognition within a GKD process. From a user perspective, GKD is a multi-faceted process that is likely to benefit from tools that support a collaborative environment that enables the integration of the perspectives of different users on a problem domain, for example, (see Brodlie et al., this volume (Chapter 21)). One important consideration when proposing a collaborative environment for a GKD process relates to the type of inference that is to be employed (Wachowicz et al., 2002; see also Fuhrmann, this volume (Chapter 31)). At early stages of GKD, when an abductive form of reasoning is likely to be critical, a collaborative
Using Multi-agent Systems for GKD Process Tracking and Steering
225
environment must be flexible enough to allow users to create new structures that help them to explain the data presented during a GKD process. Later in the process, when the objective is to locate pre-defined patterns (deduction) or learn from examples (induction), the GKD process will require more procedural operations within the collaborative environment, since the role of a user is somewhat reduced to explaining hypothesis. The main issue here is to make a GKD process very flexible and facilitate intuitive exploration of spatio-temporal patterns using very large data sets. This chapter describes the design of a multi-agent collaborative environment for manipulating spatio-temporal data to find, relate, and interpret interesting patterns in very large land use data sets. We explain how different geovisualization and datamining methods and functionalities can be combined in order to design a GKD process for the identification and interpretation of the space-time variability in both composition and structure of changing and use patterns. Few studies exist that translate the knowledge of an environmental process to an explicit pattern context (for a recent review, see Wachowicz, 2002). In general, among empirically based work, there is a tendency to find highly quantitative studies on patterns and more qualitative studies describing their corresponding processes. There is a better understanding of the diversity of types of patterns (e.g., types of land uses) than of the processes responsible for their generation (e.g., fire, urbanization, and climate change). One of the main impediments to linking research on process and pattern is the complexity of mapping a given pattern to a given process. Two independent processes may produce the same pattern. Two types of patterns may vary in terms of their spatial arrangement of units and variability through time. Therefore, it is often insufficient to identify a given pattern using a single composition (e.g., NDVI values from a satellite image) or a single structure (size, shape, adjacency, and sinuosity). In fact, the identification and definition of the space-time variability of both composition and structure of a pattern will determine what form the pattern will take and therefore distinguish it from other patterns. This requires a good understanding of how patterns can be found using a variety of methods such as association, correlation, causality, partial periodicity, sequential, and emerging patterns. Therefore, a GKD process is proposed in this chapter for creating a dynamic method for finding, relating, and interpreting interesting, meaningful, and unanticipated patterns in large land use data sets. The goal is to develop a conceptualization of a GKD process that involves different users achieving insight about patterns of land use change within a collaborative environment, which facilitates the understanding of these patterns and their relation with the corresponding process in the real world (e.g., urbanization, deforestation). This GKD process involves both interaction and iteration operations, through which users can achieve insight by manipulating large land use data sets using datamining techniques for discovering changes in land use practices, and geovisualization techniques for steering the GKD process as it unfolds, visualizing land use changes on the fly. A tool prototype (Land Use Change Explorer) has been developed to allow different users to discover land use change patterns and understand their corresponding process in the real world. The Land Use Change Explorer was implemented as a multi-agent collaborative environment. In this environment, agents
226
M. Wachowicz, X. Ying & A. Ligtenberg
help users to perform a series of steps of a GKD process, namely: selecting land use data sets; selecting proper representations to visualize land use change; choosing sequences of mining operations; geovisualization functionalities. Ferrand (1996) defines an agent as an entity living in an environment and able to modify both its environment (communication, decision, action) and itself (perception, reasoning, learning). In a multi-agent collaborative environment, agents are capable of exhibiting flexible problem solving behaviour in pursuit of their design objectives. They are able to respond in a timely fashion to changes that occur in the collaborative environment and act in anticipation of future goals. Use of the prototype is demonstrated for two types of users. They represent two stakeholders: an agricultural policy maker and an urban planner. The aim is to show how their pooled expertise affects the GKD process within a collaborative environment. When stakeholders with different backgrounds and interpretations of a problem domain need to discover solutions that satisfy opposing interests, a collaborative environment is necessary so that knowledge may be shared or examined from another's perspective. In the GKD process, the discovered knowledge may have conflicts, tensions, and biases within it, and these too will have to be dealt within the collaborative environment. At this stage, the Land Use Change Explorer prototype supports the abductive mode of reasoning. We outline the GeoInsight approach in w11.2 by describing the main modes of inference in a GKD process. A description of the main characteristics of MAS is provided in w and the role of MAS in a GKD process is defined. The Land Use Change Explorer prototype itself is described in w11.4 and conclusions and future directions are detailed in w11.5.
11.2
Modes of Explanatory Reasoning" The GeoInsight Approach
Reasoning can be interpreted in a variety of logical ways (Harman, 1965). However, only two conditions are present in all and every logical account of reasoning. The first condition attempts to capture the condition that background theory does not explain observations, which may be a novel phenomenon, or they may actually question the theory. The second requires an explanation to account for the observations (Hempel, 1965). These explanations may have various forms: facts, rules, or even theories. Therefore, explanatory reasoning is triggered by a surprising phenomenon in need of explanation. On the basis of these conditions, a taxonomy of different modes of explanatory reasoning used in the GKD process can be defined as containing the abductive, inductive, and deductive modes of reasoning (see Gahegan, this volume (Chapter 4)). Most of the automated methods for GKD have been developed to learn from examples that are presented or selected (induction), or to attempt to locate pre-defined patterns (deduction). Induction involves the classification or generalization of examples for explaining tendencies in observations, in which the hypotheses become predicate clauses (Figure 11.1). It is definitely the most reliable means of computationally enabled
Using Multi-agent Systems for GKD Process Tracking and Steering
227
Figure 11.1. GKD process described in terms of the main steps found in inductive reasoning.
knowledge construction due to the availability of many robust and automated algorithms in GKD. On the other hand, deduction can only be applied when objects, categories, or relationships have already been defined (Figure 11.2). It usually forms the basis of most exploratory analysis and modelling because it can be verified in relatively straightforward ways (Simoudis et al., 1996). In a GKD process, abductive reasoning involves the adoption of an existing or non-existing hypothesis for the explanation of a given observation, in which the explanation is usually, represented as propositional clauses. Abduction is flexible because it is not restricted to using existing patterns, but is instead free to create new patterns that help to explain the data (Figure 11.3). Abduction is the most flexible reasoning mode because it requires neither the target nor the hypothesis to be pre-defined. It is therefore, highly suited for the initial exploratory phase of knowledge construction, especially if little is known concerning the structures in the data. Perhaps, the single most important factor that sets the GKD process proposed in the GeoInsight approach apart from other KD processes is the connection to the user (expert) as a rich source of interpretation for the uncovered pattern, thus opening the way for abductive reasoning to take place.
Figure 11.2. GKD process described in terms of the main steps found in deductive reasoning.
228
M. Wachowicz, X. Ying & A. Ligtenberg
Figure 11.3. GKD process described in terms of the main steps found in abductive reasoning.
In the GeoInsight approach, statistical methods offer a useful and practical framework for supporting the abductive mode of reasoning. For instance, in the Land Use Change Explorer prototype (w11.4), unsupervised methods are used to uncover unknown spatio-temporal patterns (e.g., land use changes that explain an urbanization process in the real world). The prototype allows users to apply statistical approaches (probability distributions, hypothesis generation, model estimation and scoring) for generating hypotheses by performing the task of exploring classes, clusters or rules from a land use data set. The GKD process was built with a continuous mining strategy in mind. Data are mined in order to understand how land use has changed within a given period of time and to comprehend the real-world processes that influence this change. To achieve this, the GKD process was designed to allow reasoning according to what different users want to explain (i.e., the location, changes, attributes, identity, or relationships among land use classes). Land use changes occur at different spatial and temporal scales, which are caused by the continuous impact of social, ecological, and economic factors (Brown, et al., 2000). The intricate interrelationships among these factors leave consequences on cultural, historical, and politically evolving processes. Geographical knowledge can be considered as information about land use changes and value can be added to this knowledge by interpretation based on a particular context, experience, and purpose (Longley et al., 2001). Therefore, the GeoInsight approach supports the design of a GKD process for the integration of a wide range of datamining tasks and visual representations, which are needed to allow knowledge tracking (the discovery of interesting patterns in land use changes) as well as knowledge steering (the explanation of these interesting patterns through abductive reasoning).
11.3
Multi-agent Systems
MASs focus on modelling processes such as problem solving, planning, search, decisionmaking and learning, which make it possible for agents to show flexibility and rationality in their behaviour. Conversely, conventional deterministic or empirical modelling techniques (such as PDE and interaction models, for example) offer only a partial
Using Multi-agent Systems for GKD Process Tracking and Steering
229
solution to constructing GKD processes. Here, MAS is presented as a modelling technique that can be used to deal with the complexity of GKD processes. Agents can be considered a new paradigm for developing software applications (Kalenka and Jennings, 1997) and they provide us with a natural development in the search for ever-more powerful abstractions with which to build a GKD process. A simple way of defining an agent is by considering it as a software robot that lives within a computer network, and performs tasks on behalf of humans or other agents. Wooldridge (1999) has described the main advantages of using agents rather than objects as consisting of the following: 9 Agents embody a stronger notion of autonomy than objects, and in particular, they decide for themselves whether or not to perform an action on request from another agent. 9 Agents are capable of flexible (reactive, pro-active, social) behaviour, and the standard object model has nothing to say about such types of behaviour. 9 An MAS is inherently multi-threaded, in that each agent is assumed to have at least one thread of control. Since the 1970s, distributed artificial intelligence (DAI) has developed and diversified rapidly in various applications. DAI is the study, construction, and application of MAS in which several interacting agents pursue some set of goals or perform some set of tasks. It aims to provide solutions to inherently distributed and particularly complex applications (Weiss, 1999). There are two main reasons that are primarily driving the use of MAS in GKD: 9
9
The agent concept is most suitable for dealing with datamining and geovisualization systems that are distributed, large, open, and heterogeneous. MAS offer considerable promise and potential for managing high-level interaction in a GKD process. MASs are capable of cutting across a wide range of different technology and application areas, which require a range of interaction forms, visual representations and datamining tasks.
An agent society can be considered as analogous to human society in that each agent has its life cycle - consisting of birth and death, exhibiting competition and cooperation, communicating using language, and operating under rules and expectations that define behaviours (Thissen, 1999). Certain agents may be described as being one of the following: 9 9 9 9
C o l l a b o r a t i v e agents, where the emphasis lies on autonomy and cooperation with other agents. M o b i l e a g e n t s that are capable of roaming wide area networks (i.e., the Internet) in order to find information for their datamining tasks according to users' needs. I n f o r m a t i o n a g e n t s that help users with the management of information. They can manage, manipulate or collate information from many distributed sources. R e a c t i v e a g e n t s that act/respond in a stimulus-response manner to the state of the environment they live in.
230
M. Wachowicz, X. Ying & A. Ligtenberg
9
Hybrid agents that combine some properties of two or more agent categories
9
Heterogeneous agent systems, consisting of an integrated collection of at least
described above. two agents that belong to two or more agent categories. A key pattern of interaction in MASs is goal- and/or task-oriented coordination, both in cooperative and in competitive situations. In the case of cooperation, several agents try to coordinate group efforts to accomplish the desired objective that the individuals cannot; while in the case of competition, several agents try to get what only some of them can have. MASs, as collaborative environments, have the capability to offer several desirable properties (Weiss, 1999): 9
9
9
9 9
Speed-up and Efficiency - agents can operate asynchronously and in parallel, and this can result in an increased overall speed. Robustness and Reliability - the failure of one or several agents does not necessarily make the overall systems useless, because other agents already available in the system may take over their role. Scalability and Flexibility - the system can be adapted to an increased problem size by adding new agents, and this does not necessarily affect the operationality of the other agents. Cost - MASs may be much more cost-effective than centralized systems, since they can be composed of simple sub-systems of low unit cost. Development and Reusability - individual agents can be developed separately by specialists, the overall system can be tested and maintained more easily, and it may be possible to reconfigure and reuse agents in different application scenarios.
MASs provide a computational infrastructure for interactions among agents to take place. In this prototype, agents are connected with heterogeneously distributed network resources using both FIPA and CORBA specification standards. The FIPA specification defines an agent platform, in which FIPA compliant agents can exist, operate and be managed (The Foundation for Intelligent Physical Agents, 2003). It also defines capabilities such as the location of individual agents by their services or name, message routing and life cycle management (Figure 11.4). Agents must be registered on the platform to be able to use the specific elements of that infrastructure and further interact with other agents on that platform or other platforms. The infrastructure also includes protocols for agents to communicate and interact, which are also specified by FIPA. Communication protocols enable agents to exchange and understand messages. Interaction protocols enable agents to have conversations, which for our purposes are structured exchanges of messages (Huhns and Stephen, 1999). The agent interaction protocol defines a sequence of messages that represent a complete conversation between agents. The agent communication protocol specifies the messages for a particular course of action to be exchanged between two agents. FIPA compliant agents communicate using the agent communication language (ACL), which is a semantically well defined, highly expressive language that provides a
Using Multi-agent Systems for GKD Process Tracking and Steering
231
Figure 11.4. FIPA specifies agents' interoperation. set of primitives from which other agent messages can be created. ACL is based on speech act theory. Although the term "speech" is used, agent speech acts have to do with procedural written forms of communication rather than spoken communication. Messages that are constructed using ACL are actually sent to other agents using the Internet Inter ORB Protocol (IIOP), as shown in Figure 11.5. FIPA uses IIOP as its main communication protocol that enables geographically distributed agents to contact each other via a network. The object management group (OMG) is an open membership, not-for-profit consortium that produces and maintains CORBA specifications for interoperable enterprise applications. CORBA is OMG's open, vendor-independent architecture and infrastructure that computer applications use to work together over networks. Using a standard protocol, a CORBA-based programme from almost any computer, operating system, programming language and network, can interoperate with a CORBA-based
Figure 11.5. Agents interoperate through an IIOP network.
M. Wachowicz, X. Ying & A. Ligtenberg
232
programme from the same or another computer, operating system, programming language and network. The CORBA specification provides the implementation for the IIOP communication between a client agent and a resource server, and it enables agents to communicate with one another regardless of what programming language they are written in or what operation system they are running on. A FIPA compliant agent platform must at least support IIOP, which is used to provide an address for each agent. The address will determine how a message is routed to that agent. This is a common infrastructure to ensure that agents can send each other messages, like queries and replies, including feedback to indicate whether a message was understood. MAS can cope with the complexity of distributed network systems, where the resources we require are distributed in various geographical locations. The server contains the resource implementation, such as knowledge, expertise or application techniques, and the agent client needs to connect to the resource server and gets the required services (Figure 11.6). In this structure, both server and client have an interface that specifies their behaviour. The services a resource server is willing to provide and a client agent can use are declared by their own interfaces. This hides the details about implementation through encapsulation. In fact, it is an object-based system feature - the behaviour of an object is independent of its implementation. As long as an object provides the behaviour specified by its interface, it is not necessary to know how it is implemented. The interface of each client/server side is defined using the interface definition language (IDL), which is part of CORBA specifications. IDL allows a developer to define an interface in a standard fashion, which makes the realization of sharing heterogeneous resources and interoperability between distributed agents possible.
Network
f
~
~
IIOP
Agent Client
Server
Figure 11.6. Agent client connects to a remote resource server via the network.
Using Multi-agent Systems for GKD Process Tracking and Steering
11.4
233
The Land Use Change Explorer
The Land Use Change Explorer prototype is a collaborative environment for finding, relating, and interpreting interesting, meaningful, and unanticipated patterns in large land use data sets. Users can achieve insight by manipulating large land use data sets using datamining techniques for discovering changes in land use practices, and geovisualization techniques for visualizing land use changes on the fly. Unlike conventional GIS, the prototype supports a virtual system where each user can observe other users' interaction with the system and react by making changes to its own environment. In this environment, agents help users through a series of steps of a GKD process: selecting land use data sets; selecting appropriate representations to visualize land use change; choosing sequences of mining operations; suitable geovisualization functionalities. Considering, for example, a complicated set of resource planning activities occurring in a watershed, which is largely privately owned and contains an area of internationally significant wetland, such as a NATURA 2000 site. There is a considerable concern that this unique wetland community is threatened by agricultural land use practices. The agents are helpful to address the issues of natural resource management by prioritizing natural resource problems and identifying feasible solutions among stakeholders. In this case, the GKD process can vary from introduction of a problem, creation of a common perspective to thinking through alternatives and their implications (abductive reasoning). Here, we use the perspectives of an agricultural policy maker and an urban planner to illustrate the GKD process from the selection of suitable data source, datamining technique used for the change analysis and visualization technique for the land use change patterns. Agents in the Land Use Change Explorer prototype support the roles of these stakeholders. The agricultural policy maker who wants to retain full control over their land (private property rights issue) and maximize farm revenue; and a urban planner who is looking for ways to diversify and bolster the weak regional economy of this area through agricultural, industrial, and recreational opportunities. Therefore, the main objective of the implementation exercise is to convey the "big picture" at least to the extent that a correlation can be made among the both agricultural policy maker and urban planner regarding the issue at hand. The Land Use Change Explorer prototype was implemented using an MAS model based on a client/server architecture as described in w11.3. At the server side, different resources are available such as land use databases, datamining algorithms, geovisualization and GIS functionalities. The client side consists of a graphical user interface (GUI) that allows users to connect and apply the resources available at the server side. The MAS is able to receive the request from the client, connect to the server to perform the operations using the required resource, and present the results back to the GUI. If we put all these components together as displayed in Figure 11.7, MAS can be considered as the key technology for the support of a GKD process. The Java programming language (Sun Microsystems Inc., 2003) was used for the implementation of the GUI for the Land Use Change Explorer. The Bee-gent system (Toshiba, 2003) was selected for the MAS implementation. The system is composed of
234
M. Wachowicz, X. Ying & A. Ligtenberg
Figure 11.7. The client-server architecture for the Land Use Change Explorer.
two types of agents: the agent wrapper, which wraps local functions that can be client applications or server resource utilities, and the mediation agent that migrates around the network and performs tasks by interacting with the agent wrapper. Both the mediation agent and the agent wrapper can communicate via the ACL, which supports the processing by inferring the intention of the requirements specified by different agents or requests information from them. They can be described as being mobile and reactive agents. ArcSDE (Environmental Systems Research Institute, 2003) was used for performing the GIS functionalities and geovisualization tasks since it includes a Java ODE application-programming interface that extends ArcInfo to cross-platform applications. Finally, the MineSet datamining tool (Silicon Graphics Inc., 2002) was used to perform the datamining operations. In the Land Use Change Explorer, the agents can: 9 9 9 9 9
Interact with the users; Incorporate a user's preferences in a GKD process; Perform queries related to land use changes; Display different visual representations of land use change patterns; Assist the user in the different steps of a GKD process (select a data set; perform data transformation; select a datamining task and the respective datamining algorithm; initiate a GIS function or query);
Using Multi-agent Systems for GKD Process Tracking and Steering
9 9
235
Provide reliability information to a specific user, relating to land use change patterns; Hide cumbersome operations from users.
All the activities described above allow the agents to support a GKD process in which land use change is explored stepwise and rule-based inference is used to present the uncertainty caused in each analysis step. Rules are reasonably easy to understand by humans and are also a powerful machine-based knowledge representation schema. Moreover, rule-based inference can apply human knowledge and functions at the level of a human expert, thus enabling the abductive mode of reasoning within a GKD process. Once knowledge is represented as a rule set, it is relatively easy to construct an engine that can make use of the rules in an automated reasoning system (Gordon, 2000). Several rules (geometric as well as thematic) were used to evaluate the degree of change possibility and reliability, which is closely linked to the fuzzy nature of land use classes in the data sets. Possibility is computed from the applied data analysis process, although reliability is derived solely from expert knowledge. Geometric and thematic rule sets were used to provide more information apart from just stating "change" and "no change". Significant overlap, shape factor and reliable change were directly linked to polygon characteristics to determine the degree of spatial isomorphism of a spatial object belonging to a land use class. They were also used to present spatial change possibility and spatial change reliability. The thematic criterion has been defined in rule sets to distinguish between the most likely and the least likely land use characteristic changes. Two sets of decision rules were used to distinguish "reliable change" from "unreliable change". Due to the noise embedded from the land use data sets and data processing, not all of the recognized land use change appeared to be reasonable in the real world during the GKD process. A rule set was used to describe the reliability of thematic land use change. Table 11.1 illustrates one example in which four thematic categories were considered: urban (grid_code 1), forest and nature (grid_code 2), water (grid_code 3), and agriculture (grid_code 4). Table 11.2 illustrates the second example for reliable land use change identification that was based on buffer distance. In general, similar land cover classes are easily confused with each other during the classification of remotely sensed images. This can introduce high levels of uncertainty when trying to discern land use changes from noise. Therefore, different buffer distances were used to consider the uncertainty of change according to each land cover class. Outside the buffer range, the change was considered to be of high certainty. Land use change patterns are scale dependent, therefore the Land Use change Explorer prototype is planned to provide users with two specified scales: global and local scales. At global scale, the GKD process only focuses on changes between super classes. At local scale, the GKD process is located at class level or sub-class level. Through the Land Use Change Explorer, different users can explore very large land use data sets within a collaborative environment, which can help them to share and understand each other's specific domain knowledge. This collaborative environment assists users in formulating queries rationally and clearly about land use change patterns
M. Wachowicz, X. Ying & A. Ligtenberg
236
Table 11.1. Rule set1 for thematic reliable land use change identification. Rule no.
Rule structure
1 2 3 4 5 6 7 8 9 10 11 12 13
If If If If If If If If If If If If If
(grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p (grid_code3p
= = = = = = = = = -
grid_code4p) then no_change 1 and grid_code4p = 2) then change reliability is very low 1 and grid_code4p = 3) then change reliability is very low 1 and grid_code4p = 4) then change from reliability is very low 2 and grid_code4p = 1) then chang je reliability is very high 2 and g r i d _ c o d e 4 p - 3) then chang le reliability is low 2 and grid_code4p = 4) then chant le reliability is high 3 and grid_code4p = 1) then chang je reliability is low 3 and grid_code4p = 2) then chang le reliability is very low 3 and grid_code4p = 4) then chang re reliability is very low 4 and grid_code4p = 1) then chang je reliability is very high 4 and grid_code4p -- 2) then chang le reliability is medium 4 and g r i d _ c o d e 4 p - 3) then chang le reliability is low
grid_code3p is the thematic categorical attribute in time series 1 table; grid_code4p is the thematic categorical attribute in time series 2 table.
by presenting land use changes and the uncertainty rules in an appropriate way that can be helpful for supporting users' perspectives within a GKD process. In order to match different users' perspectives, user identification is crucial to the GKD process, varying from data source selection, which determines an appropriate analysis strategy, up to the visualization of the results. As a result, the GKD process becomes quite unique for each user and it begins with user identification. Each type of user will have a different GUI design in order to provide specific applications that are well matched with their needs. The Land Use Change Explorer was designed for supporting two types of stakeholders - an urban planner and an agriculture policy maker. The collaborative environment can assist them in formulating queries using a suitable data source and datamining approach to obtain optimal land use change information. Six interface metaphors compose the Land Use Change Explorer: the menu
Table 11.2. Rule set2 for spatial reliable land use change. Buffer distance Land use type
Global scale
Local scale
Uncertainty
Forest/nature Forest/agriculture Urban/agriculture Urban/forest Water
50 40 30 20 10
25 20 15 10 5
Very high (> 80%) High (60-80%) Medium (40-60%) Low (20-40%) Very low (,--
-
Figure 14.5. Three different levels of detail for a land-use information layer, selected according to viewer's distance from the surface. The information content of the surface is changed dynamically using level of detail algorithms and volumetric textures. Images generated with LandExplorer (Dollner, this volume, Chapter 16).
Using 3D in Visualization
309
Despite the rapid developments in computer graphics technology, there continues to be a gap between the demands generated by geovisualizers and that which can be realized with current rendering technologies. For example, it is rather complicated to integrate multi-resolution terrain models with dynamic textures (D611ner et al., 2000), or even represent dynamic vector data in 3D representations (Kersting and D611ner, 2002a-c). This gap is maintained by the rising expectations of users of 3D systems and the conception of new approaches to geovisualization afforded by changing technology.
14.8
Conclusions
The modelling of phenomena using 3D is not new, but it does deserve special treatment when exploring geovisualization issues. This chapter has emphasized the distinction between the dimensionality of referents (real-world phenomena) and their modelled representation. By doing so, it becomes clearer that using 3D in visualization is not limited to data that have three obvious spatial dimensions. Multi-dimensionality should be considered equally part of the modelling, representation and exploration process. More than perhaps any other aspect of visual representation, effective 3D visualization is highly dependent on the rapid developments in the wider field of 3D technologies. This includes hardware, software and theoretical developments in rendering. Those involved with visualization in 3D can benefit from progress in these fields, as well as exert some influence over their future development (for example through collaboration with the gaming industry). The chapters in this section of Exploring Geovisualization provide more detailed examples of how developments in 3D representation and visualization can inform the process of geovisualization, as well as highlighting the issues that geovisualizers have to offer when using 3D in visualization.
Acknowledgements The authors are grateful for clarification of ideas and comments from the two anonymous reviewers.
References Angel, E., (2002) Interactive Computer Graphics: A Top-Down Approach with OpenGL. London: Addison Wesley, p. 719. Bertin, J., (1983) Semiology of Graphics: Diagrams, Networks, Maps. Madison, Wisconsin: University of Wisconsin Press. Buchroithner, M. F., and Kirschenbauer, S., (1998) "Derivation of relief information from various cartographic representation", Hochgebirgskartographie Silvretta 98 High Mountain Cartography. Vienna: Wiener Schriften zur Geographie und Kartographie, pp. 65-75. Carlberg, B., (1943) "Zeichnen kartographischer raumbilder", Petermanns Geographische Mitteilungen, 89, 302-308.
310
J. Wood et al.
D611ner, J., Baumann, K., and Hinrichs, K., (2000) "Texturing techniques for terrain visualization", Proceedings IEEE Visualization, Salt Lake City, pp. 227-234. Dykes, J. A., (2002) "Creating information-rich virtual environments with georeferenced digital panoramic imagery", In: Fisher, P., and Unwin, D., (eds.), Virtual Reality in Geography. London: Taylor & Francis, pp. 68-92. Fabrikant, S. I., (2000a) Spatial Metaphors for Browsing Large Data Archives, University of Colorado-Boulder, Department of Geography. Fabrikant, S. I., (2000b) "Spatialized browsing in large data archives", Transactions in GIS, 4(1), 65-78. Forer, P., (1998) "Geometric approaches to the nexus of time, space and microprocess: implementing a practical model for mundane socio-spatial systems", In: Egenhofer, M. J., and Golledge, R. G., (eds.), Spatial and Temporal Reasoning in Geographic Information Systems. New York: Oxford University Press, pp. 171-190. Fuhrmann, S., and MacEachren, A. M., (1999) "Navigating desktop GeoVirtual environments", IEEE Information Visualization 99, Late Breaking Hot Topics Proceedings, San Francisco, CA, pp. 11-14, Online: http://www.geovista.psu.edu/publications/amm/ammIV99.pdf (Oct. 23-28). Graf, U., (1943) "Das raum-modell bei stereoskopischen verfahren in der kartographie", Petermanns Geographische Mitteilungen, 89, 65-69. Haber, R. B., and McNabb, D. A., (1990) "Visualization idioms: a conceptual model for scientific visualization systems", In: Nielson, G., Shriver, B., and Rosenblum, L., (eds.), Visualization in Scientific Computing. Los Alamitos, CA: IEEE Computer Society Press, pp. 74-93. Hagerstrand, T., (1968) Innovation Diffusion as Spatial Process. Chicago: University of Chicago Press. Hagerstrand, T., (1970) "What about people in regional science", Papers, Regional Science Association, 24, 1-21. Heidmann, F., (1999) Aufgaben- und Nutzerorientierte Unterstiitzung Kartographischer Kommunikationsprozesse durch Arbeitsgraphik. Herdecke: GCA-Verlag. Jenks, G., and Brown, D., (1966) "Three dimensional map construction", Science, 154, 857-864. Kersting, O., and D611ner, J., (2002a) "Interactive 3D visualization of vector-data in GIS", Proceedings A CM GIS 2002, Washington, pp. 107-112. Kersting, O., and D611ner, J., (2002b) "Interactive visualization of 3D vector data in GIS", A CMGIS 2002, Washington. Kersting, O., and D611ner, J., (2002c) "Interactively developing 3D graphics in Tcl", Proceedings USENIX Annual Technical Conference, Monterey, CA. Kraak, M. J., (1988) Computer-Assisted Cartographical Three-Dimensional Imaging Techniques. The Netherlands: Delft. Kraak, M. J., and MacEachren, A. M., (1999) "Visualization for exploration of spatial data (editorial introduction to special issue)", International Journal of Geographical Information Science, 13(4), 285-287.
Using 3D in Visualization
311
Luebke, D., Reddy, M., Cohen, J., Varshney, A., Watson, B., and Huebner, R., (2002) Level of Detail for 3D Graphics. San Francisco: Morgan Kaufmann Publishers, p. 432. MacEachren, A. M., (1994) Some Truth with Maps: A Primer on Symbolization and Design. Washington: Association of American Geographers, p. 129. MacEachren, A. M., (1995) How Maps Work: Representation, Visualization, and Design. New York: The Guildford Press, p. 513. MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Science, Special Issue on Geovisualization, 28(1), 3-12. MacEachren, A. M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., (1999a) "Exploring the potential of virtual environments for geographic visualization", Annual Meeting of the Association of American Geographers, Honolulu, HI, 23-27 March, AAG, p. 371 (full paper: http://www.geovista.psu.edu/library/aag99vr). MacEachren, A. M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., (1999b) "Virtual environments for geographic visualization: potential and challenges", Proceedings of the A CM Workshop on New Paradigms in Information Visualization and Manipulation, Kansas City, KS, Nov. 6, 1999. McCleary, G., Jenks, G., and Ellis, S., (1991) "Cartography and map displays", In: Ellis, S., (ed.), Pictorial Communication in Virtual and Real Environments. London: Taylor and Francis, pp. 76-96. Monmonier, M., (1991) How to Lie with Maps. Chicago, IL: University of Chicago Press, p. 176. Okoshi, T., (1976) Three-Dimensional Imaging Techniques. New York: Academic Press. Paetsch, O. and Kuhn, W., (1999) Realtime Visualization of Steel Structures in Fire Engineering. Online: http://www.zib.de/Visual/projects/fireeng-vis/ Piper, B., Ratti, C., and Ishii, H., (2002) "Iluminating clay: a 3-D tangible interface for landscape analysis", Proceedings of Conference on Human Factors in Computing Systems (CHI '02), Minneapolis, Minnesota, USA, 20-25 Apr. 2002. Reddy, M., Iverson, L., Leclerc, Y., and Heller, A., (2001) "GeoVRML: open web-based 3D cartography", Proceedings of the International Cartographic Conference (ICC2001), Beijing, Online: http://www.geovrml.org (6-10 Aug. 2001). Rhyne, T.-M., (2002) "Computer games and scientific visualization", Communications of the A CM, 45(7), 40. Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., and Hedley, N. R., (2001) "Cognitive and usability issues in geovisualization", Cartography and Geographic Information Science, 28(1), 61-75. Stapleton, C., and Hughes, C., (2003) "Interactive imagination: tapping the emotions through interactive story for compelling simulations", IEEE Computer Graphics and Applications, 23(5), 11-15. Taylor, F. D., (1993) "Geography, GIS and the modem mapping sciences: convergence or divergence?", Cartographica, 30(2 & 3), 47-53.
312
J. Wood et al.
Upson, C., Faulhaber, T., Kamins, D., Schlegel, D., Laidlaw, D., Vroom, J., Gurwitz, R., and van Dam, A., (1989) "The application visualization system: a computational environment for scientific visualization", IEEE Computer Graphics and Applications, 9(4), 30-42. Ware, C., and Osborne, S., (1990) "Exploration and virtual camera control in virtual three dimensional environments", A CM SIGGRAPH Computer Graphics, 24(2), 175-183. Watt, A., (2000) 3D Computer Graphics. London: Addison Wesley. Whyte, J., (2002) Virtual Reality and the Built Environment. Oxford: Architectural Press, p. 150. Wise, T. A., Thomas, J. J., Pennock, K., Lantrip, D., Pottier, M., Schur, A., and Crow, V., (1995) "Visualizing the non-visual: spatial analysis and interaction with information from text documents", Proceedings IEEE Information Visualization '95, Los Alamitos, CA, IEEE Computer Press, pp. 51-58. Wood, J., Fisher, P., Dykes, J., Unwin, D., and Stynes, K., (1999) "The use of the landscape metaphor in understanding population data", Environment and Planning B: Planning and Design, 26, 281-295.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter
Multim
15
im parvo
-
Many Things in a Small Place
Jo Wood, Department of Information Science, City University, London ECIV 0HB, UK
Keywords: mipmapping, multi-scale, digital elevation model, terrain, surface modelling, level of detail, texturing
Abstract The notion of scale is decomposed into three elements, namely cartographic scale (ratio of referent to symbolic size), sampling frequency, and spatial extent. These elements are shown to affect both the measurements we make of spatial phenomena, as well the way in which such phenomena can be represented in a visualization environment. Visual techniques that highlight these scale dependencies are considered. In particular, the rendering heuristic of mipmapping is suggested as a way of allowing intelligent surface generalization techniques to be rendered and explored in real-time in a dynamic 3D visual environment.
15.1
Introduction
Representing information using a 3D metaphor is just one of many ways in which we may choose to assemble and visualize multi-dimensional data. As we have seen in the introduction to this section (Wood et al., this volume (Chapter 14)), there may be many motivations for doing this. In this chapter, we explore one application area where viewing multiple dimensions of a dataset is necessary, and the techniques of 3D visualization can prove useful - that of the scale dependency of surface properties. Scale as a notion has been viewed in several ways, depending to a large extent on the traditions of those dealing with it. Here we will consider three views of scale that correspond approximately to those identified by Lam and Quattrochi (1992). The traditional cartographic view of scale is that it identifies the relationship between the magnitude of a cartographic referent (the phenomenon being represented), and that of its cartographic representation. Large-scale maps might possess a simple linear ratio between the two, often expressed as a representative fraction (1:2500, 1 in. to the mile, etc.). This idea, while appropriate for static, paper-based maps, becomes more limited when we consider the dynamic possibilities offered by computer-based representation. 313
314
J. Wood
To those representing spatial data in a GI system, the size of cartographic representation is unlikely to be fixed. For example, the size of computer monitor will affect the cartographic scale of representation, or the user themselves may select and combine data at a range of cartographic scales. When we consider dynamic 3D representation such as that produced when simulating a "flythrough" over a landscape, the notion of a fixed scale of representation has very little meaning. In such circumstances, we can break down scale into two semi-independent properties - the level of detail with which a referent is represented (also referred to as resolution, or grain; Bian, 1997), and its spatial extent (also referred to as domain or geographic scale; Bian, 1997). In many cases, as level of detail increases, extent decreases. By varying the two properties in this way, we can simulate the process of zooming into, and out of, a cartographic representation, exploiting the similarity with the way in which our view of real-world objects changes as we move closer and further away from them. When we consider continuous surface models of spatial information, from terrain models to density surfaces to more abstract statistical fields, we can consider scale in a third way. By definition, a continuous surface cannot be completely represented using discrete data models. Whether using raster digital elevation models (DEMs) or networks of tessellating triangles (e.g., TINs), we are forced to sample a surface in order to provide its digital representation. Scale can be considered as some form of description of the nature of that sampling. For example, the resolution of a raster DEM, the minimum vertex separation in a TIN, or the sampling frequency used to model slope and curvature. The problem that all three ideas of scale present to those wishing to explore spatial data is that in many cases, the properties of those data will vary depending on the scale at which they are observed. What is more, frequently there is no "best" scale at which to make those observations. The problem becomes one of identifying the extent to which the properties of a dataset depend on their scale of observation, and in some way, of characterizing the nature of that dependency. There is a long history of attempts to characterize scale dependency of spatial data including geostatistics (Oliver and Webster, 1986; Mulla, 1988), fractal modelling (Davies and Hall, 1999), and Spectral and Wavelet analysis (Pike and Rozema, 1975; Gallant and Hutchinson, 1996). However, when visualizing data, dynamic 3D visualization has a particularly useful role since it involves rendering data at a range of scales that can change dynamically in a predictable and controllable manner (Wood, 2002).
15.2
Scale in Continuous Surfaces
Before we consider techniques for 3D representation, let us consider one example of the problem introduced by scale dependency Figure 15.1 shows a profile that could represent any 1D continuous dataset. Consider that we are interested in the properties of the dataset at location a on the curve. Point based properties are relatively unambiguously measured and visualized (a scalar quantity y measured on the Y axis). However, if we wish to observe properties that are a function of some linear trend in the data, such as roughness or slope, we must choose both an extent and level of sampling frequency over which to make our observations.
Multim
im parvo
-
Many Things in a Small Place
315
S1
IS2 Is 3
Figure 15.1. Properties of a 1 D profile. The surface characteristics at location a on the profile vary according to the scale at which they are measured. Slope, convexity, roughness, and feature type all vary with scale.
For example, measured over scale s l, the location is part of a local maximum and slopes steeply to the right. Measured over scale s2, it is approximately flat and sits within a regional minimum. Both sets of measurements may be equally valid, and when exploring a dataset it can be useful to be made aware of their uncertainty caused by such variation (Fisher and Wood, 1998; Fisher et al., 2004). This has very real implications, not only for the measurement and analysis of surface properties (e.g., those fed into hydrological or landslip risk models) but also for the way in which visualizations of such properties are interpreted by users wishing to understand the feature represented by the surface. In a simple 1D case, single static visual representation may be sufficient to indicate such scale dependencies. In 2D surfaces (occupying 3D space), such dependencies can be more difficult to represent in a single image (see Figure 15.2). Techniques such as multi-scale parameterization (Wood, 1998), or fractal characterization (Mark and Aronson, 1984; Tate and Wood, 2001) can be used to quantify the way in which measurable surface properties vary with both the level of detail and extent over which they are measured. As part of a visualization process, these properties can be measured interactively. This allows the spatial variation in scale dependency to be explored (Figure 15.3). By updating visual representation dynamically, the multiple characteristics of the same location can be explored while reducing the problem of information overload.
15.3
3D Rendering
As we have seen, dynamic rendering of data in three dimensions can provide an alternative way of visualizing multiple scales of information. Such rendering requires relatively high-powered computation in order to create the illusion of smooth movement within a 3D space. Many technologies exist for such rendering (e.g., VRML, Java3D, Direct3D, OpenGL, etc.), and cheap graphics processing hardware, driven largely by the gaming market, exists for most desktop computing platforms.
316
J. Wood
Figure 15.2. Properties of a 2D area. The 50-m square area identified in the upper photograph is roughly hyperbolic (saddle shaped). At broader scales it appears as part of a channel, a ridge and a peak. Other properties such as aspect, slope, and roughness are also dependent on scale of measurement.
Programming with these technologies tends to involve manipulating three aspects, each of which has a direct analogy with conventional GIS handling of spatial data (Figure 15.4). A 3D world must be constructed with some underlying geometry. This may be as simple as a single plane, or consists of multiple objects each with their own 3D geometry. This has a direct analogy with the spatial data structures used by GIS (spatially referenced rasters, vector data structures, etc.). In both cases, these structures may be adaptive, changing in complexity according to the amount of detail or variation that is to be represented by them (Roettger et al., 1998; de Boer, 2000). The ways in which such structures adapt are likely to be dependent on both the spatial autocorrelation and distribution of the features being modelled (Kahler et al., 2003). Projected onto geometric structures in a 3D rendering environment, must be some surface texture. This texture may be minimal, in simply providing an indication of the spatial location of the underlying geometry. However, texture is frequently used to simulate surface detail or the propagation of light over the surface. Surface textures can be regarded as analogous to the attributes associated with GIS data structures. In many
Figure 15.3. Multi-scale query of a DEM with 2 m horizontal resolution. The graph on the right shows the azimuthal direction of steepest slope measured over a range of spatial extents from 3 to 99 pixels. These represent measurements taken within local "kernels" ranging from 4 m (adjacent to centre of graph) to 200 m (outside edge of graph) wide. The location and maximum spatial extent over which aspect is measured is indicated by the cursor on the left map. As the cursor is moved, the graph updates, leaving a "trail" of previous measurements. This allows regions sharing similar scale dependencies to be identified, as well as characteristic scales at which there is little variation in measurement (approximately 120-150 m in this example).
318
J. Wood
Figure 15.4. The elements of 3D rendering: (a) geometry; (b) texture; (c) viewing transformation.
cases, it is common to map attributes directly onto textures rendered in three dimensions (the example used in Figure 15.4b uses slope magnitude). However, unlike most GIS, there need not be a direct equivalence between the scale (in particular the resolution) of the textural representation, and the scale of the underlying geometry. Finally, dynamic rendering in three dimensions requires the manipulation of the viewing transformation. By changing the projection of textures on the screen, movement within a 3D space can be simulated. In order to maintain the illusion of fluid movement within space, viewing transformations are typically calculated and changed 10-100 times per second. The analogy with conventional GIS rendering is in the calculation of map and viewing projection. Typically, a user of a GIS can control the extent and location of map representation shown on the screen. This allows a viewer to "pan and zoom" within a relatively restricted space or view global spatial data projected onto a plane. There is, however, one important difference between the way textures and geometry are related in 3D visualizations and conventional GIS. Processing geometry tends to be far more computational expensive than manipulating continuous rasters of surface texture. Therefore, in order to process scenes tens of times per second, the geometry of most 3D systems will be far less detailed than that represented by the surface texture. For example, in Figure 15.4a underlying geometry is recorded at approximately
Figure 15.5. Controlling sense of scale with surface texture: (a) underlying geometry; (b) raster resolution used for GIS processing; (c) sub-pixel surface texture.
Multim im parvo -
Many Things in a Small Place
319
60 m resolution. The calculation of surface texture shown in Figure 15.4b, however, is shown at a 4 m resolution. This is in contrast to conventional Cartography and GIS, where attributes tend to be recorded at either a similar or coarser level of detail than geometry. 3D rendering of textures allows us to exploit the fact that we judge shape visually by the interaction of the surface with light as much as, if not more than, the impact shape has on occluding parts of our view. This makes it possible to manipulate visually, a sense of scale in 3D space (Figure 15.5). By changing viewpoint we can control all three elements of scale, namely the level of detail (foreground shows more detail than background), extent of view (foreground shows a narrower view than background), and the cartographic scale (ratio between size of referent and its size on screen).
15.4
Mipmapping
Rendering textures with detail at the sub-GIS raster resolution (as shown in Figure 15.5c) could potentially lead to massive rendering overheads. A raster surface model of 1000 x 1000 pixels with a typical sub-pixel variation of 128 x 128 cells would lead to processing of 1.6 x 10 l~ texture pixels without some kind of heuristic to optimize the process. The most widely used heuristic for texture mapping is known as m i p m a p p i n g (from the Latin m u l t i m im p a r v o , or "many things in a small place") whereby the level of detail with which a texture is rendered is dependent on the apparent distance away from the viewer. In practice, this is calculated by computing the density of texture pixels according to any given viewing projection. If the density is above a given threshold, the texture is replaced by a coarser generalized version. A hierarchy of increasingly generalized textures are pre-computed by the rendering engine and the appropriate one is selected according to the texture density (Woo et al., 2003). Figure 15.6 shows typical results of mipmapping. To illustrate the process, textures of contrasting colours have been computed at different mipmap levels. As the viewer changes position relative to the surface, any given location may be mapped with a range of possible textures. This process has two important implications when considering visualization of scale-dependent behaviour. First, the heuristic, which is programmed into the hardware of many graphics processing chips, greatly increases the efficiency and speed with which texture-rich scenes can be rendered. It allows surface properties to be represented visually, simultaneously over several orders of magnitude of scale. There is, however, a less widely used application of this technique that allows us to consider scale-based variation more explicitly. Low-level 3D programming environments such as OpenGL, allow the programmer to control the textures supplied to each level of the mipmap hierarchy. It is therefore possible to measure a given property at a set of different scales, produce some visual representation of each scale, and load the representations into different levels of the mipmap hierarchy. In the case of representations of topography, this process might embody complex and computationally demanding processes of cartographic generalization, yet when rendered in this way these generalizations can be explored in real time. In the case of surface measurements,
W
w
0
i
,_
1
L
Figure 15.6. Four levels of miprnapping (yellow, blue, green, and red) showing how texture is automatically changed depending on the viewer position and the apparent distance at which six views of a surface are rendered.
I
.
Figure 15.7. Surface curvature measured at contrasting scales (left hand images) are passed to the mipmap texture buffer. Hardware selects the appropriate texture when rendering according to its apparent distance from the viewer (right hand images). Figure uses data derived from an Ordnance Survey Panorama 50m DEM, Crown Copyright.
322
J. Wood
consider the example illustrated in Figure 15.7. A property of the surface such as local curvature is computed for the entire surface over a range of spatial extents. Visual representations of each set of measurements are transferred to the mipmap buffer of the graphics hardware - each level of the mipmap hierarchy corresponding to a different scale of measurement. When rendered, the graphics hardware selects an appropriate level of the mipmap buffer according to the apparent distance any portion of the surface is from the viewer (as illustrated in Figure 15.6). As a result, distant peaks show measures of curvature calculated over large spatial extents, while foreground shows similar measures, but calculated over much more local spatial extents. As the viewer moves around the landscape, these scale-dependent representations change dynamically. The process of pre-computing mipmap textures provides an example of an important link between rendering technology and issues of cartographic representation. It allows intelligent scaled-based cartographic generalization (as opposed to simple scalebased resampling or filtering) to be implemented in real time while navigating within a 3D space. Different scale dependent generalizations of a surface can be computed, adopting as simple or complex a generalization process as desired. These pre-computed generalizations can be passed to different levels of the mipmap hierarchy as part of the rendering initialization process. During the rendering itself, efficient hardware-accelerated algorithms use the appropriate level of generalized representation according to the texture density of the viewpoint. Taking advantage of rendering hardware to increase the speed with which such a view can be changed presents us with new forms of geovisualization. In a similar way, D611ner this volume (Chapter 16) allows pre-computed representations of a phenomenon to be passed to the texture buffer of graphics hardware and then rendered via a user-controlled "magic lens". D611ner's approach gives the user explicit control over the form of visual representation using the metaphor of a moving magnifying glass. In the approach described here, the choice of visual representation is governed by the metaphor of movement in space and the properties of the view within that space. Such apparent movement in real-time can be used to explore spatial variation in scale dependency.
15.5
Conclusions
This chapter has considered three specific elements of the more general idea of scale, and highlighted ways in which 3D visualization may be used to explore these elements. Cartographic scale (the ratio between a referent's size and its representation's size), level of detail (resolution or sampling frequency) and spatial extent all affect the measurements and observations we can take from a phenomenon. This scale dependency is well suited to dynamic 3D representation since our every day interpretation of changing 3D views involves cognitive reconstruction of some of these scale dependencies. By taking advantage of the heuristics designed to improve the efficiency of 3D surface rendering, we can explicitly map scale dependent measures in a dynamic 3D environment. This extends the purpose for which mipmapping was originally designed, but in doing so offers new opportunities to create environments in which to explore scale dependencies.
Multim im parvo - Many Things in a Small Place
323
Acknowledgements The author is grateful for input and clarification of ideas from the other authors in this section as well as comments from two anonymous reviewers.
References Bian, L., (1997) "Multiscale nature of spatial data in scaling up environmental models", In: Quattrochi, D., and Goodchild, M., (eds.), Scale in Remote Sensing and GIS, pp. 13-26. Davies, S., and Hall, P., (1999) "Fractal analysis of surface roughness by using spatial data", Journal of the Royal Statistical Society, Series B, 61 (1), 3-37. de Boer, W., (2000) Fast Terrain Rendering Using Geometrical MipMapping. Online: http://www.flipcode.com/tutorials/tut_geomipmaps.shtml (23/10/03). Fisher, P., and Wood, J., (1998) "What is a mountain? Or the Englishman who went up a Boolean geographical concept but realised it was fuzzy", Geography, 83(3), 247-256. Fisher, P., Wood, J., and Cheng, T., (2004) "Where is Helvellyn? Fuzziness in multi-scale analysis of landscape morphometry", Transactions of the Institute of British Geographers, 29(1), 106-128. Gallant, J., and Hutchinson, M., (1996) "Towards an understanding of landscape scale and structure", In: NCGIA, (ed.), Proceedings of the Third International Conference on Integrating GIS and Environmental Modelling. Santa Barbara: University of California, Online" http://www.ncgia.ucsb.edu/conf/SANTA_FE_CD-ROM/ sf_papers/gallant__j ohn/paper.html Kahler, R., Simon, M., and Hege, H.-C., (2003) "Interactive volume rendering of large sparse data sets using adaptive mesh refinement hierarchies", IEEE Transactions on Visualization and Computer Graphics, 9(3), 341-351. Lam, N., and Quattrochi, D., (1992) "On issues of scale, resolution, and fractal analysis in the mapping sciences", Professional Geographer, 44(88-98). Mark, D., and Aronson, P., (1984) "Scale-dependent fractal dimensions of topographic surfaces: an empirical investigation with applications in geomorphology and computer mapping", Mathematical Geology, 16(7), 671-683. Mulla, D., (1988) "Using geostatistics and spectral analysis to study spatial patterns in the topography of southeastern Washington State, USA", Earth Surface Processes and Landforms, 13, 389-405. Oliver, M., and Webster, R., (1986) "Semivariograms for modelling the spatial pattern of landform and soil properties", Earth Surface Processes and Landforms, 11, 491-504. Pike, R., and Rozema, W., (1975) "Spectral analysis of landforms", Annals of the Association of American Geographers, 65(4), 499-516. Roettger, S., Heidrich, W., Slusallek, P., and Seidel, H.-P., (1998) "Real-time generation of continuous levels of detail for height fields", In: Skala, V., (ed.), Proceedings. WSCG '98, pp. 315-322, Online" http://wwwvis.informatik.uni-stuttgart.de/-~roettger/html/Main/papers.html
324
J. Wood
Tate, N., and Wood, J., (2001) "Fractals and scale dependencies in topography", In: Tate, N., and Atkinson, P., (eds.), Scale in Geographical Information Systems. Chichester: Wiley, pp. 35-51. Woo, M., Neider, J., Davis, T., and Shreiner, D., (2003) OpenGL Programming Guide: The Official Guide to Learning OpenGL Version 1.4. London: Wesley. Wood, J., (1998) "Modelling the continuity of surface form using digital elevation models", In: Poiker, T., and Chrisman, N., (eds.), Proceedings, Eighth International Symposium on Spatial Data Handling. Vancouver: IGU, pp. 725-736. Wood, J., (2002) "Visualising the structure and scale dependency of landscapes", In: Fisher, P., and Unwin, D., (eds.), Virtual Reality in Geography. London: Taylor and Francis, pp. 163-174.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter 16
Geovisualization and Real-Time 3D Computer Graphics Jfirgen D611ner, Hasso Plattner Institute at the University of Potsdam, Helmert-Str. 2-3, 14482 Potsdam, Germany
Keywords: 3D computer graphics, 3D rendering, real-time rendering
Abstract In this contribution, we would like to outline the impact of real-time 3D computer graphics on geovisualization. Various real-time 3D rendering techniques have been developed recently that provide technical fundamentals for the design and implementation of new geovisualization strategies, systems, and environments. Among these key techniques are multi-resolution modelling, multi-texturing, dynamic texturing, programmable shading, and multi-pass rendering. They lead to significant improvements in visual expressiveness and interactivity in geovisualization systems. Several examples illustrate applications of these key techniques. Their complex implementation requires encapsulated, re-usable, and extensible components as critical elements to their success and shapes the software architecture of geovisualization systems and applications.
16.1
Introduction
Geovisualization represents an integral part of most geographic information systems (GIS) and geo-related software systems. Geovisualization is migrating into many application domains as a ubiquitous element of software-intensive systems and their user interfaces due to the broad diffusion of geographic data. Car navigation systems, facility management and control systems, and location-based services providing digital maps on mobile devices represent just a few examples of widely spread uses of geovisualization. As an independent discipline, real-time 3D computer graphics has become a major field in Computer Science and media due its applications in diverse fields such as virtual reality, video games, computer-aided design, and the film and printing industries. Mass-market products such as video consoles have pushed the development of graphics hardware. As a consequence, powerful 3D graphics hardware now forms regular part of standard PCs. This rapid development is resulting in the emergence of 325
326
J. D611ner
hardware-accelerated 3D rendering techniques that provide high-quality imaging, enable interactivity in dynamic environments, and cope with detailed and complex 3D scenery. The interplay of geovisualization, Cartography and 3D computer graphics is becoming tighter because technological and theoretical advances in each of these areas depend increasingly upon each other. Their principle relationships can be characterized as follows: 9
9
9
16.2
Geovisualization represents a fundamental area of application of 3D computer graphics. Since geo-information represents an essential resource of most information systems, such as in mobile and pervasive computing, 3D computer graphics will be forced to provide solutions to fulfil the specific needs of those applications based on and handling geo-data. Real-time 3D computer graphics offer a developing array of sophisticated image synthesis and image processing technology. If dedicated to geographic information, the technology can improve and extend the capabilities of geovisualization systems because their concepts and implementations are significantly determined by 3D computer graphics technology. We identify multi-resolution modelling, multi-texturing, dynamic texturing, programmable shading, and multi-pass rendering as key techniques that will contribute to future geovisualization systems and applications. Cartography and GIS, the roots of geovisualization, can contribute strategies for visual design and representation as well as opportunities for linking visual representations with databases. For instance, cartographic generalization schemes, which address issues of scale and semantically interrelated features, can be applied to multi-resolution models. This way, geovisualization could push developments in new directions and improve existing techniques in 3D computer graphics.
Key Techniques of Real-Time 3D Rendering
Here we describe core real-time 3D rendering techniques and their applications and possible impact on geovisualization. For each technique, we give a brief definition, an outline of underlying implementation concepts, and applications to geovisualization. In computer graphics, we distinguish traditionally between image synthesis, image analysis, and image processing. Real-time 3D rendering denotes specialized image synthesis techniques and aims to achieve high visual complexity, high degrees of realism, and interactive frame rates. Implementations are mostly based on algorithms and data structures that take advantage of graphics hardware acceleration. A detailed introduction to generic concepts of state-of-the-art real-time 3D graphics and their underlying mathematics can be found in Akenine-M611er and Haines (2002). Computer graphics software systems can be classified in terms of their software architecture. 9
Low-level graphics systems such as OpenGL (Woo et al., 1999) or DirectX from Microsoft are graphics libraries that provide elementary functions and data
Geovisualization and Real-Time 3D Computer Graphics
327
structures used through an application programming interface (API) and a system programming language such as C+ + or Java. Higher level graphics systems such as OpenInventor (Strauss and Carey, 1992), Java3D, and the Virtual Rendering System (VRS) (D611ner and Hinrichs, 2002) provide object-oriented class libraries and frameworks. They abstract from underlying low-level rendering system APIs by providing building blocks that encapsulate algorithms and data structures. Graphics scene description languages such as the Virtual Reality Modeling Language (VRML/X3D) or Pixar's RenderMan Language define formats and specialized languages to describe 3D scenes and their dynamics; they aim to provide interfaces for non-programmers. Systems that support those languages are typically based on low-level and/or higher level graphics systems. The real-time 3D rendering techniques considered here are all implemented directly on top of low-level libraries. Still they are not part of higher level graphics systems or graphics scene description languages. Although, we expect them to be integrated into generic graphics systems in the long run, we do not foresee this occurring for techniques specifically relating to geovisualization.
16.2.1
Multi-resolution modelling
Definition. Multi-resolution modelling is concerned with capturing a possibly wide range of levels of detail of a given graphics object and allows for reconstructing any one of those levels on demand. Surface simplification is a restricted form of multiresolution modelling. For a given 3D polygonal model, surface simplification is used to derive approximations of that model with lower level of detail depending on defined quality criteria. The quality at a given level of detail is characterized by the model resolution, which refers to the density of geometric elements, and the accuracy, which measures the error made in approximating the original model. Level-of-detail models enable rendering techniques to cope with complex 3D models in real-time. Implementation. We can distinguish static techniques that pre-compute a fixed number of level-of-detail models, and dynamic techniques that generate level-of-detail models on the fly depending on screen resolution and camera viewing conditions, such as the view-dependent progressive meshes for general polyhedral objects introduced by Hoppe (1996). As basic geometric concepts, techniques are typically based on vertex decimation, vertex clustering, or edge contraction. Implementations that focus on terrain modelling include the restricted quad-tree triangulations for terrain models (Pajarola, 1998), the smooth view-dependent progressive mesh (Hoppe, 1998), and multitriangulations (DeFloriani et al., 2000). Geovisualization Applications. Multi-resolution models are essential for handling high-resolution geographic data. Without such models, applications would not be able to achieve interactive frame rates.
328
J. D611ner
For digital terrain models, a number of specialized multi-resolution modelling techniques have been developed that can generally be classified as either consisting of regular models for gridded (raster) data or irregular models for irregular triangle meshes. DeFloriani and Magillo (2002) give a detailed comparison of regular and irregular multiresolution terrain models and discuss advantages and disadvantages of both representation types. Recently, multi-resolution modelling started to develop customized and optimized variants for digital terrain models and city models. These are still subject to continuing research and include: 9
9
9
9
Hybrid terrain models that integrate triangulated irregular networks (TINs) and regular grids in a single multi-resolution model. In general, TINs represent explicitly modelled geometric elements of the terrain model (e.g., riverbeds), whereas grids represent the reference grid. This way, the advantages of both representation types can be combined but the rendering algorithm becomes less efficient. Appearance-preserving terrain models, which ensure high perceptual image quality regardless of the current simplification state. Illumination and shading information is encoded by 2D textures that are applied to the level-of-detail terrain surface (D611ner et al., 2000). Figure 16.1 compares Gouraud shading and appearance-preserving shading for level-of-detail terrain surfaces. Integrated multi-resolution terrain modelling and terrain texturing, which enable innovative means of visualizing thematic data associated with terrain surfaces (see w16.2.2). Multi-resolution 3D city models, which take care of the building characteristics and support generalization for blocks of buildings instead of straightforward polygon reduction.
Since multi-resolution techniques differ in strategy, quality and efficiency, there appears to be no single generic solution. Computer graphics hardware implements neither generic variants of multi-resolution techniques nor specialized variants for digital terrain models. Therefore, multi-resolution techniques represent essential components of geovisualization libraries and systems.
16.2.2
Texturing and multi-texturing
Definition. Texturing represents a fundamental graphics operation that maps multidimensional images to multi-dimensional spaces, for example, mapping a 2D digital image to a canvas or a 3D surface. The principle applications of texturing are detailed in Haeberli and Segal (1993). Multi-texturing represents a graphics operation that simultaneously maps multiple textures to a single multi-dimensional space, and defines the image operation by which texture values are combined. For example, a RGB texture and a luminance texture could be mapped onto a single surface multiplying colour values and luminance values. Texturing can be applied to any rasterized 2D and 3D primitive.
Geovisualization and Real-Time 3D Computer Graphics
329
Figure 16.1. Comparison of Gouraud-shading and appearance-preserving shading. (a) Triangulated terrain surface. (b) Gouraud-shaded terrain. (c) Texture-shaded terrain. In (c) more visual detail is visible compared to (b) using the same triangulated terrain surface. In particular, structures such as the river are preserved even at a coarse level of detail.
330
J. D611ner
Implementation. Textures require texture memory that typically resides on the graphics hardware. For each texture, the application can control texturing by parameters that include: 9 9 9 9
texture texture texture texture
coordinates (texture coordinate transformations); mip-mapping (i.e., an image pyramid - a scheme for sampling textures), filtering (e.g., nearest-neighbour interpolation, linear interpolation), mode (e.g., modulating, blending, replacing).
Multi-texturing requires multiple texturing units on the graphics hardware. The number of texturing units defines the maximal number of simultaneously active textures. If more textures should be applied, the same scene must be rendered multiple times using different textures (multi-pass rendering) to generate a single image. For each active texture, control parameters can be independently specified. For example, texture 1 can be an RGB image modulated with the primitive surface colour, texture 2 can be a luminance image blended into the primitive surface. Geovisualization applications. Texturing is the most important technique in geovisualization to make geographic data actually visible. As one of its standard applications, we can drape 2D geo-referenced images such as aerial images, encoded in 2D textures on 3D geometry such as terrain surfaces. Figure 16.2 illustrates how multiple textures are combined on top of a terrain surface. In the field of virtual reality, textures are frequently used to capture the appearance of real objects, for example the facade of a building. In general, textured surfaces improve the degree of realism in geo-environments that depend upon a realistic depiction. A direct application of 3D textures includes the encoding and visualization of volumetric data, for example, geologic data of the soil structure. In addition, volumetric phenomena can be represented, for example, moving and deforming fuzzy geo-objects such as dust clouds. Furthermore, 3D textures can be used to represent time-variant 2D textures in a single component. As the main advantage of this approach, texture filtering allows us to smoothly interpolate between discrete time slices. Since texturing is accelerated by hardware, we are able to process even large time-variant data sets. Of course, there are manifold applications for animated Cartography in this direction. With respect to maps, various applications of multi-texturing exist for geovisualization. They include: 9 9 9
Representation of complex thematic information through a collection of textures. Visual combination of independent textures. Variation of local detail of thematic data through filter textures, e.g., information lens.
Examples of information lenses are illustrated in Figure 16.3. Two independent thematic data layers such as street network and buildings are visually combined, resulting in a complex texture. Inside the lens, however, we locally fade out building information to achieve higher visual clarity. In the same way, information lenses can be used to guide the user's centre of attention through a 3D map. A luminance texture layer
Figure 16.2. Principle of multi-texturing applied to terrain surfaces. (a) Multiple 2D textures represent independent layers of information, e.g., shading, landuse information, and street networks; artificial textures represent masks and blending filters. (b) The final image results from combining texture layers in screen space. This example also illustrates how information lenses are constructed based on multi-texturing.
332
J. D611ner
Figure 16.3. (a) street and land-use information represented as independent thematic information layers, controlled by an interactive information lens that reduces information load inside the lens. Implementation based on multi-texturing. (b) highlight lens to indicate the center of attention. (Images: LandExplorer).
Geovisualization and Real-Time 3D Computer Graphics
333
encodes the shape and intensity of the visual highlight superimposed on the terrain surface. Since information lenses are generated in real-time, the user can control position, size, and shape of lenses interactively.
16.2.3
Dynamic texture generation
Definition. Dynamic texturing denotes the dynamic generation of texture data (i.e., on a per-frame basis) and the immediate deployment of the results by the rendering process. Dynamic texturing enables a wide range of data mapping techniques, in particular, for dynamic data and for animated representations. Implementation. The contents of a texture can be specified in a similar way to 2D vector graphics. The specification consists of a hierarchical description such as the scene graph and contains 2D shapes (e.g., points, lines, and polygons) as well as their graphics attributes (e.g., colour, line width, and line style). That is, it encodes scalable image contents that can be rasterized at any resolution. Prior to rendering the main 3D scene, the description is rendered into an off-screen canvas, called pixel buffer (or P-buffer in OpenGL). After completion, the P-buffer is copied into texture memory. Then, the main scene can be rendered to the on-screen canvas, and it may reference to the texture generated by the P-buffer, that is, the texture can be deployed immediately. Although conceptually simple, on-the-fly texture generation has been one of the major bottlenecks in the past. The P-buffer changed the situation; since both P-buffer and texture memory reside on graphics hardware, no texture data has to be transferred between application memory and graphics memory. Hence, the P-buffer-based texture generation is suitable for real-time rendering. Geovisualization applications. Dynamic texturing enables us to implement dynamic cartographic representations. Figure 16.4 illustrates dynamic texturing in the case of thematic data. The "vegetation" theme is visually mapped at different levels of detail using a different cartographic symbolization. In this example, the choice depends mainly on the camera distance. Each variant of the thematic texture is generated dynamically in an off-screen rendering canvas for each frame to be rendered. Figure 16.5 demonstrates dynamic texturing applied for mapping vector data (Kersting and D611ner, 2002a,b). If we represent vector data by geometric objects (e.g., polygonal borders by line segments), we must adapt them to maintain consistency with the multi-resolution reference surface (e.g., multi-resolution terrain model). If such vectors are not adapted correctly and constantly, artefacts result from improper adjustments. A texture-based representation omits these problems: Vector data can be dynamically rasterized and draped over the multi-resolution reference surface, possibly blended with other thematic textures. Through this method, level-ofdetail management is decoupled from thematic data mapping. The quality of the visual display of vector data can be significantly improved compared to a representation based on geometric objects, and the representation can be seamlessly integrated into the 3D representation of reference surfaces. Furthermore, this kind
334
J. D611ner
Figure 16.4. Different levels of detail for a land-use information layer, selected according to camera-viewer distance. (Images: LandExplorer).
of dynamic texture allows us to dynamically adapt the visual mapping and interactively manipulate vector elements seen in a texture since the underlying vector data description can be manipulated at any time and perform analytic operations such as intersection tests.
Figure 16.5. Example of the visual mapping of vector-data for the case of a pipeline. (a) Mapping based on a geometric representation. Pipeline segments are broken to adapt to the terrain surface. Visual artifacts occur. (b) Dynamic texture-based mapping avoids these problems. Pipeline segments are encoded in a temporary 2D texture and projected onto the terrain surface. (Images: LandExplorer).
Geovisualization and Real-Time 3D Computer Graphics
16.2.4
335
Advanced rendering techniques
The term "advanced rendering techniques" denotes a category of real-time 3D rendering algorithms that achieve a high degree of visual realism and dynamism. In general, these techniques rely on multi-pass rendering and programmable shading. Definition. A multi-pass rendering algorithm needs to process a scene description several times, each time using different rendering settings and resources, whereby the final pass, in general, combines the results of the previous passes into a final image. Definition. A shading language denotes a programming language that is used to specify transformations of vertices and their attributes (per-vertex programs) as well as the calculation of fragment colours based on active textures and light sources (perfragment programs). With multi-pass rendering, we are able to achieve effects such as light reflection, shadows, and bump-mapped surfaces in real-time. Shading languages complement these algorithms; they enable applications to define object-specific and application-specific geometry, lighting, and shading calculations that go beyond standard computer graphics illumination models. Implementation. Typically, the implementation of a multi-pass rendering algorithm does not follow a specific pattern, but is characterized by a direct use and manipulation of low-level rendering resources. The only commonality of most multipass algorithms, however, is the fact that the scene geometry is passed through the rendering pipeline in each pass. Each pass defines the way of interpreting and processing scene geometry. For example, most real-time shadow algorithms calculate a shadow map in the first pass, i.e., a 2D texture that shows the scene from the light source position. This map is used to decide in subsequent passes whether objects are in shadow. In the second pass, the algorithm renders objects exclusively in non-shadowed parts, and in the final pass, it renders objects only in shadowed parts of the resulting image. The concept of programmable shading was introduced originally by Pixar's RenderMan system. The technique is now available in real-time rendering systems, such as OpenGL 2.0 (3Dlabs, 2002). Per-vertex and per-fragment programs are uploaded to graphics hardware and are called for each vertex processed and for each fragment resulting from rasterizing geometric primitives. The per-fragment programs enable developers to define their own lighting and shading algorithms. Geovisualization applications. Bump-mapping (Kilgard, 2000) belongs to the advanced rendering techniques and substantially relies on multi-pass rendering and programmable geometry and shading stages of the rendering pipeline. A bump-map denotes a 2D texture that defines height-offsets for a surface (or, alternatively, normal perturbations). The height-offsets are used by the illumination calculation to redefine the surface normal at each point to be illuminated, i.e., no true geometric deformation is actually applied but the resulting surfaces appear to exhibit a fine-grained structure. In Figure 16.6, the relief-like depiction of the Earth is represented by a triangulated 3D sphere that uses bump-mapping to achieve a more vivid impression. Bump-mapping
336
J. D611ner
Figure 16.6. (a) Tessellated 3D sphere used as reference surface of a globe. (b) Using standard texturing based on an Earth image. (c) Using an Earth image as texture combined with a bump-map. (Images: NVidia Inc.).
allows us to achieve the impression of high geometric complexity without having to model such complexity expensively with geometry. This technique is analogous to the traditional exaggeration employed by cartographers, as a result is a more credible, sophisticated and pleasing visual product that is a visual interpretation, essentially independent of the measured data.
Figure 16.7. Fire simulation implemented by a particle system. Both particle system and water surface use OpenGL programmable per-vertex and per-fragment programs. (Image: HPI, University of Potsdam).
Geovisualization and Real-Time 3D Computer Graphics
16.2.5
337
Example: particle systems
A particle system, introduced to computer graphics by Reeves (1983), models a system of a large number of relatively "small" objects. It can be used to visualize appearance and behaviour of "fuzzy objects" such as natural phenomena, for example, water flows, fires, or dust. Particle systems have also been used to simulate behaviour of groups, flocks, shoals and swarms. A typical particle system consists of thousands of particles with their own states and dynamics. The state includes position, velocity, colour, transparency, weight, lifetime, and further attributes. After each time step, new particles are instantiated and those particles are destroyed whose lifetime terminates. Other attributes such as position and velocity are updated, too. In real-time 3D rendering, we represent each particle as single, independent object with an associated state and behaviour. The attributes_of the whole particle system can be mapped efficiently to graphics hardware, for example, by using the vertex arrays available in OpenGL. In this way, large numbers of geometric primitives can be processed. Attributes are evaluated for lighting and shading purposes by appropriate pervertex and per-fragment programs. Particle systems can be embedded in virtual environments to offer high levels of dynamism and realism. In Figure 16.7, we see an example where particle systems are used to simulate a fire in real time. The dynamic particles are reflected back by the moving surface of the water, which can itself be interacted with. In geovisualization applications, real-time particle systems enable users to see dynamic geographic phenomena and interact with them in their environment. The systems extend the methods of representation that are available and offer a flexible and powerful primitive through which we can visualize and interact with real world phenomena.
16.2.6
Example: interactive, dynamic 3D maps
Interactive, dynamic 3D maps represent visual interfaces used to present, explore, analyze and edit spatial and spatio-temporal data (D611ner and Kersting, 2000). They are used, for instance, in a radio network planning system as main visual interface that allows planers to interactively explore field strength values and to configure the network (Figure 16.8). 3D maps provide powerful design capabilities for map contents compared to current map support in GIS. Their underlying object model introduces abstract building blocks, which are used to construct individual 3D maps. These building blocks do not only include visual primitives but also structural and behavioural primitives. Structural primitives allow map editors to arrange and hierarchically organize 3D map contents. Behavioural primitives define the dynamics and interactivity of 3D maps. The building blocks also support the dynamic design of map contents to facilitate visualizing temporal data and phenomena. Important applications of this level of flexibility include interactive, animated Cartography, virtual geo-environments, and exploratory, visual interfaces for GIS. With respect to real-time 3D rendering, multi-resolution modelling is deployed for digital terrain models and city models. In particular, a hybrid terrain model (D611ner
338
J. D611ner
ffJ e-ct--
O
|
e--
O
O
r ~
co
E c3
CO
O'I U.
Geovisualization and Real-Time 3D Computer Graphics
339
et al., 2000) composed of a base grid, which can be refined by additional TINs and overlaid by multiple textures, provides a high level of expressiveness. Multiple thematic information layers are represented by individual textures and rendered by multitexturing. Vector-data are rasterized dynamically in 2D textures, which can be visualized together with static 2D and 3D textures using multi-texturing. Finally, 3D city models can be integrated into the terrain representation whereby a combined geometry-driven and semantic-driven generalization scheme is used to reduce their graphical complexity to achieve real-time frame rates.
16.3
Complexity of Geovisualization Software
Geovisualization systems incorporate a number of graphics-related subsystems, which, in general, also represent the system layers of their software architecture: 9 9 9
9
Low-level rendering system. Responsible for 2D and 3D image synthesis, e.g., OpenGL. High-level graphics system. Responsible for scene composition, scene animation, and user interaction with scenes, e.g., OpenInventor. Visualization system. Responsible for mapping data to 2D and 3D graphics, and provides a number of domain-specific visualization strategies and algorithms. Examples are VTk as a general-purpose visualization toolkit and GeoVISTA Studio (Takatsuka and Gahegan, 2002) for geocomputation and geographic visualization. User-interface system. Responsible for building and managing graphical user interfaces. Examples include Trolltech's Qt library or the GIMP toolkit GTK +.
Figure 16.9 illustrates the general components of a geovisualization software system. Typically, the graphics and rendering systems provide basic functionality to
Figure 16.9. Principle software components of geovisualization systems and applications.
340
J. D611ner
integrate graphics into any kind of user-interface system. Most user-interface systeh, provide specialized user-interface components, called widgets, for frequently use~ graphics and rendering systems. In addition, the architecture may also includ, subsystems that are responsible for data management and application-specific services Due to the number of subsystems and their functionality, environments designet and suitable for geovisualization are complex software systems. The implementation of: geovisualization technique or system exposes developers to the complexity of each o these subsystems. That is, not only will developers have to program based on the API o the visualization layer but also will have to at least understand the API of the graphics anl rendering systems. In the case of software libraries, complexity is indicated by th, number of data structures, functions, and classes provided of the API. The VRS, fo example, comprises about 300 classes with a total number of more than 6000 methods Once a system has been built, software complexity has a significant impact o maintenance and on its extensibility and reusability.
16.3.1
Geovisualization software challenge
Geovisualization is deeply associated with two major issues that form a signif software challenge (and result in a software dilemma): 9 9
There is a large gap between today's 3D graphics technology and the tech currently used in geovisualization systems and applications. There is a large gap between current software technology and technolc in many scientific and commercial geovisualization projects.
Rapid progress in real-time 3D computer graphics, in particular with hardware and rendering techniques, and increasing software complexity in software dilemma.
16.3.2
Software engineering for geovisualization
A major reason for the technology gap between geovisualization and computer graphics is the inherent complexity of implementations of t techniques. For example, the integration of multi-resolution terrain modr terrain texturing causes technical problems: Each technique traverse description multiple times. In each pass, the rendering algorithm resources such as the colour, depth, and stencil buffers or textur solutions exist to implement a single technique, but it is difficult to cc techniques in a single rendering algorithm. The source code of th~ cannot be simply nested or merged because resource sharing Consequently, it is difficult to design and implement general-purpo' these advanced rendering techniques. Today's low-level 3D rein intend to make these rendering techniques available at an abstrac' because they are still the subject of ongoing research. Higher le, faced with the same situation. Furthermore, the complexit
Geovisualization and Real-Time 3D Computer Graphics
341
techniques has reached a level at which even computer graphics scientists are confronted with barriers in system development. Component-oriented software development (Szyperski, 1997) for geovisualization techniques appears to be a solution compared to the case of general-purpose solutions. A geovisualization component could be defined as a unit that implements a specific geovisualization strategy or metaphor, offers a software interface to access its services, and can be integrated in applications as part of their user interfaces. With respect to the software architecture, geovisualization components encapsulate the visualization system layer, the low-level 3D rendering layer, and the higher level 3D graphics layer (Figure 16.10) and, therefore, simplify the API from the point of view of the geovisualization system developer. For example, the concept of dynamic, interactive 3D maps can be designed and implemented as a geovisualization component. Such a component has to encapsulate various real-time 3D rendering techniques, for instance, multi-resolution terrain models, multi-texturing, and dynamic texturing. This approach has been exemplified by the LandExplorer project (Hasso-Plattner-Institute, 2003), which defines a software framework for developing interactive dynamic 3D maps. Equally, an analogous 3D landscape visualization component could provide photo-realistic views of landscapes including their vegetation and buildings. Geovisualization components could serve as a basis for using and reusing visualization strategies and metaphors; as building blocks they could be deployed in user interfaces of geovisualization environments and GIS. However, the component design is faced with limitations in today's component technology. For instance, components are mostly bound to certain software and/or hardware platforms. As a trade-off, we can achieve a high degree of reusability and compatibility if we encapsulate geovisualization components as user-interface components for
Figure 16.10. Geovisualization components and their position in the software architecture of geovisualization systems.
342
J. D611ner
frequently-used user-interface systems, assuming that application developers are likely to integrate geovisualization components as part of an interactive software system.
16.4
Conclusions
Real-time 3D computer graphics, in particular real-time 3D rendering, offers a variety of advanced solutions for geovisualization systems and environments. The combination of multi-resolution modelling, multi-texturing, dynamic texturing, and advanced rendering techniques enable developers to implement interactive, dynamic 3D representations. The resulting visual quality and means of expression represent a major technological step. The implementation complexity of real-time 3D computer graphics can be encapsulated and managed by appropriate methods of software engineering, for example, component-oriented design. Encapsulated, reusable, distributable, and competing geovisualization components will emerge in the long run. These ready-to-use geovisualization components facilitate construction, deployment, specialization, and comparison of geovisualization concepts, techniques and metaphors; they raise the level of abstraction at which geovisualization applications can be built. These are key requirements of the geovisualization community as indicated by the recent research agenda for geovisualization (MacEachren and Kraak, 2001; Dykes, MacEachren and Kraak, this volume (Chapter 1)). We should also note that we have not as yet invested in open-source initiatives similar to those for games engines. Such a step could also vitalize software development in geovisualization. In the field of geovisualization, advances in real-time 3D computer graphics should be rapidly considered, evaluated and deployed. In many cases, we will have to further customize and optimize individual techniques for geovisualization purposes. The techniques introduced here and architecture suggested offer ample opportunity to achieve these objectives.
Acknowledgements The author would like to thank Konstantin Baumann, Henrik Buccholz, and Oliver Kersting for their collaboration.
References 3Dlabs, (2002) 3Dlabs OpenGL 2.0 Specifications. Online: http://www.3dlabs.com/support/developer/ogl2 (23/10/03). Akenine-M611er, T., and Haines, E., (2002) Real-Time Rendering, A.K. Peters. DeFloriani, L., and Magillo, P., (2002) "Regular and irregular multi-resolution terrain models: a comparison", Proceedings ofACM GIS 2002, pp. 143-147. DeFloriani, L., Magillo, P., and Puppo, E., (2000) "VARIANT: a system for terrain modeling at variable resolution", Geoinformatica, 4(3), 287-315.
Geovisualization and Real-Time 3D Computer Graphics
343
D611ner, J., and Hinrichs, K., (2002) "A generic 3D rendering system", IEEE Transactions on Visualization and Computer Graphics, 8(2), 99-118. D611ner, J., and Kersting, O., (2000) "Dynamic 3D maps as interfaces to spatio-temporal data", Proceedings of ACM GIS 2000, 115-120. D611ner, J., Baumann, K., and Hinrichs, K., (2000) "Texturing techniques for terrain visualization", Proceedings IEEE Visualization, Salt Lake City, pp. 227-234. Haeberli, P., and Segal, M., (1993) "Texture mapping as a fundamental drawing primitive", Proceedings of 4th Eurographics Workshop on Rendering, pp. 259-266. Hasso-Plattner-Institute (Computer Graphics Systems), (2003). LandExplorer. Online: http://www.3dgeo.de (23/10/03). Hoppe, H., (1996) "Progressive meshes", Computer Graphics SIGGRAPH '96 Proceedings, pp. 99-108. Hoppe, H., (1998) "Smooth view-dependent level-of-detail control and its application to terrain rendering", Proceedings of IEEE Visualization, 35-42. Kersting, O., and D611ner, J., (2002a) "Interactive 3D visualization of vector-data in GIS", Proceedings ACM GIS 2002, Washington, pp. 107-112. Kersting, O., and D611ner, J., (2002b) "Interactively developing 3D graphics in Tcl", Proceedings USENIX Annual Technical Conference, Monterey, CA. Kilgard, M. J., (2000) "A practical and robust bump-mapping technique for today' s GPUs", In: Game Developers Conference, Advanced openGL Game Development, (July 2000). Online: http://www.nvidia.com MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Science, Special Issue on Geovisualization, 28(1), 3-12. Pajarola, R., (1998) "Large scale terrain visualization using the restricted quadtree triangulation", Proceedings of IEEE Visualization, pp. 19-26. Reeves, W. T., (1983) "Particle systems - a technique for modeling a class of fuzzy objects", Computer Graphics, 17(3), 359-376. Strauss, P., and Carey, R., (1992) "An object-oriented 3D graphics toolkit", Proceedings of SIGGRAPH '92, 26(2), 341-349. Szyperski, C., (1997) Component Software: Beyond Object-Oriented Programming. New York: ACM Press. Takatsuka, M., and Gahegan, M., (2002) "GeoVista Studio: a codeless visual programming environment for geoscientific data analysis and visualization", Computers and Geosciences, 28(10), 1131-1144. Woo, M., Neider, J., and Davis, T., (1999) OpenGL Programming Guide. The Official Guide to Learning OpenGL, Version 1.2, Addison Wesley.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 17
Interactive Approaches to Contouring and Isosurfacing for Geovisualization Adriano Lopes, Department of Informatics of the Faculty of Science and Technology/CITI, New University of Lisbon, Lisbon, Portugal Ken Brodlie, School of Computing, University of Leeds, Leeds, UK
Keywords: interactive, contour, isosurface, correctness, robustness, accuracy, Marching Cubes, bilinear/trilinear interpolation
Abstract This chapter describes ways in which contouring and isosurfacing techniques for 3D computer graphics have evolved. In particular, we show how they are strongly intertwined and how they have moved from being non-interactive to interactive processes, making them appropriate for the kinds of dynamic representation required for geovisualization. The analysis relies upon data defined on a regular, rectilinear grid, and a bilinear or trilinear data model within each 2D or 3D cell.
17.1
Introduction
Geospatial data is inherently structured into 2, 3 or 4 dimensions. For geovisualization, when this data is a scalar field, a very common approach is to show the set of points where this scalar field has a certain value, or threshold. For 2D data this is contouring, whilst for 3D data, this is isosurfacing. The two techniques are strongly related: both involve extracting geometry from data, by taking cross-sections through the space of the dependent variable. In the case of contouring, lines (isolines) of equal value are extracted from 2D data; in the case of isosurfacing, we extract surfaces (isosurfaces) of equal value from 3D data. Likewise, the development of techniques for generating isolines and isosurfaces are strongly intertwined - new approaches for one can lead to new approaches for the other. Traditionally, both contouring and isosurfacing have been seen as non-interactive processes in an interactive environment to support visualization in its broadest sense and geovisualization in particular (MacEachren and Kraak, 2001). In the early days of contouring, packages such as SYMAP (Fisher, 1963) would generate detailed maps that were plotted on paper for subsequent, off-line examination. Today, contouring remains a useful technique, but the style of working has changed and we need techniques that perform 345
346
A. Lopes & K. Brodlie
well in an interactive environment to support the process of geovisualization. We need to be able to change contour levels and see the effect immediately; we need to be able to zoom in and out; and we may want to keep the contour levels constant, but vary the data to form an animation. In each of these scenarios, we need the contouring technique to be fast, robust, and smooth with respect to changes of data and contour levels. For 3D data, in the early days of geovisualization, the most common approach was to slice through the dataspace, so that we reduce the visualization problem to 2D, and contouring can be applied. A sequence of slices can be generated, at intervals through the 3D space. However, as technology has developed over the years, it has become possible to treat the 3D data as a single entity, and use a 3D visual representation - in this case, isosurfacing to view a surface of common value. There are significant advantages to be gained from a 3D approach (see Wood et al., this volume (Chapter 14)). Isosurfacing naturally has a much shorter history than contouring, but in many ways a very similar one. In the early days, for isosurfacing this was the late 1980s, it was seen as a relatively slow operation, particularly for large data sets, and this parallels the early days of contouring. Today, advances in computing hardware allow the surface extraction to be achieved much more quickly, and advances in graphics hardware boards allow fast rendering of the extracted surfaces; see Wood et al., this volume (Chapter 14) and D611ner, this volume (Chapter 16). Thus isosurfacing has now become an interactive technique, and we can set out very similar requirements to those for contouring. In this chapter, we describe how methods for contouring and isosurfacing have evolved, and in order to show how strongly related the two topics are, our description will intertwine the two. We will show how improvements to the methods have been achieved as a result of a clearer definition of a model underlying the data, i.e., a model of the entity from which we suppose the data has been sampled. Ultimately, we show that for highquality visualization in an interactive environment, we need to base our approach on a visualization of that underlying model. Only then can we adjust thresholds, apply zooming or vary the data values, and maintain a consistent visualization.
17.2
The Evolution of Contouring and Isosurfacing
This chapter traces the evolution of contouring and isosurfacing in a number of stages. We assume throughout that the data is defined on a regular, rectilinear grid. We are interested in estimating the behaviour throughout the region, not just at the grid points, and we build a bilinear, or trilinear, model within each 2D or 3D cell. We make the fundamental assumption that this order of interpolation is an appropriate compromise between accuracy and speed. In w17.2, we describe the essential problem to be solved in generating contours or isosurfaces, and then sketch the outline of the remainder of the chapter.
17.2.1 Contouring Contouring aims to represent scalar data of constant value on a 2D plane, by lines joining those points of equal value (isolines or contour lines). Though an old technique it is still
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
347
very useful, as shown in many examples: isobars and isotherms in weather forecasting, isolines of digital elevation data (contours) in Cartography, etc. The data is defined point wise on a 2D rectangular grid and it is common practice to assume it is modelled as a bilinear interpolant on each cell of the grid (linear along the edges). Therefore, a contouring method should construct an approximation to the contours of the bilinear interpolant. Since the fundamental 2D drawing element is a line, we seek a piecewise linear approximation to the contours. A common strategy is to scan and examine each cell in sequence, independently. When there are grid values higher and lower at the comers of the cell then part of the contour lies within the cell. Formally, the problem is to compute and display isolines of a threshold value giving the values of a bilinear function F 2D at the vertices of the rectangular cell. For convenience and without loss of generality the cell domain is transformed into a unit square [0, 1] x [0, 1] and the threshold value is zero. F 2D (x, y) -- ax + by + cxy + d,
(17.1)
with a - Flo - Foo, b - Fol - Foo, c -- Foo + F l l - FOl - Flo and d - Foo, where Foo , Fol, Flo and Fll are function values at the vertices of the cell. Assuming c ~ 0 the contour curve within the cell is a hyperbola (with c - 0 it is a line).
17.2.2
Isosurfacing
Isosurfacing aims to represent a surface of a given threshold value drawn in the interior of volumetric scalar data. It is the natural extension of contouring; the boundary surface separates points with values greater or equal to threshold from those with values less. Data is defined pointwise on a 3D rectangular grid and the model within each cell is the trilinear interpolant - linear along edges, bilinear across the faces of the cell. As in contouring, a common strategy is to scan and examine each cell in sequence, independently. If grid values spanning the threshold are found, then part of the isosurface is within the cell. Without loss of generality the cell domain is assumed to be a unit cube with corner at origin, and the threshold value is considered as zero. The interpolant F 3D can be written in terms of the vertex values F~ik, i,j, k = 0, 1 or in a equivalent way: F 3D(x, y, z) - a + ex + cy at- bz + gxy + f x z - dyz + hxyz
(17.2)
with a = F000, b - F 0 0 1 - F000, c : F 0 1 0 - F 0 0 0 and so on. Any representation should provide an approximation to the surface of the trilinear interpolant. Since the triangle is the basic drawing element in 3D, we seek a piecewise linear approximation forming a triangulation in 3D.
17.2.3
From passive to interactive
We trace the evolution !n a series of four steps - with each step we improve the representation of the underlying bilinear, or trilinear, model. By the final step, we have a representation that is suitable for interactive work where we wish to explore rather than just present.
348
A. Lopes & K. Brodlie
In the first step, we use the fact that the model is linear along the edges in order to accurately determine the intersections of the interpolant with the grid lines. However, in this first evolutionary step, no attempt was made to properly follow the curved nature of the interpolant within a cell, because this is computationally expensive. Instead, an ad hoc strategy was typically used in order to connect the intersection points with lines in the case of contouring, or triangular pieces in the case of isosurfacing. This was the normal practice in the early days of contouring (1960s) and isosurfacing (late 1980s). As explained in w17.3, it was almost immediately realized that there are cases where a number of alternative ways of joining up the points are possible - the so-called ambiguous cases. An ad hoc strategy here is fraught with danger. Hence, in the second evolutionary step, described in w17.4, the solution was to decompose the cell into triangular, or tetrahedral, parts. The advantage is that the contours are now exactly straight lines, the isosurface exactly flat planes, and so very fast to draw. This represents a major improvement, but the model being piecewise linear within the grid cell, is not continuous in first derivative across the cell interior. Thus in a third evolutionary step, described in w the strategy returns to consideration of the bilinear, or trilinear, interpolant. An attempt is made to approximate the interior topology of the interpolant. This turns out to be non-trivial particularly for the isosurface case, but we reach a stage where the correct topology is generated in a piecewise linear representation (lines or triangles). Notice the contrast between the approaches: in the second evolutionary step, the cell is decomposed into pieces and we fit linear models in each piece (with exactly linear isolines or isosurfaces); in the third step, we fit a bilinear, or trilinear, model in the cell (with nonlinear isolines or isosurfaces) and construct a piecewise linear approximation to the isolines/isosurfaces. In the final step, described in w attention turns to improving the representation of the previous step. We not only want the correct topology but also want the representation to work well in interactive exploration of the data.
17.3
Correctness of Boundary Points
17.3.1
Contouring
Consider the case of a function defined at the corners of a unit square as described in w17.2.1, where we are interested in drawing the zero contour and where we assume a bilinear interpolant as in equation (17.1). The intersection points of the hyperbolae with the cell edges are easily and accurately calculated by inverse linear interpolation. The basic method is then to approximate the hyperbolae by straight lines connecting those intersection points. There are 16 different vertex cases depending on whether the values at the corners are positive or negative. The 16 cases can be reduced to just four canonical cases as shown in Figure 17.1: no contour at all; a single segment cutting off a corner; a segment
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
1
2
349
3
Figure 17.1. Canonical configurations for rectangular cells when data varies linearly between adjacent grid points. Positive vertices are marked.
cutting off two corners; or two segments cutting off opposite corners. However, this final case requires an arbitrary decision to be taken as regards which corners are cut off: it occurs when one pair of opposite vertices are of different sign from the other pair. This is discussed as a difficulty in many early papers on contouring (Sutcliffe, 1980). Ad hoc strategies, such as always "keeping high ground on the right" emerged to provide some consistency, if not correctness. So the main features in this basic method are: 9 9 17.3.2
correctness of boundary points so there is consistency from one cell to the next; no guarantee of topological correctness. Isosurfacing
The classical and simple approach to isosurfacing is the Marching Cubes (MC) algorithm, proposed by Lorensen and Cline (1987), with a similar suggestion from Wyvill et al. (1986). Each vertex of a cube can be either greater than or less than the threshold value, giving 256 different configurations. The intersections of the isosurface with the edges of the cube are easily and accurately calculated by inverse linear interpolation. Then the set of intersection points are triangulated to yield an approximation to the isosurface within the cube. Lorensen and Cline argued that for reasons of symmetry and complementarity there are only 15 canonical configurations, and proposed corresponding triangulations of the isosurface. For a given configuration (from the set of 256), they provide a look up table to give the corresponding canonical configuration and hence its triangulation. But a further case can be removed by reflectional symmetry, leaving a canonical set of 14 cases. These are shown in Figure 17.2. Unfortunately, the efficient code of the small look up table caused inconsistent matching of surfaces between adjacent cells, so that "holes" could appear as first reported by Durst (1988), within a short time of the publication ofLorensen and Cline' s paper. The behaviour of the interpolant F 3D inside the cube is non-trivial and is a cubic surface. The difficulties identified by Durst were exactly the problems identified by the early contouring pioneers, when using ad hoc strategies to represent the contours of a bilinear interpolant simply by joining up intersection points. As we see later the problem is deeper in 3D since we need to understand the correct behaviour both on the faces, and in the interior.
350
A. Lopes & K. Brodlie
0
1
2
3
4
5
6
7
8
9
10
11
12
13
Figure 17.2. Basic cases in the Marching Cubes algorithm, and examples of how the points of intersection between the isosurface and the edges of the cube can be connected. Positive vertices are marked.
The main features of this basic MC algorithm are: 9 9
9
17.4
correctness along the edges of the cube; potential flaws as a consequence of mismatched patterns between adjacent cells. Therefore, no topological correctness on the boundary faces, and so no consistency of solution from cell to cell; topologically is incorrect in the cell.
Grid Decomposition
In w17.4, we see how decomposition of the basic rectangular shaped cell into triangular (or tetrahedral) shaped pieces addresses some of the problems discussed in w17.3.
17.4.1 Contouring As an attempt to solve the ambiguity in the basic contouring method, several authors proposed decomposing the basic square cell into four triangles (Dayhoff, 1963; Heap, 1972). The value at the centre of the cell is taken as the average of the four corner points. Then two extra intersection points lying on the diagonals of the cell are obtained by using inverse linear interpolation (Figure 17.3). Notice that the averaging process at the centre corresponds to bilinear interpolation, but is then followed by a linear interpolation within each triangle.
Interactive Approaches to Contouring and Isosurfacing for Geovisualization + 10
,m=
w~
-4
+10
-7
+5
-7.5
+5
351
s S
w
IP'
-7
Figure 17.3. Four-triangles method, with two extra points lying on the diagonals of the basic square cell (Lopes, 1999). The solution on the left is topologically correct, but that on the right is topologically incorrect. It is only correct when both the centre point C and the saddle point S have the same sign.
The main features of this method are: 9 9
9
correctness of boundary points so consistency of solution from cell to cell; no guarantee of correct topology as Figure 17.3 shows. It is only correct when both the saddle point and the centre lie in the same region of the hyperbolae. In other words, when both have the function value with the same sign; discontinuity of derivative of the contour as the diagonal is crossed.
17.4.2 Isosurfacing Similar to the four-triangles method for contouring, the Marching Tetrahedra (MT) method (Doi and Koide, 1991) emerged largely as the result of trying to solve the problem of ambiguity in the MC algorithm. In this case, the cube is decomposed into tetrahedra and then the MT method is applied to each tetrahedron. Any additional vertex values that are needed can be obtained by trilinear interpolation. The advantage of this decomposition is similar to the four-triangle contouring method: a linear interpolant is fitted in each tetrahedron, based upon the four data values at the vertices" FT ( x , y , z ) -- a 4- bx 4- cy 4- dz
(17.3)
The exact isosurface within a tetrahedron is a plane (one or two triangles). The main features are as follows: it is simple to triangulate the surface due to the linear interpolant; it is unclear how best to decompose the cube into tetrahedra; it does not always provide topological correctness within the cube (when compared with the isosurface of the trilinear interpolant); there is discontinuity of the model across the faces of the tetrahedra.
17.5
Topological Correctness
We discuss the next stage in w Rather than decomposing the cell, we aim to work with the bilinear, or trilinear, interpolant within the whole cell, and try to
352
A. Lopes & K. Brodlie
understand its behaviour so we can represent it correctly from a topological viewpoint.
17.5.1
Contouring
To the best of our knowledge, this stage emerged first as an improvement to isosurfacing, and by implication, becomes an improvement to contouring. It was developed by Nielson and Hamann (1991), who were trying to solve ambiguity in MC, looked at the behaviour of the bilinear interpolant on the cell faces - this is exactly a contouring problem (the intersection of an isosurface with a cell face is a contour line). Nielson and Hamann (1991) looked at the asymptotes of the hyperbolic arcs and the behaviour at their intersection, the saddle point. When one pair of opposite vertices have data values of different sign from the other pair, the pairwise connection is established so as to "cut off" the vertices of opposite sign to the saddle point value. Figure 17.4 depicts this method the previous basic method with asymptotic decider to resolve the ambiguity. However, the decision on which connection to make "flips" as the contour level moves through the saddle value. This makes it unsatisfactory for an interactive application where we are smoothly changing the contour levels. The main features are: 9 9 9
17.5.2
correctness of the boundary points so consistency of solution from cell to cell; correctness of topology; discontinuity of visual representation with respect to changes in contour level (or changes in data).
Isosurfacing
As mentioned earlier, the classical MC algorithm has a naive approach to approximating the interpolant F 3D. Not only can holes between cells appear when two adjacent cells have certain configurations but also the interior of the cell is not represented in a topologically correct way. +10
-4
S O
+ 0.847
-
-7
ii IF
+5
Figure 17.4. The asymptotic decider to solve ambiguity. F 2D of equation (17.1) in the region between the two contour sections is positive, which includes the saddle point S - the intersection of the two asymptotes Fx2D = 0 and F 2D - - 0, i.e., the point (-b/c,-a/c).
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
353
The asymptotic decider described earlier for contouring is able to give positional continuity and correct topology on the faces of the cell. Of the 14 cases in Figure 17.2, six involve at least one ambiguous face. The problem now is to deal with the exact shape of the interior surface. Natarajan (1994) finds that further ambiguities can occur. For example, consider a cube where one pair of opposite vertices are positive, while the remaining six vertices are negative (this is configuration 4 in Figure 17.2). A simple triangulation will cut off the positive vertices with single triangular pieces, but these approximate a curved surface that bows out towards the centre of the cell. Imagine the data values all increasing uniformly so the zero isosurface pieces move towards each other - the simple-minded triangular approximation remains separated as two pieces, but the true trilinear surface pieces will come into contact with each other, and as that happens they merge into a single surface with a tunnel. This is an internal ambiguity, rather than the face ambiguity treated by the asymptotic decider. This is illustrated in Figure 17.5 where we see the two situations (a) and (b) that can occur internally for a single vertex configuration. Natarajan (1994) shows that a key to identifying tunnels is the value at the body saddle point, which is the 3D equivalent of the saddle point whose value was exploited by Nielson and Hamann (1991) to determine face ambiguities in the asymptotic decider. The body saddle is located where the transition from one to two pieces occurs. Figure 17.5a shows the situation with two pieces whereas in Figure 17.5b the pieces have merged with a tunnel appearing. A treatment of topological correctness in isosurfacing on rectilinear grids is presented by Chernyaev (1995). He identifies some 33 canonical configurations, covering both face and interior ambiguities. More recently, Lopes and Brodlie (2003) indicated that two cases can be removed as they were not canonical. Figure 17.6 shows the 31 cases. The first number denotes the original MC case number as in Figure 17.2; the second indicates the resolution of a face ambiguity; and the third the resolution of an interior ambiguity. Other contributions to solve internal ambiguities in the MC algorithm are worthy of mention. In particular, Cignoni et al. (2000) extended the 256 cases to some 798 different cases, but only 88 of these were distinct configurations. They proposed an "extended look
Figure 17.5. The two possible topologies (a) and (b) in MC configuration 4 (see Figure 17.2). The value of the body saddle point can discriminate the two situations.
354
A. Lopes & K. Brodlie
Figure 17.6. The 31 cases in the Marching Cubes algorithm. The first label number denotes the original MC case number as in Figure 17.2; the second indicates the resolution of a face ambiguity (six ambiguous faces in the group of case 13, three in the 7, two in the 10 and 12, one in the 3 and 6, none in the others); and the third the resolution of an interior ambiguity.
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
355
up table" (ELUT), which aims to provide explicit triangulations for a large number of cases based on face saddle, and body saddle, values. Also, Matveyev (1994) has considered the behaviour of the interpolant along the cell diagonals in order to solve ambiguities. Many of these efforts attempt to provide explicit triangulations of the surface. However, it is important to note that many fail to do this correctly, because they propose triangulations that involve triangles in the face of a cell, which should not be allowed, van Gelder and Wilhelms (1994) identified this important principle: in order to obtain a continuous triangulated surface (C0) each polygon edge must belong to no more than two triangles. This forbids any triangles from lying in the face of a cell, since these will have an edge belonging to two triangles from that cell, and to at least one triangle from the adjoining cell. Generally, the choice of additional interior points needed to complete the triangulation is made in an ad hoc manner; in particular, they are typically not chosen so as to lie on the exact surface. As with the asymptotic decider for contouring, these methods take decisions on how to represent the isosurface that are not continuous with respect to the threshold value. This makes them unsuitable for interactive work. In conclusion, the main features of the MC algorithm with asymptotic decider and body saddle point test are: 9 9 9 9
17.6
correctness of the boundary points so consistency of solution from cell to cell; correctness of topology both on the boundary and in the interior of the cell; the triangulation methodology is not very clear as additional points in the interior are sometimes needed; a discontinuity of visual representation with respect to changes in contour level (or changes in data).
Interactive Working
Cartographic maps are no longer seen as static entities, but rather as dynamic interfaces, responding to interaction by an investigator. This engagement between map and investigator is key to improving the thinking process, as geospatial data is explored (MacEachren and Kraak, 2001). The challenge for the geovisualization community is to provide techniques that are fast enough to be responsive (to allow this interaction), yet are of high integrity in the representation of the data so that correct inferences are drawn. The methods introduced earlier indicate that we are now able to achieve topological correctness, with respect to the exact isolines of a bilinear, or exact isosurface of a trilinear. However, there remains a quality problem, in that the visual representation is discontinuous as the data changes. This is exactly the situation we would have in an interactive setting when the investigator alters the threshold value. Thus, we need to further develop methods that are robust when used in a setting where any of the following operations are carried out: 9 9 9
the contour levels are altered, or the isosurface threshold is modified; the data values are continuously changed; the viewpoint is continuously changed, for example to zoom in on the contours or isosurface.
356
A. Lopes & K. Brodlie
The solution is to take more care in the representation of the isoline, or isosurface, in the interior of the cell. At the same time, we cannot afford the solution to be inefficient, because speed is essential for effective interaction. Thus, we pay attention to the way contour lines are approximated as polylines, and isosurfaces as triangular meshes. 17.6.1
Contouring
For contouring, Lopes and Brodlie (1998) have modified the basic method with asymptotic decider by adding an extra point for each pair of intersection points, the socalled shoulder points. Each contour section is now drawn as a polyline from one intersection point to the shoulder point and then to the other intersection point. This gives a two-piece linear approximation to the conic arc. The shoulder point is the point on the conic parallel to the chord joining the endpoints. It is an optimal point to choose in forming this approximation, as it is the furthest point on the arc from the chord. This is shown in Figure 17.7: P and Q are the end-points of the hyperbolic arc, and R is the shoulder point of the arc. R is quite easy to calculate as it lies on the line joining M, the mid-point of the chord PQ, and S, the saddle point of the bilinear interpolant. The significant advantage of this simple extension is that the visual representation now moves smoothly as the contour level moves smoothly through the saddle value, compared with the visual "jump" that occurs in the unmodified method. Figure 17.8 depicts this property. Moreover, the process of adding an extra point can be applied recursively as we zoom in on a contour line. Thus, we have an approach that is robust enough to be used in an interactive setting.
17.6.2 Isosurfacing This work has been extended from contouring to isosurfacing (Lopes and Brodlie, 2003; Lopes, 1999) with the objective of achieving a high quality internal representation by adding a minimal number of carefully chosen extra points, so that a minimal number of extra triangles are created. First, as in the classical approach, the intersection points of the isosurface with cell edges are calculated to form an initial polygonal outline of
Figure 17.7. Shoulder point R as an extra point on the hyperbolic arc. It lies on the line MS as well as on the line parallel to PQ.
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
__J y #
357
,Ih ,qr
//]
.
Figure 17.8. Shoulder points (right) help to move smoothly from top to bottom row in comparison to the basic method with asymptotic decider (middle) described in w
the isosurface. The edges of the polygon lie on the cell faces, and are approximations to the isocontour lines on the faces. This polygon is then extended by adding shoulder points, exactly as in the contouring. This improves the accuracy on the faces. In the interior there are two special classes of points, which help to define the internal behaviour of the interpolant. One class is called bi-shoulder points, which are 3D analogues of shoulder points; the other is called inflection points, which are generalizations of the body saddle point. With these points it is possible to generate efficient triangulations that correctly represent the interior topology, as well as increasing the accuracy. Figure 17.9 shows examples depicting how these classes of points help to nicely delineate the shape of the surface. As with the contouring extension, we get a smooth transition of the visual representation with changes to the threshold level and to the data. In conclusion, the main features of this method are: 9 9
9
The solution is topologically correct and accurate. It is robust in the sense of being continuous with respect to changes in the data or isosurface level. As the threshold changes so that a different choice is made to resolve an ambiguity, so the representation does not display a visual discontinuity - the transition between the different states in the ambiguous cases is smooth. While achieving correctness and robustness, care has been taken to add as few additional points to the triangulation as possible (compared with the standard MC algorithm). The cost of computation of this superior representation is of
358
A. Lopes & K. Brodlie
17.9. With the help of special classes of points - bi-shoulder points and/or inflection points it is possible to generate efficient triangulations (right) to nicely delineate the shape of the surface (left) (Lopes, 1999). Figure
-
course higher. But the major extra cost is indeed the increased number of triangles as computation of those points is quite simple. The extra points that are added, are chosen to lie on the isosurface of the trilinear interpolant, and to be optimal in terms of accuracy of polygonal representation. A straightforward set of triangulations is provided in contrast to look-up tables. The classification is ultimately based on the type of boundary polygon, i.e., the number of intersection points, number of faces intersected twice, and internal points. The four cases with no faces intersected twice have two sub-cases: either with or without an interior hole. This gives 14 canonical cases in total. Figure 17.10 shows a set of isosurfaces distributed over a 3D grid of volatile organic compounds. An isosurface is by definition only a subset of the data, and it is a natural part of the visualization process to modify thresholds in order to sweep out areas of interest in the data (Shneiderman, 1996).
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
359
Figure 17.10. Set of isosurfaces depicting concentrations of volatile organic compounds in a 3D grid of size 60 x 60 x 50. The data was kindly supplied by City University, using EarthVision(R) software by Dynamic Graphics, Inc., Alameda, California, USA. The visualization was created by ourselves, using the IRIS Explorer visualization system from NAG Ltd and in particular a module developed by one of the authors (Lopes) and now available through the IRIS Explorer Centre of Excellence (http://www.comp.leeds.ac. uk/iecoe).
Figure 17.11. Interactive environment to depict concentrations of volatile organic compounds using the IRIS Explorer visualization system.
360
A. Lopes & K. Brodlie
Figure 17.11 shows the sort of interactivity that is possible. In the interface panel shown, the user can alter the isosurface threshold by means of the dial. In a dataflow visualization system such as IRIS Explorer (as shown in the figure), a change of threshold triggers a recalculation of the isosurface and the resulting geometry is passed automatically to the rendering module for display. Thus, the picture changes immediately to reflect the dial change. This dynamic representation adds significant value to the geovisualization process. Notice that any algorithm that was not robust to threshold changes would be quite unsatisfactory in such an interactive environment.
17.7
Conclusions
This chapter has traced the history of contouring and isosurfacing techniques, showing how their evolution has been intertwined over the years. The evolution has taken us from initial approaches which aimed to satisfy a passive style of computing, where the graphic depiction is used as a presentation medium - to the modern era where interactivity allows the opportunity to explore the data in order to gain deeper insight through visualization. Our final stage of the evolution has yielded an approach that offers efficiency, through economizing on either straight lines for contours or triangular pieces for isosurfacing, the latter mapping well on to fast triangle rendering hardware such as PC graphics boards supporting OpenGL. More than efficiency, the approach offers a visual continuity as the data is explored through variations in threshold level.
Acknowledgements Adriano Lopes is grateful to the Department of Informatics of the Faculty of Science and Technology, New University of Lisbon, Portugal, and its associated research centre CITI, for their travel support. Figures 17.10 and 17.11 were produced in collaboration with City University using the EarthVision (R) software and data from Dynamic Graphics.
References Chernyaev, E., (1995) Marching cubes 33: construction of topologically correct isosurfaces, (CN/95-17). CERN. Cignoni, P., Ganovelli, F., Montani, C., and Scopigno, R., (2000) "Reconstruction of topologically correct and adaptive trilinear surfaces", Computers and Graphics, 24(3), 399-418. Dayhoff, M., (1963) "A contour-map program for X-ray crystallography", Communications of the ACM, 6(10), 620-622. Doi, A., and Koide, A., (1991) "An efficient method for triangulating equi-valued surfaces by using tetrahedral cells", IEICE Transactions on Communication Electronic Information Systems, E-74(1), 214-224. Durst, M., (1988) "Letters: additional reference to 'Marching Cubes'", Computer Graphics, 22(2), 72-73.
Interactive Approaches to Contouring and Isosurfacing for Geovisualization
361
Fisher, H., (1963) SYMAP. USA: University of Harvard. Heap, B., (1972) Algorithms for the Production of Contour Maps, Report NAC-10, National Physics Laboratory, Teddington, UK. Lopes, A., and Brodlie, K., (1998) Accuracy in contour drawing, Eurographics UK 98, Leeds, UK, pp. 301-311. Lopes, A., and Brodlie, K., (2003) "Improving the robustness and accuracy of the marching cubes algorithm for isosurfacing", IEEE Transactions on Visualization and Computer Graphics, 9(1), 16-29. Lorensen, W., and Cline, H., (1987) "Marching Cubes: a high resolution 3D surface reconstruction algorithm", Computer Graphics, 21 (4), 163-169. MacEachren, A. M., and Kraak, M. J., (2001) "Research Challenges in geovisualization, cartography and geographic information science", Special Issue on Geovisualization, 28(1), 3-12. Matveyev, S., (1994) "Approximation of isosurface in the marching cube: ambiguity problem", In: Bergeron, R. D., and Kaufman, A., (eds.), IEEE Visualization 94. Washington, DC: IEEE Computer Society Press, pp. 288-292. Natarajan, B., (1994) "On generating topologically consistent isosurfaces from uniform samples", The Visual Computer, 11, 52-62. Nielson, G., and Hamann, B., (1991) "The asymptotic decider: resolving the ambiguity in Marching Cubes", In: Nielson, G., and Rosemblum, L., (eds.), IEEE Visualization 91. San Diego, CA: IEEE Computer Society Press, pp. 83-90. Shneiderman, B. (1996), "The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations", In Proceedings IEEE Symposium on Visual Languages, Washington: IEEE Computer Society Press, pp. 336-343. Sutcliffe, D., (1980) "Contouring over rectangular grids and skewed rectangular grids", In: Brodlie, K., (ed.), Mathematical Methods in Computer Graphics and Design. London: Academic Press, pp. 39-62. Wyvill, G., McPheeters, C., and Wyvill, B., (1986) "Data structure for soft objects", The Visual Computer, 2, 227-234. van Gelder, A., and Wilhelms, J., (1994) "Topological considerations in isosurface generation", ACM Transactions on Graphics, 13(4), 337-375. Lopes, A., (1999). Accuracy in Scientific Visualization, School of Computer Studies, University of Leeds, p. 168.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 18
Applying "True 3D" Techniques to Geovisualization: An Empirical Study Sabine Kirschenbauer, Institute for Cartography, Dresden University of Technology, Dresden, Germany
Keywords: autostereoscopic display, true 3D, depth perception, user analysis, experience, cognitive style Abstract "True 3D" techniques have a fairly long history but have been comprehensively adapted to geovisualization only recently. True 3D describes techniques such as stereoscopic displays, anaglyphs, immersive workbenches and holograms, in which the visualized third dimension appears either behind or in front of the display plane. Inspired by developments in technology, the variety of true 3D representations means that this has become an increasingly important research topic. Yet, this particular domain is lacking in fundamental theory. This chapter gives an overview of true 3D techniques applied to geovisualization and reviews relevant issues such as 3D perception. An empirical study was conducted, investigating the impact on different users of the true third dimension realized by an autostereoscopic display. The experiment may be used as an example of how a structured investigation can offer deeper insights into the map-reading process in order to expose beneficial properties of a true 3D technique and its stereo-image, respectively. The results are discussed as they relate to the usefulness of such true 3D visualizations. Three characteristics are found to be particularly relevant to successful map reading when using true 3D displays of spatial information. These include a human user's visual capacities, their level of experience and the purpose of the visualization.
18.1
Introduction
This chapter introduces "true 3D" techniques for geovisualization. Different factors involved in the true 3D map-reading process are highlighted in order to demonstrate the unique nature of the technique. Within this framework, an empirical study on a true 3D visualization technique is offered: the impact of the true third dimension on the map reader's operation is analysed, whilst the behaviour of various users when working with different maps is also explored. 363
364
S. Kirschenbauer
True 3D map reading is being addressed holistically, particularly with regard to the potential for true 3D to be used as an efficient tool for geovisualization. A theoretical framework in which to evaluate the utility of true 3D representations provides a means of guiding decisions in order to ensure the useful application of the technique.
18.2
The Definition of True 3D
The term 3D is used with various meanings in a range of disciplines both within and beyond the realm of geovisualization; (see Wood et al., this volume (Chapter 14)). Therefore, the term true 3D as used in this framework needs clarification at the outset. In the abstract domain of data and visualization, three-dimensionality is determined in the Cartesian coordinate system using x, y and z values. However, the human perception of 3D stimuli is dominated by a coordinate system that emanates from an egocentric space. Here, the x and y values define a visual field complemented by the z dimension which is responsible for depth; (see Ware and Plumlee, this volume (Chapter 29)). The different way in which we think of space is the most obvious reason for the difficulties of defining "3D" in a universally valid manner. Five elements of dimensionality can be distinguished that are relevant to stages of the visualization process. These include dimensionality of: raw data, assembled data, the visual representation, the display medium and the schemata; (see Wood et al., this volume (Chapter 14)). The dimensionality of the schemata, the internal constructions made to organize and interpret visual stimuli, will depend upon the percept. The percept is characterized as the "visual image" that the observer perceives directly. Based upon the abstract distinction between perception and cognition, the image is not cognitively processed in this state. Definitions that include all of these areas are scarce as data are often processed differently in each stage. Much more common are approaches that refer only to one of these stages or differentiate between each of the areas. A comprehensive approach is offered by Terribilini (2001), who embraces the different stages of the visualization process. Kraak (1988) identified a map as 3D "when it contains stimuli which make the map user perceive its contents as 3D." Drawing on an approach reported by Jensen (1978), who defined "3D" only on the basis of perceptual characteristics of the map, Kraak likewise refers to the dimensionality of percept with his own definition. Thus, 3D maps include any kinds of stereograms, holograms, etc. This definition has served as a basis, but requires refinement to differentiate between true 3D and 3D: 9
9
A map or an image is termed true 3D if its generated percept is figurative: when perceiving the visualized third dimension, either it lays behind and/or in front of the display plane which can be paper (or other media such as glass, foil, etc.) or screen. The spatial, figurative effect emerges with any physiological cues and the psychological depth cues in combination (see w Examples are anaglyphs, immersive workbenches, (auto) stereoscopic displays, holograms, etc. Whereas 3D characterizes any maps or images that create a depth effect, the true 3D effect occurs only on the basis of psychological depth cues or exceptionally
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
365
due to the physiological cue "motion" through motion parallax, for instance (hill-) shading, panoramic views, etc. The definition of true 3D, as it is meant here, refers to the dimensionality of the percept. The peculiarity of this percept is the compelling active linkage between the percept, the displaying medium and the observer, by whom this percept will be created. The definition is independent of the stages of data, representation and of the display medium. The reason for doing so, beside the fact that it might sometimes be a mechanical habit, lays in the goal of the geovisualizer: effective depiction of our 3D environment as most kinds of geovisualization deal with 3D phenomena. Different techniques support different forms of 3D perception in the sense of their effectiveness and the efforts required to interpret the geographic information. In the case of true 3D, the percept plays the decisive role (in the majority of cases): in a so-called "2D-map" such as those containing contour lines to symbolize elevation, the third dimension is represented by numeric specifications. By adding the psychological depth cue "shading" the observer might perceive depth. The map can be called 3D. By means of a technique using the cue "binocular disparity", the same map can be realized as a true 3D map. The data behind each of these percepts might be of the same nature or quality, but the results differ significantly. The result is a direct conjunction between the technique and the human user (as mentioned earlier). Thus, it appears that the output is one of the central links in the chain of the visualization process which is reflected in the definition.
18.3
How We Perceive True 3D
"Why do things look as they do?", asked Koffka (1935). Several characteristics must be considered to answer this question in the context of 3D perception in particular. The answer involves depth cues, spatial sense, experience and practice. Our environment offers us many valuable clues for 3D perception. Images take advantage of mechanisms of human vision developed to interpret a 3D reality in order to communicate information about spatial depth: depth cues such as overlapping, perspectives or texture gradients are the sources for perceiving three-dimensionality. However, both quality and accuracy of 3D perception depend strongly upon the spatial sense of the individual, especially in virtual environments and images. This is also influenced by practice to a degree. Experience of depth perception is obtained by humans throughout our lives. The nature of our experience in organizing visual depth in reality becomes evident when we close one eye. Though we need two eyes for stereo viewing (i.e., 3D perception) we can still interpret depth relatively well with one eye closed. The brain stores sensory experience over a long period and this can be used advantageously when we lose our stereo viewing ability. Some of the cues are learned and therefore assisted by experience: the psychological ones for example (Kohler, 1951; Julesz, 1971).
366
S. Kirschenbauer
In m o d e m psychology, depth perception is based on ten cues (Okoshi, 1976). Four are physiological and 6 psychological: Physiological Cues: 9 9 9 9
Accommodation - the adjustment of the focal length of Convergence - the angle made by the two viewing axes Binocular disparity - the disparity between images of the onto the two retinas. Motion parallax - the results of changing positions of an either the motion of the object or the viewer's head.
the lens. of the eyes. same object projected object in space due to
Psychological Cues: 9 9 9 9 9 9
Retinal image size - the larger the image of an object the closer it appears. Linear perspective - the gradual reduction of image size with increasing distance of an object. Areal perspective - the "haziness" of distant objects. Overlap - the effect that continuous outlines appear closer to the observer than interrupted ones. Shadows and shading - these cause the impression of convexity or concavity based on the fact that most illumination tends to be from above. Texture gradient - a kind of linear perspective describing levels of "roughness" of a uniform material as it recedes into the distance (Okoshi, 1976; Buchroithner and Kirschenbauer, 1998).
When combined, these cues greatly enhance depth perception. Most true 3D geovisualization applications rely on the binocular cue binocular disparity in combination with psychological cues: true 3D topographic representations that use binocular disparity and the monocular cue shadows and shading are particularly popular. True 3D panoramic views often apply linear perspective and may also take advantage of areal perspective, both in combination with binocular disparity. More details on these cues can be found in Okoshi (1976) and Albertz (1997). For a review in relation to Information Visualization see Ware (2000) and for an assessment of the cartographic context see Kraak (1988) and MacEachren (1995). In principle, binocular parallax is the foundation for depth perception (excluding the case of long-distance viewing where we benefit from our experience and knowledge and, depending on the distance, we may also use the psychological cues that the environment offers - see Ware, 2000). Our eyes perceive the environment from slightly different angles: focussing on a point x in the environment, the eyes converge as their viewing axis hits this point. Neighbouring points (y) are projected onto positions of the retina that lie at a certain distance from the viewing centre. These distances differ in both eyes. The difference of these distances is called binocular disparity. When spatial depth increases binocular disparity increases accordingly (Poggio, 1987). The effect is illustrated in Figure 18.1.
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
Figure
18.1.
Binocular Parallax
-
367
the basic principle of depth perception.
Stereoscopic viewing of images, i.e., "shape-from-stereo" is achieved in a similar manner. It is fundamental for most of the true 3D techniques in general and in particular for the medium used for the empirical study, an autostereoscopic display: Shape-from-stereo is attained by taking advantage of the binocular parallax. Two images of one object are spatially separated and presented to a viewer' s right and left eye at the same time. The pair of images displays the object from a slightly different point of view according to geometry of the eyes. The viewer visually fuses the stereo pair into one spatial image and an impression of depth results (Rock, 1985). Most of the true 3D techniques are based on this technique for stereoscopic viewing, but a number of different implementations can be achieved. Examples to illustrate this principle will be given in w18.4. The shutter technique also uses this principle, but takes advantage of saccadic eye movement. When fixing a point, the human eyes move constantly. These movements are made up of short, jerky jumps, which are called saccades. Instead of presenting the two images concurrently, they are displayed in rapid succession that is not discernable to the user. 18.4
True 3D Techniques
True 3D techniques can be considered to be emerging in terms of their use in geovisualization applications. This has been the case for almost a decade. Though a great number of these techniques are available, only a few are currently used in a geovisualization context. The comparatively high technical and economic expenditures required to use the techniques might be a reason for this situation. Furthermore, the benefit of true 3D representations in comparison to other 3D techniques using "only" monocular cues is often doubted (Ware, 2000). This has a significant impact upon uptake when the economic impediments are considered. However, theoretical debates regarding the application of true 3D techniques suggest that they are worth careful consideration and empirical research attention (Kraak, 1988, 1993; Ellis, 1991; Toutin, 1997; MacEachren et al., 1999a-c; Ware, 2000).
368
S. Kirschenbauer
True 3D techniques can be categorized according to various characteristics, such as the methods used or the quantity of information that they require (e.g., Okoshi' s (1976) classification of 3D techniques). Schenkel (2000) gives a detailed grouping in terms of their technical potential describing their technical feasibility and suitability for geovisualization respecting viewing condition, resolution, etc. Anaglyphs, chromostereoscopy and the Dresden 3D LC Display for autostereoscopic displays are described in w18.4 to illustrate possible approaches to true 3D that are applicable to contemporary geovisualization. These techniques are a selection that exemplifies the basic methods of various true 3D techniques: anaglyph is a classical stereoscopic technique that is based upon binocular disparity. Chromostereoscopy represents an alternative technique that does not rely on binocular disparity, but on chromatic aberration. The autostereoscopic display constitutes a modern digital true 3D technique that adapts the original principle of binocular disparity and the involved image separation to novel technologies. Other true 3D techniques including holography and lenticular lenses are addressed in relevant literature (MacEachren et al., 1999a-c; Fuhrmann and MacEachren, 1999; Okoshi, 1976; Hariharan, 1996; Buchroithner, 2000; Frank, 2002). The examples that are introduced below do not consider the quality of the techniques in terms of perception and resolution. However, these are important issues from the observer's perspective, and are highly dependent upon the data to be visualized using a true 3D technique, the nature of the task in hand, and the characteristics of the user. An observer's assessment of any technique is in addition likely to be heavily dependent upon the context in which the viewing takes place.
18.4.1
Anaglyphs
In order to achieve a stereo-effect the anaglyph method depicts a 3D scene from two viewpoints at once using complementary colours. A stereo pair of viewpoints on the same scene is overplotted in complementary colours such as red and green or sometimes red and blue. By means of coloured glasses that contain one lens coloured in each of these hues, each eye receives a single stereo-image corresponding to the complementary colour. The observer's visual system and brain merges these two images into a spatial black-and-white image (Falk et al., 1986). Contemporary anaglyphs are often used to depict topography and are found either in printed form (Imhof, 1965; Ambroziak and Ambroziak, 1999) or as digital representations available on the World Wide Web (North Dakota State University, 2002).
18.4.2
Chromostereoscopy
This emerging approach is based on chromatic dispersion. The fact that electromagnetic radiation with a shorter wavelength is refracted more significantly than radiation with longer wavelength in the human eye, red hues appear to be closer to us than blue. This can be used to the advantage of geovisualization if the different planes of depth of an image are encoded using colours that are ordered according
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
369
to wavelength (Steenblik, 1986, 1991). A special double-prism glass is necessary for this technique, which intensifies the effect of chromatic aberration (Figure 18.2). A number of examples of the use of this method in geovisualization applications exist (Toutin, 1997; Hoeppner, 1999). Both, anaglyphs and chromostereoscopy have a considerable graphic drawback affecting their unrestricted utilization: the graphic degrees of freedom available in true 3D techniques are a visual property of some significance. This degree of freedom describes the independent application of graphic design elements such as colour, brightness, size, etc. to the technique. This parameter can strongly influence the decision to use a certain technique since the degrees of freedom might be a constraint on the depiction. For example, chromostereoscopy cannot map colour hue as the planes of depth are encoded using hue. Common anaglyphs where the eyes are fed by colour-separated stereo pairs are also limited in their ability to depict additional information in this way. In addition to these graphic drawbacks, the glasses that are needed to generate the perceivable stereo-effect might also be seen as a further drawback of these two techniques.
18.4.3
Dresden 3D LC display
The Dresden 3D LC Display (D4D) is a flat autostereoscopic display for visual presentation of geographic data, as shown in Figure 18.3a. It runs on any type of Pentiumbased personal computer with a standard graphic card and on SGI Workstations. The observer can view the display within an angle of about 50 degrees and is able to move laterally with freedom within this range. The most important components of the D4D are the prism mask technique, the eye finding system and the liquid crystal (LC) screen. Two stereo images of an object are supplied to the observer's eyes through a prism mask. A liquid crystal display (LCD) is driven by transmitted light (Figure 18.3b). Both images of the stereo pair are interlaced columnwise, such that the right stereo-image is written into the even display columns and the left one into the odd columns. Once a light ray passes through an even LC column, it is directed to the fight eye of the observer by means of a prism mask. For the left stereo image, the light is correspondingly directed to the left eye. The D4D principle allows the observer to move laterally. A camera-based eye finder, a system that is based on pattern recognition, locates the observer. In order to follow the movement of the observer and to present the spatial image without any crossover, the optical path of the stereo pair has to be moved in correspondence with the viewer. This is achieved either by shifting the stereo pair electronically or by shifting the prism mask mechanically (Schwerdtner, 2003). Further information in terms of geovisualization is provided by Liehmann (2002), whose work provides an overview of implemented applications such as interactive overflights, static and dynamic panoramic views and topographic representations. Technical features and aspects of design are analysed in order to adjust this true 3D technique to geovisualization.
370
S. Kirschenbauer
Figure 18.2. Principle of Chromostereoscopy: Blue colours appear behind the image plane whilst red colours appear in front of it when using the double prism mask. Yellow colours are perceived to reside on the image plane itself.
The Dresden 3D LC Display was employed as the test medium in an empirical study of true 3D presented below. The Dresden 3D LC Display has been chosen because 9 9
9
it is one of the most forward-looking true 3D techniques in technological terms; it allows the direct comparison between the 2D map and the analogous true 3D map enabling the effect of the visualized true third dimension on task-specific map reading to be isolated and analysed independently of other factors; some of the questions regarding the utility of this technique for geovisualization could be addressed directly.
Figure 18.3. (a) The Dresden 3D LC Display (D4D) - Research Group 3D-Display. Dresden University of Technology. (b) Principle of the D4D technique.
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
18.5
371
The Link between True 3D Techniques and Geovisualization
Geovisualization is primarily concerned with representing and displaying the 3D physical environment. In the majority of cases, the visualizer is constrained by the need to portray 3D information on a 2D medium or display device. This dimensional reduction necessitates the use of various graphic methods to simulate 3D, such as depth cues (see w18.3). They are used to substitute the "lost" third dimension. In the long history of Cartography, a great variety of such 3D simulation methods have been emerged (Kraak, 1988). Hill shading is probably the best known. One of the peculiarities of effective methods for simulating 3D is that they are taking advantage of characteristics of the human visual system developed for experiencing the real world to give an illusion of three-dimensionality in a 2D depiction. Recent exciting developments in computer technology offer novel methods and techniques for 3D geovisualization. These include advances in both software and hardware. One example involves technologies that generated motion parallax. However, the Cartography and conventions of the past and many of the maps of the digital epoch share an important characteristic: their medium for representation is 2D. The paper plane has been complemented or replaced by the similarly 2D plane of the screen. Thus, the depiction of the third dimension remains as a prevailing and important research subject that has been energized by the recent technological advances. True 3D technologies are an alternative to current, standard two- and 3D geovisualization methods. Whilst also relying upon 2D media (the flat display screen) they make it possible to utilize the third dimension in our graphic representations by offering a virtual 3D (in the sense of a stereogram) or a real 3D space. True 3D techniques have a fairly long history. Graf (1943), Carlberg (1943) introduced cartographic true 3D methods in the 1940s. But the methods have been applied for geovisualization only relatively recently; (see Wood et al., this volume (Chapter 14)). The recent profound advances in the techniques that make true 3D achievable have benefited the implementation and use of these technologies. But under what circumstances might such a 3D map be effectively used? In order to answer this question considerable attention has to be paid to the purpose of the map, the tasks that are to be solved by using the map and to the potential user groups working with the map. Basically, true 3D techniques are utilized in order to display phenomena with significant spatial expansion such as relief or objects with a relative spatial expansion like urban environments. Interactions of different phenomena, movement of phenomena in space (e.g., in geology, climatology, landscape architecture), etc. are also popular contents of depiction. Fields in which true 3D techniques have been applied in geovisualization include: 9
9
planning processes - e.g., climatological planning or landscaping with an immersive workbench (Fuhrmann and MacEachren, 1999), also Uhlenktiken et al. (1999), Zehner and Ktimpel (2001); analyses - e.g., environmental analyses with holography on the basis of remotely sensed data (Benton et al., 1985);
372
S. Kirschenbauer
9 9
9
18.6
education - e.g., the use of anaglyphs for education in geology and Geography at North Dakota State University (2002); information sources - e.g., the "SRTM (endeavour) mission" representation of 3D data with various stereoscopic techniques (NASA, 2003), or the "True 3D Atlas" on the Dresden 3D LC Display (Liehmann, 2003); maps as art - e.g., the work of Ambroziak and Ambroziak (1999) who provide numerous examples in their latest publication.
Cartographic Theoretical Framework for True 3D Techniques
In order to assess the potential and utility of true 3D techniques for geovisualization, it is essential to build a solid theoretical base. Cartographic theories as well as perceptual and cognitive theories serve as the primary constituents. The application of various true 3D techniques in geovisualization allows visualization researchers and domain scientists to evaluate the potential of the technique informally, and their findings on the benefits and usefulness of the technique can be applied to refining the technology or its application. Yet, there is a need for empirical analysis and objective measurement to acquire more robust knowledge about the utility of the possibilities afforded by true 3D; (see Wood et al., this volume (Chapter 14) and Fuhrmann et al., this volume (Chapter 28)). Some of the cartographic theories provide approaches that can be adjusted for use with true 3D techniques. For example, the theories of map properties and cartosemantics. Others, such as the theory of cartosyntactics or the Graphic System, cannot cover the unique properties of true 3D at all. Yet, cartographic theories regarding representation using true 3D are essential for efficient, purpose- and user-oriented design, and so new theories have to be developed. An appropriate application of Gestalt elements such as depth cues, but also colour, brightness, etc. according to the particular technology is one of the main goals of these theories. In addition, it is necessary to delineate the numerous parameters involved in map-reading processes for which the technique is expected to be useful. This touches upon both cartographic theories and perceptual/cognitive theories. The latter are indispensable for the assessment of true 3D. Providing the contributory thematic information, perceptual/cognitive theories facilitates our understanding of the ways that humans see and think, of humans perceptual/cognitive capacities and of their limitations as well enabling us to understand and take advantage of the principle of true 3D techniques discussed earlier. When considering these theories, one thing becomes clear: opinions concerning the use of true 3D techniques in the context of geovisualization remain to be confirmed empirically. In doing so, the theoretical knowledge will be augmented with experimentally derived substance. Two selected theoretical approaches are introduced in the remainder of this chapter. The value of both models is corroborated by several psychological and accepted cartographic theories. Taking these models as a basis upon which to structure the research
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
373
and applying it to the complex realm of true 3D techniques offers a conceptual framework through which we can acquire generic knowledge about the technique. Gilmartin (1981) has provided a universally valid approach for empirical investigations in Cartography, describing the elementary domains of a map-reading process. The map-reading process is divided into three domains using Gilmartin' s approach: primarily psychophysical; primarily cognitive; integrated psychophysical-cognitive. The primarily psychophysical sphere spans the scope of perception. It identifies the relationship between the (objective) physical stimuli, emitted by a true 3D image or one of its single symbols and their perception by the observer as well as her/his (subjective) sensations that are stimulated as a result. The primarily cognitive sphere includes the processing of the stimuli perceived from the true 3D image. This sphere of "information processing" is strongly influenced by a number of factors including the observer's experience, knowledge and sentiments. The psychophysical-cognitive sphere, which integrates both domains, relates to the complete map-reading process as a whole. The results of a cognitive experiment involving the evaluation of a map-reading process in terms of cognition might be influenced by psychophysical factors and vice versa. Research into the integrated psychophysical-cognitive sphere focuses on such possible interdependencies. Here, light is shed on overall-coherence of different influencing components. True 3D techniques, as a novel method of depiction in geovisualization, deserve investigations in terms of all the three spheres. As the technique is currently far from common, cognitive factors may be expected to have a strong influence. Many answers to open questions regarding the utility of true 3D might be unearthed when considering the cognitive sphere. In the second theoretical approach introduced here, characteristics of true 3D images are listed. Here, the coherent classification of Castner and Eastman (1984) serves as a general source. Basically, they distinguish between "spontaneous looking" where the observer does not have predetermined information needs and "task-specific viewing" where the observer views an image in order to fulfil a specific goal. This distinction is of great importance as particular properties of an image affect and influence these different viewing processes in a number of ways. Image characteristics are of three kinds: 9 9 9
Physical properties - design elements (single symbols or stimulus) such as colour, brightness, shading, texture, etc. Gestalt properties - cohesive design complexes (more than one single symbols or stimulus) such as figure-ground, parallelism, similarity, proximity, etc. Cognitive properties - intellectual associations stemming from user's individual experience and knowledge. Specific cultural and the familiar physical environments also impact the associations and expectations of individual users.
Physical and Gestalt properties of a (true 3D) image have a greater effect on perception processes when images are viewed spontaneously, yet when an image is viewed in order to solve a given task, cognitive properties come to the fore. When viewing is task-specific, graphic design elements have a much less significant effect on the perception process. Their visual stimulus is closely related to the given task. Alternatively, when viewing a map spontaneously, graphic design elements "regulate"
374
S. Kirschenbauer
the points or areas of attention. The amount of impact that each of the properties ultimately has in these perception processes, is highly dependent upon the individual user. For instance, any image displayed by an immersive workbench will generally result in greater visual attention amongst those users whose experience with this technique is relatively modest or who are not familiar with such a technique. Two different examples illustrate the serious differences between Physical, Gestalt and Cognitive properties: generally, true 3D techniques create a high figural dominance which results in a comparatively clear figure-ground relationship. The stimuli determining a figure-ground relationship attracts the visual system to a high degree (Kahnemann, 1973). Hence, any true 3D image might be very attractive to its user when being viewed spontaneously. In this stage of perception, graphic elements have no semantic significance. This issue changes when the use involves a specific task that must be completed. In this case a dominant figure-ground relationship can affect the analytical process adversely by superimposing detailed information of the visual ground. Phillips and Noyes (1978) provide additional evidence. Results of their experiment show a less impressive performance by users of 3D panoramic views than those employing topographic or layer tint maps. Yet, map users, especially lay people, often prefer panoramic views as they comply with the natural view. Some of the characteristics of a true 3D image are comparatively straightforward, but most of them have to be examined in terms of their effects and impacts in the context of specific applications and their potential users.
18.7
Empirical Study to Examine True 3D
The research design used here focuses attention on the role of the true third dimension in geovisualization, with particular consideration of image's properties and its users. Thus, the distinction between spontaneous looking and "task-oriented viewing" discussed above provides a fundamental component of the strategy developed for judging success of a true 3D image. The specific problem considered here, is the ability of users to extract 3D information from topographic relief maps. The test maps used in the empirical study reported below are of two kinds: a topographic 2D paper map, where contour lines display the measurable third dimension along with the monocular depth cue "shading", and a map containing each of these techniques that has been implemented in true 3D using the autostereoscopic Dresden 3D LC Display. Here, the spatial information has been enhanced using the binocular depth cue binocular disparity. The true 3D realization is compared with the 2D paper map containing the same information, in a task-specific context. The topographic test map is described as 2D even though by the definition described in w it can be considered a 3D map. It is described as 2D as this is a conventional accepted term for this type of paper map. Whether the true third dimension influences users' operations, and if so, how does this manifest, is a question that is directed towards the integrated psychophysicalcognitive sphere. The goal of the empirical study reported here is to provide a broader
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
375
picture of the whole process. The research question is one that relates to many aspects of true 3D, especially when users are considered. In this experiment the experience with topographic maps, existing knowledge and cognitive style - field dependency/independency - classify different user groups. Field dependency/independency is thought to be important since it can indicate the ability to deal with figure-ground information as arising through shape-from-shading or shape-from-stereo, accurately and quickly (Riding and Rayner, 1998). Knowledge and prior experience enable us to read or interpret a (true 3D) image (Gregory, 1991). By studying internally homogenous user groups with different levels of experience and knowledge the effect of these factors on task-oriented true 3D map reading can be assessed. The tasks considered in the experiment were designed to be as realistic as possible. Real topographic maps were used, as opposed to simplified synthetic cartographic products. This practical nature is also reflected in the user groups that were tested. 18.7.1
Subjects
A total of 90 subjects were subdivided in three different homogenous user groups, each consisting of 30 individuals who participated in the test. Three groups of typical map users were selected representing "experts" (mountain guides), "users" in the wider sense (military jet pilots) and "neophytes" (lay people). These three test groups were chosen due to their knowledge of reading and using topographic maps (of high mountain areas in this instance) and their experience of using them for a number of tasks. 18.7.2
Apparatus
A topographic sheet published by the German Alpenverein was selected to serve as test map. It displays a part of the Austrian Alps. The area chosen for the experiment shows relief in an almost pure state as additional features such as ski-lifts, pipelines, and the like are absent from the selected section. In order to compare the original 2D paper map with its copy portrayed as a true 3D image, the topographic map was edited to support presentation on the Dresden 3D LC Display. This was achieved by scaling the map in order to meet the maximum of resolution, presently available by the 3D Display. Different signatures such as lines for trails and points for huts have been drawn in both maps. They served as target points in the map for some of the tasks. Each signature was positioned differently in both maps in order to allow the same questions for different targets. A test question booklet was designed consisting of questions that cover the nonexpert use of this type of map as an information resource and orientation aid. Characteristic tasks include the estimation of heights, height differences and distances as well as the identification of landform and slope. For example, in order to test the ability to assign heights and height differences, subjects were asked the highest and lowest point on the map and questions were addressed such as "what is the height difference between points A and B?" or "what level is hut X standing on?". Users also had to identify the highest and lowest point on a specific trail plotted on the map. "Identification of landform
376
S. Kirschenbauer
and slope" was tested by asking subjects about the shape of relief in three different locations. For the completion of this task, subjects were given a choice of possible shapes including "concave", "convex" and "linear". In another task, users were required to identify the course of a line printed on the map. They had to find out how often this line ran uphill and downhill. An additional question asked subjects to locate the steepest slope on a defined area of the map.
18.7.3
Research design
In order to achieve an integrated psychophysical-cognitive research design, three different types of experiments were combined in the study: 9
9
9
18.7.4
In the first part of the experiment a standardized psychological test was conducted examining the cognitive style "field dependency/independency". This test consists of 40 tasks to be solved in 3 min. For example, one question shows five different figures (triangle, rectangle, and figures "U", "T" and "L"). In each case, one of these figures was hidden in a complex configuration from which the subjects have to visually extract the information. This test was used to determine abilities to organize stimuli accurately and quickly. In the main part of the experiment, subjects were presented successively with the 2D paper map and the digital true 3D map and asked to solve given tasks. The accuracy of responses and time taken to achieve them were recorded in order to generate a performance profile of the groups for the 2D paper map and the true 3D map. After each of the subjects had performed first on the 2D paper map and then on the 3D display, another short test was attached: subjects were required to solve a set of questions selected from the test question booklet on two further paper maps. One of these used the main test map at an enlarged scale, the other used the original scale of the topographic map (1:25,000). This additional test was conducted for two reasons: First, to evaluate the scale difference whether it has any impacts on the performance. Second, to examine learning effects that had arisen during the test. If there had been a learning effect as a result order in which tasks were attempted during the experiment it would have become evident in the additional test. At the end of the experiment, an informal interview was conducted in which qualitative data were collected. In a "free" conversation, subjects were asked to describe their opinions about the two maps presented and their experiences made in the test particularly in respect of their use of the true 3D map. These statements contain rational qualitative opinions on the experience and have been an inspiring supplement to the quantitative data.
Procedure
Each subject was tested on an individual basis. A single session took between 35 and 45 min. Before starting the test, the subject was briefed on the purpose of the experiment.
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
377
Tasks dealing with the maps were answered verbally, whilst the psychological test was conducted in written form. The test started with the psychological test followed by the main part of the experiment. The investigator read the instructions to the subject. Subjects then started with the 2D paper map and continued afterwards with the true 3D map. The two question booklets were used containing the same questions, only using different targets in the 2D map and the 3D map and in different order (to avoid any familiarization to the questions). The 2D paper map was presented to subjects vertically at a distance of 60 cm. This corresponded with the way in which the true 3D map was displayed on the autostereoscopic screen (Figure 18.4). Before starting with the 3D map, stereo viewing capabilities of subjects were tested. Subsequently, the additional test was conducted. The informal interview was used to close each session.
18.7.5
Results
The results presented below are a summary of the outcomes of the empirical study (for more detail see Kirschenbauer, 2003). A further detailed description of the results is beyond the scope of this chapter, however, those obtained so far provide some fundamental insights. The principle results of the experiment indicate that stereoscopic display can lead to an improved comprehension of spatial information such as relief information. The degree to which this is true varies according to users and tasks. This is reflected primarily in the time taken to achieve the answers to the questions, and also in terms of the overall accuracy of responses. Figure 18.5 shows that the true 3D image realized using an autostereoscopic display may significantly enhance the perception of spatial surfaces. Statistical analysis conducted on the measured time taken to complete tasks used a T-test to compare the two maps within a single test group. This revealed a highly
Figure 18.4. Empirical study in the reconnaissance squadron 51"1" of the German Federal Armed Forces in Kropp, Summer 2000.
378
S. Kirschenbauer
Figure 18.5. Performance spectrum. A correlation coefficient of time and accuracy was used to calculate the performance which was transformed in a non-standardized performance spectrum. This spectrum runs from low to high performance. It was undertaken in order to gain an overall impression of groups' performance and the way in which it varied according to whether the 2D paper map or true 3D map were used.
significant difference between the 2D map and the true 3D map for each of the groups, p --< 0.001 (Figure 18.6a). Essentially, two types of tasks can be distinguished: (i) tasks that refer to the spatial surface, i.e., domains/symbols representing depth, (ii) tasks that address noncontinuous information such as point symbols, text, etc. Figure 18.6b and c illustrates the different average times taken to complete the two types of tasks by each of the groups. There is a clear trend that both differences are statistically significant. Evaluations of these results using a one-way analysis of variance (ANOVA) showed significant differences between the groups (2D map (total time): p - < 0.001; 3D map (total time): p - < 0.01). Having these significant differences between all of the groups a Duncan' s Multiple Range Test (adopting the p < 0.05 level) was used to detect which particular group differs from the others (Tables 18.1a and 18.1b). Accuracy of task completion measured by number of correct answers given, was categorized in three classes with a maximum of 18 possible correct answers: category I: 0 - 8 ; category II: 9-13; category III: 14-18 correct answers. Using the Wilcoxon Test to analyse accuracy in terms of differences between the 2D map and the true 3D map,
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
379
Figure 18.6. (a) Total time (mean value) of each group. (b) Time (mean value) needed for tasks referring to non-continuous information. (c) Time (mean value) needed for tasks referring to spatial surfaces.
Figure 18.7. Analysis of the results demonstrates that users with a broad knowledge of topographic maps and great experience in using them (pilots) clearly improve their performance when using the true 3D map. Yet users with only modest knowledge and experience (laymen) achieve levels of accuracy with the true 3D map that are only equivalent to those obtained using the 2D paper map. The latter case is also true for the experts, the group of mountain guides with pertinent knowledge and relevant experience of topographic high mountain maps. (a) Sum of correct results, medium: 2D paper. (b) Sum of correct results, medium: true 3D map.
380
S. Kirschenbauer
Table 18.1a. Mountain guides required the most time for task completion, whereas laymen and pilots can be seen to have similar time scores on the 2D map. Subgroup for Alpha = 0.05 2D Map
N
1
2
Pilots Laymen Mountain guides Significance
31 29 29
232.9032 254.6207 0.246/ns
293.8966 1.O00/ns
different performances of each of the groups were revealed in a manner that was not anticipated (Figure 18.7a and b). Mountain guides did not demonstrate an improvement when using the true 3D map in terms of accuracy (p > 0.05), nor did the laymen (p > 0.05), although a graphic analysis of the results indicates a slight improvement on the true 3D map, especially on the tasks relating to non-continuous information. However, pilots achieved significant better accuracy on the true 3D map (p -< 0.001). Interpreting these results reveals that, the accuracy achieved when using the maps corresponds with the degree of knowledge and experience: the "experts" achieved the best results followed in turn by the "users" and "neophytes". Table 18.2 displays the results of an analysis of differences between each of the groups in terms of accuracy using the Kruskal-Wallis Test and for further details the U-Test of Mann and Whitney. Clear differences in performance are evident, particularly in relation to the 2D map, between the mountain guides on one hand and the pilots and laymen on the other. Mountain guides performed much more successfully on the 2D map than did the other two groups. However, when working with the true 3D map the performance of the pilots approached that of the expert group. The relationship between the groups also changed, with the mountain guides and the pilots on one side in this instance and the laymen on the other. The results of the standardized psychological test are listed in Table 18.3. The test uses the standardized C-values to analyse raw data. Here, the pilots can be Table 18.1b. On the true 3D map, task completion time of mountain guides decrease to a greater extent than it does with the other two groups. Considering the results in detail, the group "laymen-mountain guides" appears more homogeneous than a group "pilotslaymen" would do. Subgroup for Alpha = 0.05 True 3D Map
N
1
2
Pilots Laymen Mountain guides Significance
31 29 29
146.5161 168.7241 O.093/ns
168.7241 186.4828 O. 178/ns
Applying "True 3D" Techniques to Geovisualization: An Empirical Study
381
T a b l e 18.2. Groups that differ significantly from each other in their performance.
Medium
Group comparison
2D
Laymen-Pilots Laymen-Mountain guides Pilots-Mountain guides
3D
Laymen-Pilots Laymen-Mountain guides Pilots-Mountain guides
Measurand
Results in total category
Non-continuous results category
Spatial surface results category
Z P Z P Z P
- 0.243 0.808 - 3.078 0.002 - 2.905 0.004
- 1.003 0.316 - 1.378 0.168 - 2.443 0.015
- 0.373 0.709 -3.601 0.000 - 3.303 0.001
Z P Z P Z P
- 2.189 0.029 - 3.522 0.000 - 1.221 0.222
- 1.652 0.099 - 1.819 0.069 - 0.103 0.918
- 1.968 0.049 - 2.983 0.003 - 1.202 0.229
seen to have performed considerably better than the lay people although this result was not statistically significant (p > 0.05) and significantly better than mountain guides (p Michael Schroeder < / a u t h o r > . However, XML is only a first step, as another web page might refer to the same author using different XML tags: Michael Schroeder < / c r e a t o r > , thus making it difficult to automatically determine the equivalence of the two names. This shortcoming of XML is addressed by the semantic Web by using local and global ontologies to specify the schemas and meta data of the contents. Before we can give an example of such a global ontology, we need to resolve how to represent this information. Besides XML, there is the resource description framework (RDF), which allows one to capture meta data. RDF is based on triples of a subject, predicate, and object. A triple (s, p, o) expresses that a resource s has a property p with value o. Therefore, p is a binary relationship. However, RDF can express relationships of any arity (number of parameters) by simply splitting them into more than one triple. An object can also be a value, enabling triples to be chained, and in fact, any RDF statement can itself be an object or attribute - this is called reification and permits nesting. RDF Schema is to RDF what XML Schema is to XML: they permit the definition of a vocabulary. Essentially, RDF Schema provides a basic type system for RDF such as Class, subClassOf and subPropertyOf. RDF Schema is itself valid RDF expressions. To continue the authorship example above, there is one bit missing: a global ontology we can refer to. One example of such an effort is Dublin Core (Dublin Core Metadata Initiative, 2003), which defines standards for meta data. Dublin Core defines, for example, tags for the creator of a document.
482
M. Schroeder
Let us now use Dublin Core as a point of reference to describe authorship of a document building on XML and RDF: 1 < ?xml version= "1.0" ?> 2 4 5 Michael Schroeder 6 Information Agents 7 Information agents provide solutions for 8 information integration infrastructure, 9 consistency management, and information 10 visualization. 11 2002-10-10 12 te xt/html 13 en 14 < dc :publisher > ICA 15 16 Line 2 states that this XML document will contain RDF triples using the syntax referred to in the given URL. Line 3 imports the meta data tags defined in Dublin Core. Furthermore, line 2 and 3 abbreviate the corresponding pointers, which are used from line 4 to 16, as rdf and dc, Lines 5 to 14 contain the actual predicates and their values and among others the tag dc:creator followed by the author' s name. Because this tag refers to a global ontology, which is accessible to others, it can be used to create a joint understanding of the data across applications. In this particular example, all applications using Dublin Core (e.g., several well known search engines) will be able to answer a query for documents authored by Michael Schroeder properly. However, a global ontology may not be appropriate for all domains; therefore, global and local ontologies can be used. In domains, where cohesion is necessary and fruitful, organizations will develop standard ontologies (examples exist for the telecommunication industry, business processes, the paper supply chain, and human resources). This is also an active area in GIS (David et al., 2000; Fonseca et al., 2000, 2002; see Chapter 25). Fonseca et al. (2002) develop the idea of ontology-driven GIS. They show how to integrate geographical information systems using ontologies based on existing approaches such as WordNet (Fellbaum, 2000). The authors then argue that such ontologies can be mapped to interfaces, which connect the software components of the GIS. In contrast to the semantic Web, such an approach is not really open, as it does not provide a global ontology that is accessible online. Rather it hard-codes an ontology into the glue that integrates different components. A similar approach is followed by the OpenGIS consortium (Open GIS Consortium Inc., 2003a-c), which specifies ontologies for geographic objects using interface definition languages such as Corba IDL or Microsoft' s COM.
Intelligent Information Integration
483
A final step is the development of ontology mark-up languages that enable reasoning. DAML + OIL is such an effort, which defines a language to express relationships between classes and supports reasoning about these relationships. Another effort that focuses specifically on the use of rules to specify integrity constraints, deduction, and reactive behaviour is RuleML (Boley et al., 2001). RuleML (Boley, 2003) aims to standardize a rule mark-up language, which facilitates the interchange of rules. Such rules could be used to specify, for example, the semantic integrity of data.
24.4
Information Visualization and Visual Datamining
With the data consistently integrated, we can turn to the next challenge: how to transform it into knowledge? To this end, the idea of visual datamining is gaining momentum. While datamining focuses on algorithms to analyse the data, visual datamining emphasizes that the task is a human-centred process. In GIS, this idea of integrating traditional datamining techniques with interactive, visual exploration is actively pursued (Andrienko et al., 2001; Andrienko and Andrienko, 1999b; Gahegan et al., 2001; Guo et al., 2002; MacEachren et al., 1999a-c). The basis of such an approach derives from the three distinct areas of Information Visualization (comprehensively summarized in Ware, 2000; Spence, 2001), human-computer interaction, and datamining. From a human-computer interaction point of view it is important that the visual datamining process supports operations such as projections, filtering and selection, linking and brushing, zooming, details on demand, overviews, and visual querying. Besides supporting the human's interaction in the datamining process, visual datamining deploys Information Visualization techniques, which can be broadly classified as geometric, iconbased, pixel-oriented, and hierarchical (Keim and Ankerst, 2001; Keim, 2001; Keim, 2002). Here, we briefly review some of these techniques and put them into context.
24.4.1
Geometric techniques
Two very general geometric techniques are scatter plots and parallel coordinates (Inselberg and Dimsdale, 1990). In their most basic form, scatter plots depict objects with associated x- and y-value at the corresponding position of a coordinate system (Figure 24.1). The basic 2D approach can be extended to 3D, but suffers then from well-known problems of 3D such as occlusion and difference in perception of depth in comparison to height and width. It is also not obvious how to visualize high-dimensional data with a scatter plot. One approach is to apply dimension reduction, which can create difficulties in interpretation as information is lost when the data are transformed. Another approach simply plots all of the variables against each other, creating a quadratic number of 2D scatter plots. While no information is lost, the interpretation is nonetheless difficult, as many different plots need to be mentally linked. Furthermore, it is difficult to label objects in large scatter plots and the Euclidean space that the scatter plots use may not be appropriate for data that originates from a space with a different topology. However, scatter plots are simple, can give a good overview and depict the basic structure and are therefore very common. Another technique that is fairly general, simple, and therefore wide spread is parallel coordinates (Figure 24.1). Parallel coordinates also use a coordinate system as basis. In contrast
484
M. Schroeder
Figure 24.1. A scatter plot, parallel coordinates, and an information landscape tree created with Mineset (Mineset, 2002). The tree uses the coordinates of the plane to map the network and make use of the third dimension to represent information at nodes.
Figure 24.3. A pixel-oriented technique for multi-variate Information Visualization. This colour map shows variation amongst nine attributes for five entities.
Intelligent Information Integration
485
to scatter plots, they can display high-dimensional data, by associating an object's attributes with values on the x-axis. The corresponding value of the object' s attributes are then plotted along the y-axis creating a graph representing the object. Even if many graphs are plotted at the same time, a general trend can still be seen. However, a big problem is that often the order of attributes along the x-axis is arbitrary, although it has a significant effect on the parallel coordinates and so potentially upon the interpretation. Different orderings can lead to more or less "overplotting" of the graphs. User manipulation or computational sorting can address the ordering issue. Two less prevailing Information Visualization techniques are information landscapes and pro-section views. In information landscapes (see Figure 24.1 and discussion on "spatialization" in Chapter 35) two dimensions are used for spatial layout, while the third dimension is used for data display (Bray, 1996). Such landscapes are not truly 3D and do not suffer from many of the problems associated with true 3D views. However, they do exhibit some of the virtues of 3D. It is possible to seamlessly zoom from an overview to a detailed view of the data. Pro-section views (Furnas and Buja, 1994; Spence, 2001) are related to scatter plots. They address the problem of displaying high-dimensional data with a scatter plot. The idea is to reduce dimensionality by applying projections and sections to the data (hence the name).
24.4.2
Icon-based techniques
Icon-based techniques aim to preserve all of the information by mapping attributes to different visual features of an icon representing the object as a whole. Two prominent members of this class are starplots (Fienberg, 1979) and Chernoff faces (Chernoff, 1973). Star-plots (Figure 24.2) represent the value of an attribute through the length of lines radiating from the centre of the icon. The lines for all the attributes are distributed at an even angle around the centre, thus creating a star shape. Often the tips of the star's beams are connected in order to create a closed shape. Similar to parallel (a)
(b)
Figure 24.2. Icon-based techniques for multivariate Information Visualization. Chernoff faces and star plots concurrently depicting variation in eight attributes for four entities.
486
M. Schroeder
coordinates, star plots succeed in displaying high-dimensional data without any dimension reduction. But they also suffer from the same problem: the order of attributes has an impact on the resulting overall shape and therefore on how the data is perceived. Furthermore, starplots are difficult to compare to each other as it is difficult to quantify the differences. This applies also to Chernoff faces, which map attribute values to up to 18 facial features such as lips, nose, ears, etc. of a stylized face. The idea behind this mapping is that human cognition is especially adapted to recognize and compare faces. However, it is not clear whether this also applies to the stylized faces. In fact, experiments (Morris et al., 1999) indicate that perception of Chernoff faces is a serial process and not pre-attentative. While the faces are intuitive and compact (for up to 18 variables), they suffer from some problems. The display is limited to 18 variables and the facial features cannot be easily compared with each other. How does the size of an ear compare to the angle of an eyebrow? This is particularly problematic, as the facial properties have very different visual properties: perception of the area an oval covers (the face) is not comparable at all to perception of angles, line width, and curviness. As a result, a different assignment of attributes to facial features will change the perception of the face radically and the mapping of which variable to assign to which feature greatly influences the interpretation (Chuah and Eick, 1998). Additionally, the values of variables cannot be read from the faces' features, there may be an emotional component when interpreting faces, the faces' symmetry means redundancy of information, and the display of many faces may create a texture, which distracts from the interpretation of the individual faces. Nonetheless, Morris et al. (1999) report that Chernoff faces are useful for trend analysis, but less so for decision making. Overall, Chernoff faces are intuitive, but due to the above limitations they may be difficult to use effectively. Two other techniques, which use icons and can represent high-dimensional data are stick figures (Pickett and Grinstein, 1988) and colour icons (Keim and Kriegel, 1994). The former maps attribute values to angles between "sticks" that represent the attributes. Thus each object is mapped to a concatenation of sticks. A criticism that applies is that sequences of angles may not be optimal for perception. If, however, the spatial arrangement of a large number of stick figures is chosen appropriately, then they create a texture and can give a very good overall impression on the data as a whole. So, although they generally suffer from similar problems as Chernoff faces do, they can be used for a different purpose. Colour-icons map attribute values to colour, where the attribute itself has a fixed location in the icon. Again, the same problem arises as with many of the techniques above: the spatial arrangement of attributes is of great importance for the end result, especially, since neighbouring colours influence each other's perception by the user. The relative effectiveness of Chernoff faces, star plots, scatter plot matrices, and parallel coordinate plots is currently being investigated in the context of geospatial metadata visualization (Chapter 32). For examples of icon-based techniques in geovisualization see Gahegan (1998) and Dorling (1994).
Intelligent Information Integration
24.4.3
487
Pixel-oriented
While most geometric techniques tend to work well for a small numbers of attributes, iconbased approaches are not suitable for large and very large multi-variate data sets. Pixeloriented techniques complement this picture, as they tend to work well to get an overview over a large set of objects, which possibly have a large set of attributes. Colour maps (Bertin, 1981 - see Figure 24.3) are the most prominent pixel-oriented technique. Typically, colour maps are tables, whose rows contain the objects and columns the attributes. Each cell is then coloured according to the value of the column' s attribute for the row' s object. A limiting factor for the value range to be displayed relates to our perception of colour. While the space of different colours may be huge, creating linear colour scales with a high perceived resolution is not straight-forward (Ware, 1988, 2000). Cartographic research may provide some solutions (Brewer, 1994; Olson and Brewer, 1997). Using a single pixel on the screen as a cell, colour maps can easily display data sets as large as the medium's resolution caters for. However, in common with the other techniques, colour maps highly depend on the order of columns and rows and of the choice of colour mapping. One approach is to cluster objects and attributes and order them according to their similarity. This will ensure that the neighbourhood of a cell is not too different from itself, so that colours re-enforce themselves creating regions with boundaries instead of seemingly random spots of colour spread all over the display (Bertin, 1981; M~ikinen and Siirtola 2000; Siirtola 1999).
24.4.4
Hierarchical techniques
This type is particularly useful if the given data is already hierarchical by nature. One way of generating hierarchies from non-hierarchical data is to use hierarchical clustering. This produces binary trees, whose leaves represent the objects and whose parent nodes correspond to clusters of objects. Hierarchical clustering is used in Guo et al. (2002) to cluster multi-variate spatial data. The clusters are often displayed as dendrograms drawings of binary trees, with the additional convention that the difference in height between parent and children indicates the similarity of the two children. Consider the example shown on the left of Figure 24.4. Let us assume that objects A, B, C, and D have the values 1, 3, 8, and 12, respectively. If we define the similarity between two objects as the absolute difference between the corresponding values, then A and B with a similarity of 2 are a first cluster and C and D are a second cluster with a similarity of 4. If we define the similarity between clusters as the similarity of their nearest neighbours, then the first cluster has a similarity of 5 to the second one. The above hierarchical clustering is depicted in the dendrogram in Figure 24.4. If the clusters are associated with a value representing the cluster, then tree maps (Shneiderman, 1992; Herman et al., 2000) are useful visualizations that convey the value attached to a group of objects effectively. In tree maps, an object or group of objects is represented as a rectangle, whose size reflects the attribute value. Figure 24.4 provides an example, where the rectangle for a group of objects is filled with the rectangles representing its members. As a result, tree maps are good at conveying aggregated single attribute values of a hierarchical structure. For the example, the tree map
488
M. Schroeder
Figure 24.4. Hierarchical techniques for multi-variate Information Visualization. The dendrogram and tree map show four cases classified into two clusters. The cone tree can be "spun" interactively to reveal detail on demand (image courtesy of Jameel Khan).
in Figure 24.4 shows two main rectangles of an area of 1 + 3 = 4 and 12 + 8 - 20. However, a key difficulty is associated with tree maps. Should doubling of the attribute's value lead to an area of double the size or a base line of doubled length (and thus a quadrupled area)? Invariably, no matter which choice is taken, users may have the opposite expectation. Cone trees are a final hierarchical technique (Robertson et al., 1991; Robertson et al., 1993). Effectively, they consist of 3D representation of dendrograms. Like other 3D representations, they can have the benefit of allowing users to easily zoom between overview and detail. But they also suffer from the problem of occlusion and difficulty to navigate and find the desired information. An example is shown in Figure 24.4. To summarize, visual datamining builds heavily on Information Visualization techniques to turn datamining into a human-centred process. Different Information Visualization techniques are useful for different types and sizes of data and at different stages of the datamining process. The hierarchical techniques are suitable for tree structures, which can result from hierarchical clustering. High-dimensional data can either be reduced in its dimensionality or directly visualized. In the former case, scatter
Intelligent Information Integration
489
plots are very useful. In the latter case, parallel coordinates and stick figures can provide an effective overview over the data even if it is of high dimensionality. If the data does not contain too many attributes (up to 20 as an approximation), then the icon-based methods (Chemoff faces and star-plots) can be effectively used. To produce the data suitable for the above techniques a host of algorithms for clustering and dimension reduction are applicable. The data and the analysis algorithms link visual datamining to the Grid and semantic Web as discussed in w and w respectively. The data to be visualized will often come from different sources that require consistent integration, which can be achieved through global taxonomies as promoted by the semantic Web. The analysis algorithms transforming and preparing the data for visualization and the visualization itself are often computationally very intensive and could use the Grid to run efficiently.
24.5
Cross-Fertilization
To put it in a nutshell, information agents perform intelligent information integration. To implement such agents, an infrastructure is required that supports gathering and efficient processing of large data sets. This can be achieved by a Grid. The agents need to consistently integrate the data, which can be supported by the semantic Web, and finally the agents need to present results to the user, which can be done using visual datamining techniques. How do these trends - the Grid, the semantic Web, and visual datamining relate to geovisualization? MacEachren and Kraak (2001), pose a number of research challenges to geovisualization. One of three main challenges relates to visualizationcomputation integration and in particular: "3. To address the engineering problem of bringing together disparate technologies, each with established tools, systems, data structures and interfaces. Four specific problems identified are: [3.1] to develop computational architectures that support integrating databases with visualization; [3.2] identify the database functions needed to support the real-time interaction demanded by visually facilitated knowledge construction; [3.3] determine the impact that underlying data structures have on the knowledge construction process; and [3.4] develop mechanisms for working discovered objects back into a consistent data model." (MacEachren and Kraak, 2001) An example, where the above problems are tackled for a specific system has been implemented by Andrienko and Andrienko (1999a,b). Their system integrates the datamining tool Kepler and the geovisualization system Descartes. Kepler analyses economic and demographic data for different European countries. It accesses the appropriate databases and runs learning algorithms to relate the different relations such as gross domestic product or infant mortality. The results of Kepler are then visualized as a map by Descartes. Both systems run independently and act as servers, which cooperate and which are accessed by a user's clients, which in turn are linked, too.
490
24.5.1
M. Schroeder
The Grid
Given the challenges and the above example system, how can the open problems be addressed in a principled and general way? Problems 3.1 and 3.2 allude to the infrastructure required to facilitate information integration including tools and data, which arc often physically distributed, as in the example system used by the scenario developed in the introduction to this section (Brodlie et al., this volume (Chapter 21)). The reference in point 3.2 to real-time interaction, which is also present in the example system, also means that computations have to be fast. This is often only possible with bundled computational power. Both concerns - the transparent access to distributed resources and the provision of computational power - are catered for by the Grid. The Grid can therefore form the backbone for such systems. In the system developed by Andrienko and Andrienko (1999a,b), Descartes and Kepler could use the Grid to perform computationally intensive tasks such as rendering of maps and the execution of the datamining algorithms. In their current clientserver implementation the servers could become bottlenecks if too many clients access them. In a Grid implementation of such a system, the clients could directly initiate the computations of Descartes and Kepler in locations, which make the best trade-off between, for example, available CPU cycles and proximity to the client to reduce latency due to transfer of large data sets.
24,5.2
Semantic W e b
Challenges 3.3 and 3.4 identified by MacEachren and Kraak (1991) touch on the importance of data structures, data models, and consistency in general. In the Andrienko's system the integration is hard-coded and mappings between the different systems have to be carefully designed. The general problem of semantic consistency and inter-operability between distributed data sources is the concern of the semantic Web. The semantic Web can provide the technology needed to define standardized ontologies, which can be complemented by local ontologies where appropriate. In such a system, data sources can be linked automatically, as the tags of a data entry refer to a common global ontology and thus indicate that a concept used in one source has the same meaning as in another. Such a mechanism contributes to the solution of consistent data models put forward by MacEachren and Kraak (2001), which is a prerequisite for automated integration. This work is particularly interesting as standard ontologies for GIS are currently being developed (Fonseca et al., 2000, 2002; David et al., 2000; Open GIS Consortium Inc., 2003a-c).
24.5.3
Visual datamining
MacEachren and Kraak (2001) explicitly refer to the integration of Knowledge Discovery and Datamining (KDD) and the example system by Andrienko and Andrienko is such an integration of a KDD and a GIS tool to facilitate explorative data analysis and visual datamining. All results in these two areas regarding processes and techniques may be useful for more specific geographical data as well. However, a specific concern relates to "how to incorporate the location and time components of multi-variate data within visual and analytical methods" (MacEachren and Kraak, 2001). This question is not (yet)
Intelligent Information Integration
491
answered by visual datamining, but it does make an important contribution, emphasizing interaction and visualization as essential components of datamining. Interaction is a challenge for both geovisualization and datamining due to the exploratory nature of the activities (see Section B on 'Creating Instruments for Ideation'; for example, Andrienko et al., this volume (Chapter 5)).
24.6
Conclusion
To summarize, in this article we have reviewed three major enabling technologies for intelligent information integration, namely Grid computing, the semantic Web, and visual datamining. We have discussed their relation to geovisualization by showing how they address geovisualization challenges put forward in MacEachren and Kraak (2001). In future geovisualization systems, the Grid could provide the infrastructure for transparent access to distributed data and computational resource, the semantic Web could be used to achieve automatic, dynamic, and consistent data integration, and visual datamining could be used to visually explore geographic data.
Acknowledgements I would like to thank Jason Dykes, Menno-Jan Kraak, Alan MacEachren, Penny Noy, and the anonymous reviewers for their valuable and detailed comments on this chapter, which have helped inform and develop the ideas presented and improved their communication. Thanks to Jameel Khan for the cone trees shown in Figure 24.4.
References Andrienko, G., and Andrienko, N., (1999a) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999b) "Knowledge-based visualization to support spatial data mining", In Advances in Intelligent Data Analysis (3rd International Symposium, IDA-99, Amsterdan, The Netherlands, August 9-11, 1999), Lecture Notes in Computer Science, Vol. 1642 (eds., Hand, D.J., Kok, J.N., and Berthold, M.R), Berlin: Springer Verlag, pp. 149-160. Andrienko, N., Andrienko, G., Savinov, A., Voss, H., and Wettschereck, D., (2001) "Exploratory analysis of spatial data using interactive maps and data mining", Cartography and Geographic Information Science, 28(3), 151-165. Berners-Lee, T., Hendler, J., and Lassila, O., (2001) The Semantic Web, Scientific American. Online: www.sciam.com/article.cfm?articleID = 00048144-10D2-1C7084A9809EC588EF21 Bertin, J., (1981) Graphics and Graphic Information Processing. Berlin: Walter de Gruyter, p. 273. Boley, H., (2003) RuleML Homepage. Online: http://www.ruleml.org (23/10/03).
492
M. Schroeder
Boley, H., Tabet, S., and Wagner, G., (2001) "Design rationale of RuleML: a markup language for semantic web rules", Proceedings of SWWSO1, Standford, CA, USA. Bray, T., (1996) "Measuring the web", 5th International WWW Conference. Brewer, C. A., (1994) "Color use guidelines for mapping and visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography, Vol. 2. Oxford: Elsevier Science Ltd., pp. 123-148. Chernoff, H., (1973) "The use of faces to represent points in k-dimensional space graphically", Journal of the American Statistical Association, 68(342), 361-368. Chuah, M. C., and Eick, S. G., (1998) "Information rich glyphs for software management data", IEEE Computer Graphics and Applications. David, M., Egenhofer, M., Hirtle, S., and Smith, B., (2000) UCGIS Emerging Research Theme: Ontological Foundations for Geographic Information Science, Technical Report: UCGIS, Online: www.ucgis.org/ermerging/ontology Distributed Computing Technologies, I. (2003) distributed.net: Node Zero. Online: http://www.distributed.net (23/10/03). Dorling, D., (1994) "Cartograms for visualizing human geography", In: Hearnshaw, H.M., and Unwin, D. J., (eds.), Visualization in Geographical Information Systems, pp. 85-102. Dublin Core Metadata Initiative (2003) Dublin Core Metadata Initiative (DCMI). Online: http://www.dublincore.org (23/10/03). Fellbaum, C., (2000) WordNet: An Electronic Lexical Database, MIT Press. Fienberg, S. E., (1979) "Graphical methods in statistics", American Statisticians, 33, 165-178. Fonseca, F., Egenhofer, M., Davis, C., and Borges, K., (2000) "Ontologies and knowledge sharing in urban GIS", Computer, Environment and Urban Systems, 24(3), 232-251. Fonseca, F., Egenhofer, M., Agouris, P., and Cfimara, G., (2002) "Using ontologies for integrated geographic information systems", Transactions in GIS, 6(3), 231-257. Foster, I., (1995) Designing and Building Parallel Programs, Addison-Wesley. Foster, I., and Kesselman, C., (1999) The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers. Furnas, G. W., and Buja, A., (1994) "Prosections views: dimensional inference through sections and projections", Journal of Computational and Graphical Statistics, 3(4), 323-353. Gahegan, M., (1998) "Scatterplots and scenes: visualization techniques for exploratory spatial analysis", Computers, Environment and Urban Systems, 22(1), 43-56. Gahegan, M., Harrower, M., Rhyne, T. -M., and Wachowicz, M., (2001) "The integration of geographic visualization with databases, data mining, knowledge construction and geocomputation", Cartography and Geographic Information Science, 28(1), 29-44. Guo, D., Peuquet, D., and Gahegan, M., (2002) "Opening the black box: interactive hierarchical clustering for multi-variate spatial patterns", Proceedings, The lOth A CM International Symposium on Advances in Geographic Information Systems, McLean, VA, pp. 131-136.
Intelligent Information Integration
493
Herman, I., Melancon, G., and Marshall, M., (2000) "Graph visualization and navigation in information visualisation: a survey", IEEE Transactions on Visualization and Computer Graphics, 6(1), 24-43. Inselberg, A., Dimsdale, B., (1990) "Parallel coordinates: a tool for visualizing multidimensional geometry", Proceedings Visualization 90, San Francisco, CA, pp. 361-370. Keim, D. A., (2001) "Visual exploration of large data sets", Communications of the A CM (CA CM), 44(8), 38-44. Keim, D. A., (2002) "Information visualization and visual data mining", IEEE Transactions on Visualization and Computer Graphics, 8(1), 1-8. Keim, D. A., and Ankerst, M., (2001) Visual Data Mining and Exploration of Large Databases (Tutorial at ECML/PKDD01). Keim, D. A., and Kriegel, H. -P., (1994) "VisDB: database exploration using multidimensional visualization", Computer Graphics and Applications, 6, 40-49. MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization, cartography and geographic information science", Special Issue on Geovisualization, 28(1), 3-12. MacEachren, A. M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., (1999a) "Exploring the potential of virtual environments for geographic visualization", Annual Meeting of the Association of American Geographers, Honolulu, HI, 23-27, March: AAG, pp. 371 (full paper: www.geovista.psu.edu/library/aag99vr). MacEachren, A. M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., (1999b) "Virtual environments for geographic visualization: potential and challenges", Proceedings of the A CM Workshop on New Paradigms in Information Visualization and Manipulation, Kansas City, KS. MacEachren, A. M., Wachowicz, M., Edsall, R., Haug, D., and Masters, R., (1999c) "Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods", International Journal of Geographical Information Science, 13(4), 311-334. Meuer, H. -W., Strohmaier, E., and Dongarra, J., (2002) "Top500 list", High Performance Networking and Computing Conference. Online: http://www.netlib.org/benchmark/top500.html Morris, C. J., Ebert, D. S., and Rheingans, P., (1999) "An experimental analysis of the effectiveness of features in Chernoff faces", Applied Imagery Pattern Recognition '99: 3D Visualization for Data Exploration and Decision Making. Olson, J., and Brewer, C. A., (1997) "An evaluation of color selections to accommodate maps users with color-vision impairments", Annals of the Association of American Geographers, 87(1), 103-134. Open GIS Consortium Inc., (2003a) Geographic Objects Initiative (GO-l). Online: http://ip.opengis.org/gol/(23/10/03). Open GIS Consortium Inc., (2003b) Geography Markup Language (GML 3.0). Online: http://www.opengis.org/docs/02-023r4.pdf (23/10/03).
494
M. Schroeder
Open GIS Consortium Inc., (2003c) Open GIS Consortium, Inc. (OGC). Online: http://www.opengis.org (23/10/03). Pickett, R. M., and Grinstein, G. G., (1988) "Iconographic displays for visualizing multidimensional data", Proceedings IEEE Conference on Systems, Man and Cybernetics, Piscataway, NJ: IEEE Press, pp. 514-519. Platform Computing Inc. (2003) Platform Computing - Accelerating Intelligence - Grid Computing. Online: http://www.platform.com (23/10/03). Robertson, G. G., Mackinlay, J. D., and Card, S. K., (1991) "Cone trees: animated {3D} visualizations of hierarchical information", Proceedings Human Factors in Computing Systems CHI 91 Conference, New Orleans, LA, pp. 189-194. Robertson, J. G., Card, S. K., and Mackinlay, J. D., (1993) "Information visualization using 3(D) interactive animation", Communications of the A CM, 36(4), 57-71. Shneiderman, B., (1992) "Tree visualization with Treemaps: a 2-d space-filling approach", A CM Transactions on Graphics, 11 (1), 92-99. Spence, R., (2001) Information Visualization. Harlow: Addison Wesley/ACM Press Books, 206 pp. The Globus Alliance (2003). The Globus Alliance. Online: http://www.globus.org The Legion Project (2001) Legion: Overview. Online: http://legion.virginia.edu/overview.html (23/10/03). The TeraGrid Project (2003) TeraGrid. Online: http://www.teragrid.org (23/10/03). Ware, C., (1988) "Color sequences for univariate maps: theory, experiments and principles", IEEE Computer Graphics and Applications, 8(5), 41-49. Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, 384 pp. Wiederhold, G., and Genesereth, M., (1996) The Basis for Mediation, Technical Report: Stanford University.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 ElsevierLtd. All rights reserved.
Chapter 25
A Knowledge-based Collaborative Environment for Geovisualization: Ontologies for Multiple Perspectives on Distributed Data Resources Zarine Kemp, Computing Laboratory, University of Kent, Canterbury, Kent CT2 7NF, UK
Keywords: data-centred geovisualization, semantic knowledge, ontologies, multidimensional analysis, knowledge discovery, user interfaces
Abstract This chapter contributes to the theme "connecting people, data and resources" of the ICA research agenda on visualization and virtual environments. It focuses on the semantic capabilities required to enable sharing of distributed, disparate data resources within a geo-scientific community. The chapter presents the rationale for underlying semantics to be integrated into the interface in a computational environment where the data are multi-dimensional and the geovisualization requirements open-ended. The concept of ontological hierarchies in space, time and scientific dimensions is presented using illustrative examples. The conceptual design of the interface and the architecture of the prototype system are described and examples presented. The chapter also describes other scientific visualization systems and reflects upon the underlying issues and problems.
25.1
Introduction
The introductory chapter in this section focuses on the disparate themes that need to be addressed to enable geovisualization in a distributed environment, (see Brodlie et al., this volume (Chapter 21)). This chapter concentrates particularly on the computational support required to enable distributed communities of users to share heterogeneous data resources for geovisualization in a collaborative environment. Application domains such as environmental informatics require capabilities for managing and visualizing georeferenced data. With the widespread use of the Internet, the World Wide Web and related technologies, access to distributed heterogeneous information repositories has become a reality for users. This is particularly the case in scientific research 495
496
Z. Kemp
communities where observational and experimental data is time consuming and difficult to acquire and may be required over a long time span for comparative research and analysis. Computer Science research and its perspectives on information system technology plays a foundational role in encompassing the complexity, scale and special characteristics of the data and the need to integrate spatio-temporal structures and processes into the geovisualization framework. The focus in these domains is on the evolution of geovisualization from cartographic representation as an end product of spatial analyses to that of visualization for manipulation and understanding of data, reasoning and knowledge discovery (Gahegan et al., 2001; Wachowicz, 2001). The motivating examples for the research described in this chapter arise out of prototype marine environmental applications. The first one, Pontus (Frank and Kemp, 2001), is a geographical information system (GIS) for marine biodiversity research and the other, FishCAM (Kemp and Meaden, 2001) is a system for fisheries monitoring and decision support. It should be emphasized that the characteristics of the data resources and requirements discussed are not unique to marine systems, but arise in many domains where the observational data are geo-referenced and where cartographic visualization is an integral part of the process of understanding and evaluating the knowledge encompassed in the observational data resources (MacEachren et al., 1999c). Intelligent use of global information resources requires harnessing the information to analysis and interpretation techniques linked to scientific visualization within a system's framework (Ault and Luo 1998; Upson et al., 1989). The World Wide Web provides the technical infrastructure to manage distributed data resources. When the distributed data sets consist of complex structures and relationships, the inclusion of domain-related concepts provide a unified substrate for information sharing. This semantic model of the data resources provides a controlled terminology for flexible access to the different data resources by users. Often the semantic meta-information forms a hierarchical taxonomy of concepts in the domain of discourse. The conceptual architecture of the prototype framework is modular. It can be thought of as a set of interdependent processes, analogous to Brodlie's notion of the visualization pipeline (see Brodlie, this volume (Chapter 23)). Thus, the framework encompasses data management, statistical, modelling and visualization capabilities. The visualization component of the framework provides tools to enable the researcher to extract required subsets of the problem data space and graphical capabilities to visualize the characteristics and relationships underlying observational data resources and the processes they represent. The framework is heavily dependent on a flexible set of visualization tools to make the "unseeable" visible and provide insights that might otherwise be hidden in the vast volumes of data. This chapter is organized as follows. The characteristics and requirements for sharing are considered in w whilst w describes classifications representing semantic knowledge. Conceptual models of the architecture are presented in w with illustrative examples and the chapter concludes in w with a discussion of the issues raised.
A Knowledge-based Collaborative Environment for Geovisualization
25.2
Rationale and Requirements for Shared Resources
25.2.1
Data and functional resource management
497
Within the GI research community, various groups of users with their own perspectives and temporal horizons are involved in input of data and output of information from the heterogeneous data resources. Information systems for ecosystem management tend to be highly distributed (Mesrobian et al., 1996). The data capture is often the responsibility of different agencies that maintain subsets of the information, which need to be integrated for modelling purposes. These users contribute different data sets to the information base, which, in its entirety, enables short- and long-term investigation of complex ecosystems. Thus, sharing, integration and interoperability of diverse data sets is a major requirement; the diversity may consist of differing resolutions, formats as well as sampling frequencies that may have to be reconciled (Varma, 2000). The structural complexity of the application domain makes demands on the functional and analytical capabilities of the system as well, because transparent mapping between diverse data formats must be provided by the system. Many GI researchers have focused on "geovisualization to mediate and enhance collaborative knowledge construction among environmental scientists" (Brewer et al., 2000). In this chapter, the focus is on a data-centric view of resource sharing: providing tools to enable navigation through and visualization of a multi-dimensional problem space to provide decision support (Woodruff et al., 1995). Another major requirement is for visualization of the statistical properties of aspatial attributes in the context of their spatio-temporal dispositions. This requirement to enhance a GIS with spatial statistical capabilities has been noted by several researchers (Wise et al., 2001; Cook et al., 1996). Theus, this volume (Chapter 6), makes a cogent case for the integration of statistical tools to enable interactive exploration of aspatial attributes in the problem space. Additional functional capabilities for modelling and simulation also have to be included in the geovisualization infrastructure. For example, attributes sampled as point data have to be interpolated over the field of interest. The prediction of biomass stocks and sustainable levels of relevant species are based on complex interconnected calculations. The collaborative geovisualization framework therefore requires integration of structural and functional capabilities (Gardels 2000; Ungerer and Goodchild, 2002).
25.2.2
Visualization of multiple dimensions and hierarchies
Cognitive studies in the geographic domain have shown that one of the core aspects of geo-analyses is the identification and separation of the where, when and what aspects (Mennis et al., 2000). Most geographic databases contain variables that address the spatial and temporal dimensions and a number of thematic or scientific dimensions as well. Thus, the very basic requirement of a spatial analysis geovisualization system is the incorporation of these dimensions in the visualization interface. In the environmental domain this is even more critical. As noted by many ecologists and oceanographers (Dickey, 1992), phenomena of interest occur in different space-time frames. For example, water temperature may be recorded using a CDT logger at various sample
498
Z. Kemp
points. If a study focuses on the diurnal and nocturnal movements of planktonic biomass, the temperature data would be aggregated into daily slices whereas if the requirement were to consider seasonal trends, then the temperature values would be considered at a different dimension of the space-time composite. In a shared, distributed geovisualization environment, the same data will be visualized at different spatial and temporal scales depending on the requirements of the particular problem being analysed. This implies that the geovisualization interface has to provide functionality for identifying and extracting the required multi-dimensional subset of the geo-space, at the requisite scale and resolution. There are some geovisualization tasks where the temporal dimension is dominant. Developments in telecommunications technology have filtered through geographic applications where devices such as GPS receivers are used to track the objects in space (see Coors et al., this volume (Chapter 27)). Mountain, this volume (Chapter 9) describes a system that enables the capture of spatio-temporal histories using location point data and using the observed histories to explore patterns of movement over space and time. In the marine domain, examples of this type of application are vessel navigation systems or electronic chart display systems (ECDIS). The requirements here are for real-time display of the object itself augmented by symbols or references to relevant objects along the trajectory or path of the vessel. The geovisualization requirement is for clear interactive display of movement with fast response showing potential hazards. A similar application involves tracking vessel movements using GPS receivers and satellite technology to relay the movements of many vessels in a spatial extent. Here the space-time history is linked to a dedicated rule base to enable identification of vessels that may be flouting regulations such as "closed" fishing areas or "closed" seasons. The geovisualization here is required for real-time monitoring of activities and requires the display of aspatial attributes such as identity, size, country of registration of the vessel. Although here too, the response time of the geovisualization is important, it is not as critical as in the navigation example. In environments where visual exploration of data is linked to scientific knowledge exploration, such as tracking and quantifying changes in biodiversity, the visualization requirements are different. In such applications, response time is not a major issue. What is relevant is the ability to link heterogeneous data sets from disparate sources, apply appropriate transformations and use the visualization for insights into the causes of change over space and time. An increasingly widespread and popular way of achieving interoperability in such a collaborative environment is to use the contextual or shared domain knowledge that exists in a research community. Ontologies have been included in the experimental prototype marine biodiversity GIS as described in w At this stage it would be appropriate to reflect on some aspects of support that the computational infrastructure should provide for geovisualization in a shared environment. The myriad contexts in which visualization is used make it difficult to identify specific themes and issues as core requirements for collaborative geovisualization. A comprehensive discussion is presented in Fairbairn et al. (2001). However, it is possible to identify the following capabilities that should be made available, from the dual perspectives of support to be provided by the underlying spatio-temporal
A Knowledge-based Collaborative Environment for Geovisualization
499
information system and the functionality to be provided to the user controlling the visualization. 9
9 9 9
9
25.3
A core requirement is a viable data model that captures the phenomena being studied. Capabilities within the interface should enable the user to identify phenomena of interest and the system should be able to interpret the required concept using the underlying data sets. The interface should also enable the user to specify the "levels of detail" for each dimension or attribute contributing to the visualization. If additional exploratory statistical properties of the variables being visualized are required, the user should be able to request these. The interface should be able to hide the physical location of the data and computational resources from the users so that, in some sense, they are seeing a "virtual" data and computational tools resource. The actual style and appearance of the visualization is much more problematic and dependent on many criteria such as personal preferences of the user, domain-related conventions, network speed, response time required, display device, level of interaction required and so on. However, despite these imponderables, it should be possible to guide the user towards an "appropriate" representation based on an analysis of the task using a combination of the dimensions involved, the type of data to be visualized and the technical resources available.
Ontology-Driven Geovisualization
25.3.1 Ontologies for knowledge discovery There has been much research carried out in the specification of ontologies and their use in formalizations such as description logics and reasoners (Borgida, 1995). An introduction to the related terms, ontologies, semantic knowledge and the semantic Web are presented in the explanatory key topics in the introductory chapter to this section (see Brodlie et al., this volume (Chapter 21)). Ontological specifications in an information system capture salient concepts and their relations within a domain of discourse. Ontologies have been used for classification, information integration, modelling systems, computer reasoning and datamining. Several notations and tools have been developed for semi-automated generation of ontologies and reasoning with them. The semantic information encapsulated in the ontologies is usually based on a widely accepted corpus of knowledge, concepts and beliefs within a community to which the term ontology is frequently applied (Guarino, 1998). For collaborative geovisualization it is necessary not just to be able to access and transfer actual observational data sets but to convey the underlying domain context as well, as noted by Chen et al. (1997); Fonseca and Egenhofer (1999); Kavouras and Kokla (2002). This notion is of particular relevance when federating data and computational resources to support the analytical processes required in Web-based geovisualizations. Ontologies can be used for several purposes: context definition and interchange, querying and
500
Z. Kemp
retrieval, mediation between multiple ontological specifications and resolution of conflicts should they arise. Ontologies in our prototype are used to address most of these issues and are therefore an integral component of the geovisualization architecture. In many fields, an overarching ontology can be used to articulate the domain semantics. This is not the case in GIS. GIS data sets are inherently multi-dimensional; the dimensions of space and time will have their own separate ontological structures. Moreover, the data will invariably refer to one or more scientific dimensions with their own ontological specifications that need to be reflected in the knowledge base. This is of particular relevance in the environmental domain as the specification of varying combinations of where, when and what drive the geovisualizations that support decision-making. Support for multiple ontological specifications for a particular dimension may also be required. A detailed exposition of the use of ontologies with reference to knowledge discovery is beyond the scope of this chapter, but we include illustrative examples of space, time and taxonomic ontologies that support collaborative geovisualizations in w We present a few observations here to explain the principles of ontology design in the prototype geovisualization system: 1. 2.
3.
4.
25.3.2
The ontologies are dimension specific and hierarchical so that multi-variate data can be visualized at the appropriate scale (Kemp and Lee, 2000). There may be more than one ontology for a particular dimension thus encompassing multiple user perspectives for each dimension. For example, a particular spatial analysis task may require visualization of a particular habitat type alongside a map of the same spatial extent tiled into regions that may be "fiat" objects (Smith and Mark, 1998) such as fishery control zones. Thus, geographical space may require definition by the spatial concept habitat_type as well as by a region defined in vector format using an identifier. The terms and concepts in the ontology can be used from the interface to drive the geovisualization. From the geovisualizer' s perspective, the use of ontologies makes the analysis more task-oriented by enabling the user community to use domain specific terms and concepts that are common currency within it. Ontologies are mapped into appropriate data structures within the system that provides the functionality to traverse the levels in a hierarchy, to aggregate/ disaggregate the data appropriately, or apply functions to materialize the data required for the visualization.
An illustrative ontology for the spatial dimension
In the context of geospaces in marine research, the recent EU initiative on habitat classification (Eunis, 1999) consists of a hierarchy of space-related descriptors, as does the related US effort detailed in the NOAA Technical Memorandum NMFS-F/SPO-43, July 2000. These classifications can be thought of as conceptualizations of space that are used and accepted within that research community. These spatial concepts can be integrated into the visualization interface to reflect the users' collective understanding of
A Knowledge-based Collaborative Environment for Geovisualization
501
Figure 25.1. Diagrammatic representation of marine habitats (Eunis, 1999).
space and to facilitate the analyses that may be performed on the underlying data sets. Figure 25.1 illustrates the conceptual underpinnings of this ontology. The definition or specification of space here is expressed in terms of variables such as distance from the shore, the depth of the sea bottom, the topography of the sea floor and the structure of the substrate. The concepts or descriptors form a hierarchy that represents finer classifications (analogous to spatial resolution) of space.
A MARINE---.1 HABITATS
stratum I (17)
bed
Permanently ~ water-covered? (18)
hard or non-mobile | A1 substrates ~ -- Littoralrock and ~ |other hard substrata J Substrate I ~
No
"1 (19/
Yes
soft or sediments
Littoral sediments
er column
I A7 I Pelagic water column
No Yes
hard or non-mobile ~
substrates
A3 Sublittoral rock and
other hard substrata
I Slope and rise? I I
y/
(22)
"~
Bathyal zone
Figure 25.2. Criteria for marine habitats to level 2.
soft or mobile sediments
A6 Abyssal zone
A4 Sublittoral sediments
502
Z. Kemp
Figure 25.2, also from the EUNIS habitat specification represents a rule-based tree structure defining how the spatial concepts in the ontology are evaluated in terms of the physical parameters that exist in the underlying database. In this hierarchical habitat classification, the uppermost concept, marine habitats, subsumes concepts at lower abstraction levels. This tree-structured hierarchy can be integrated with the observational data sets by using a similar hierarchical data structure in the data server. Evaluating the habitat types requires computational support: the functional evaluation includes capturing the criteria for each habitat type and evaluating them against the appropriate parameters in the observational data sets to arrive at the appropriate habitat classification. In the prototype environmental system, the observational data sets include a range of physical and biological variables and plankton abundances by species at selected sampling points. The habitat classification is used alongside the criteria values in the observational data to identify the spatial-habitat type applicable to each sampling station on the fly. It is also possible to use the observational physicochemical data to generate habitat maps of the spatial extent of interest and use these stored maps as representative of spatial tilings by habitat type. The former method affects response time in an interactive environment, but can be applied to all currently available data sets when the visualization is being generated. The latter method may not use the most recent data, but is faster to extract and distribute if response time is critical. As noted earlier, in some cases, multiple overlapping classifications may be required for analysis. This is particularly true if data from heterogeneous data sets are being integrated, such as in a federated marine environmental system, which may consist of data resources linked over the internet. In these cases, each data resource exports its data schema including its ontologies or semantic metadata to a global catalogue using a Web-based data exchange standard such as XML. Consider, for example, environmental monitoring authorities wishing to evaluate the effects of fishing activities on sensitive marine habitats. The fish catch data will be classified by ICES divisions and statistical rectangles (ICES) whereas the biodiversity data will be classified by habitat types. The catch data and the habitat types can be compared by using the appropriate semantic classification and visualizing the maps in close proximity.
25.3.3
Domain related temporal ontology
Unlike space, time is usually either linked to spatial entities, defining a space-time object, or to an aspatial entity setting up a time-series. Since time is closely linked with other dimensions, the ontology representing time is used to stress the underlying environmental processes to support knowledge discovery. There may be several temporal topologies relevant to a Web GIS; depending on the analysis, certain periodicities are more relevant than others. A standard one may be a temporal interval of a calendar year based on the Gregorian calendar. Marine environmental systems are employed to capture temporal spaces covering periodic events, for example, short-time cycles such as tides to annual events. A spawning stage as illustrated in Figure 25.3 is a relatively short time span in spring. If the analysis involves extracting spawning stage
A Knowledge-based Collaborative Environment for Geovisualization
503
annual
I
spawning stage
I
I
spring
summer
I
I
I
autumn
winter
I
juvenile
stage
Figure 25.3. A seasonal temporal ontology.
data over a small extent of marine space, such as the English Channel, the concept may be realized by using a simple lookup table to indicate the appropriate dates at that latitude in the Northern hemisphere. If however, the geovisualization compares values over spaces that are located at very different latitudes a function would be required to realize the concept of "spawning stage" applicable to the different spatial extents. The system would use a function or a lookup-table as appropriate to implement a temporal ontology.
25.3.4
Ontologies for thematic dimensions
One of the major themes in environmental informatics is the sharing of data resources for research into biodiversity, as described in Bowker (2000), where global synoptic databases are used and shared for species taxonomies. Our prototype consists of selected taxonomies for marine species and the implementation enables visualization of species data at the appropriate level, e.g., family or genus or selected species. In this case, the classification forms a tree-structured hierarchy where nodes at different levels in the tree are linked to each other by is_a relationships forming a specialization/generalization hierarchy in structural terms. The elements at the lowest level of the hierarchy (species or subspecies) are members of the parent node in the hierarchy. This relationship applies all the way up the classification tree; each node in the hierarchy is a member or instance of its parent node. The classification function representing these is_a relationships is one of subsumption, and is also transitive, so elements of nodes at lower levels of the classification tree are also elements of taxa at higher levels. A consequence of incorporating this classification hierarchy with the data set containing species abundance data is that the biological data can be retrieved at the level aggregation appropriate to the problem being solved. An important aspect of the way we have implemented ontologies is that they can be conceptualized as semantic knowledge objects and are held alongside the observational database or in the global schema that has links to the associated database. A consequence of this design decision is that the geovisualization interface enables the user to express data subsetting requirements in terms of conventional spacetime parameters (for example latitude/longitude and a pair of timestamps) or using
504
Z. Kemp
ontological concepts such as marine habitat type bathyal_zone. The functionality provided with the ontologies makes use of domain-relevant classifications thus enabling the visualization to present data at the appropriate level of abstraction.
25.4
Conceptual Architecture of the Geovisualization Framework
25.4.1
Overview
The ideas discussed so far have been explored in the prototype implementation referred to in the introduction. The details of the implementation are beyond the scope of this chapter; here we describe, briefly, the layered architecture of the system to identify the different components. The modular structure of the conceptual architecture of the framework enables the system to be flexible and expandable. The data server manages the observational data sets, derived data that may be generated using techniques such as interpolation and the semantic classifications. The data resources and the knowledge base may be stored in an object-relational or a relational database. However, the hierarchical structures of the ontologies and the associated transformation mappings are easier to represent in an object-oriented software substrate. The knowledge base, implemented as knowledge objects with complex structures, establishes mappings between the high-level conceptual abstractions encapsulated in the knowledge objects and the data objects in the observational data sets. It is created and maintained by the ontology manager component of the data server. The computational server provides capabilities for hydrographic modelling, statistics, and retrieval of the required subsets from the multi-dimensional problem space. For example, additional computational functionality may be included as required to implement statistical, mathematical or process models (Tyler, 2000) that generate or summarize the visualization over space. Information about thematic variables can thus be augmented by the inclusion of graphs and statistics. Moreover, the ability to algorithmically specify how additional variables may be derived extends the capability of the observational data sets. The interactive user-level interface is mediated via the knowledge base to enable the user to be aware of, and use the space-time-scientific attributes context of the data resource. The interface is flexible so that users may identify the spatial extent of relevance to a problem using georeferenced parameters or habitat classifications. This flexibility extends to the temporal and thematic dimensions as well. The output from the query interface is routed through a visualization engine to select appropriate display methods for different styles of analysis/query. The WWW server component is responsible for converting the local ontology to the global marine ontology using knowledge exchange standards. For example, the semantic knowledge base as well as the metadata for the schema could be exported using a meta-language such as the eXtensible Markup Language (XML) standard proposed by the World Wide Web Consortium (W3C). There are several possible approaches to integrating disparate local ontologies (Wache et al., 2001)
A Knowledge-based Collaborative Environment for Geovisualization
505
Application Interface
Geovisualization Server
4, Space-time-thematic parameters Parameters to specify what is to be visualized
Application domain models (derived geodata)
Graphing
Selection or input of formulae (if required)
Parameters
to
specify how data are visualized
Visualization style specification (icons, colours, 2d/3d ) ortask-based visualization engine
Figure 25.4. Conceptual hierarchy of the visualization interface.
25.4.2
The visualization interface
Geovisualization can be considered from two major perspectives: what is to be viewed and how it is to be visualized. The interface of the component-based framework (Figure 25.4) enables users to specify parameters to control the content and style of the cartographic visualization. We illustrate the flexibility that enables users to determine what is to be viewed using the example in Figure 25.5. The core of the interface for the fisheries management system prompts the user for parameters to determine the spatial extent and grid size, the time span (not illustrated in the figure), the thematic dimension (fish species in this case) and the value to be calculated and displayed in the gridded cartographic display. In the example, instead of the total catch being displayed, a normalized value, catch per unit effort (CPUE), is being calculated and the user has flexibility in defining the formula to calculate this; either a predefined formula is selected or the user can provide a customized algorithm. The benefit of a space-time-theme core with options enables this geovisualization interface and associated graphics library to be used by several user groups in the industry to analyse harvest data; skippers at the vessel level, regional and national authorities at the appropriate spatial resolution and at a Europe wide level (Kemp and Meaden, 2001).
506
Z. Kemp
The interface enables the user to switch between conventional space-time parameters and domain dependent classification. For example, the interface illustrated assumes the user requires the values to be visualized within statistical rectangles. If however, spatial subdivision is by fishery ICES areas (divisions for monitoring fishery activity in the North Sea and the Atlantic) then an appropriate classification is used (ICES). The structure of the framework can also be used to provide a flexible interface for how the geovisualizations are presented to the user. Effective visualization depends on a variety of factors such as the colours, design, placement and cartographic iconography used. There have been many formalisms proposed for cartographic visualization including those by Bertin (1983), Tufte (1990) and Buttenfield (1996). In scientific analyses, change or evolution of variables is frequently a focus of study. There is much discussion in the cartographic literature on the efficacy of dynamic visualization in displaying time-variant data (Andrienko et al., 2000; Kraak and MacEachren, 1994). In the context of temporal change, our experience is that movement of objects are obvious contenders for dynamic output. On the other hand, analysis of change in an aspatial attribute over time requires a different type of dynamic property. Here, scientists need to view the two (or more) states concurrently; effective highlighting of change over time is presented by animating the change in the icon representing the attribute being visualized and enabling users to explore values of associated variables. Visualization preferences and styles that may be prevalent in a particular user community are an important aspect of the capabilities to be provided. For example, in the marine research community hydrodynamic data is frequently represented using spatial vectors and speed-direction icons. Bathymetric data may be visualized using user-defined grid spacing or contour intervals. The provision of a cartographic display library within the framework enables users to select the most appropriate visualization tools for the problem being considered. The appearance of geovisualizations within specific applications is also conditioned by the context in which the applications are expected to function. For example, in a mobile interactive application such as navigation, the display device is a monitor so that change in the immediate environment is depicted by a dynamic display reflecting the movement of the object through space. We have not experimented with multi-modal interfaces, but the proximity of a hazard such as a sandbank or wreck could be effectively signalled using sound and/or highly dynamic visual icons. On the other hand, if the geovisualization is part of a spatio-temporal decision system then environmental change in thematic or space-time dimensions are envisioned using alternative formalisms. We present a few examples of the visualizations that can be produced by our prototype using an interface similar to the one illustrated above using the marine biodiversity data resources. Figure 25.6a uses the habitat classification to generate the habitat map in contour format for the family of species Gadidae. The sampling locations are indicated on the map using point data. Figure 25.6b and c shows the disposition of observed values of fish larvae abundances of two specific species, whiting and bibwithin that family. The visualization interface enables the user to select the interpolation method to be used; here two options are illustrated, the bibdistribution is interpolated using an inverse distance function and the whiting distribution on a regular grid.
A Knowledge-based Collaborative Environment for Geovisualization
507
Figure 25.5. A typical GUI for multi-dimensional visualization.
Figure 25.6. Prototype visualization interface enabling users to control cartographic parameters and select interpolation method used. (a) habitats as contoured surface; (b) disposition of bib; (c) disposition of whiting.
508
25.4.3
Z. Kemp
Geovisualization for group collaboration
Many general-purpose visualization systems such as AVS (Upson et al., 1989) are predicated on a data flow model that includes filter-map-render operations to abstract required data, map the data into underlying graphics primitives and render the data for visualization. A similar system, Tioga-2 (Aiken et al., 1996) is a direct manipulation browser specifically designed for the exploration of multi-dimensional spatio-temporal data sets. Tioga is a database visualization environment that enables users to specify, graphically, a set of operations (boxes) that identify the data to be visualized, the transformations that may be applied and a viewer that enables selected objects to be placed and rendered on a 2D canvas. Tioga provides three "displayable" types with associated viewing attributes. By using the base types and operations provided, visualizations can be composed in the same or different viewing spaces, and the viewing spaces can be grouped to provide visualizations according to user specifications. Actual visualization of data is controlled by specifying a display attribute, which specifies the visual form of the variable on the display canvas. Sequences of primitive operations can be stored as programmes to enable the visualization pipelines to be reused. In addition, a notion of semantic zooming is provided, which enables the user to control the level of detail that is viewed at any one time (this is analogous to the notion of map layers being activated at different spatial resolutions). Most generic scientific visualization systems attempt to strike a balance between enabling users the flexibility to use their own paradigms for visualization and, at the same time, providing an easy programmable interface to specify the templates for the visualizations. These systems recognize, but do not provide solutions to the performance problem when dealing with large volumes of spatio-temporal data. A similar Query Execution Engine that is part of an Analytical Abstraction Layer for hierarchical reasoning in space and time is described in Lee (2000). Like other generic scientific visualization systems it uses the dataflow paradigm where each query is conceptualized along an execution pipeline. The pipeline supports three kinds of objects: source, transform and output objects. Source objects provide a generic interface to data sources, transform objects are functional objects that perform various analytical tasks, transforming data in one form to another and output objects are responsible for committing data to some persistent source, or a visualization engine. A specific query is constructed by instantiating an execution pipeline template to perform the required tasks. The dataflow model is strongly typed so transform output objects will only accept data with specific formats. This model works well where the requirements for collaboration are confined to sharing data and computational resources. Unfortunately, in application areas such as emergency management where response is very time-dependent additional capabilities are required. A possible solution would be to adapt the query engine model to allow specification of multiple transform and output objects to enable a range of devices to be used. Alternative triples would be determined by including a set of task criteria which consider response time and resources to be catered for. The e-Grid infrastructure is an alternative mechanism that provides high-end distributed computational resources to enable interoperability see Brodlie, this volume
A Knowledge-based Collaborative Environment for Geovisualization
509
(Chapter 21) and Schroeder, this volume (Chapter 24). If collaboration requires users to interact with visualized objects in synchronous or asynchronous mode then an additional layer of software is needed at the interface level. For synchronous mode collaboration, mechanisms are needed for session management as well as shared information spaces. Users can share "whiteboards" or "clipboards", all operations on these may be initiated and visualized by members of the group. An example of interactive collaboration is a visualized road network with participants in the field returning locations of blockages, due to landslides or other events, which are displayed on the communal whiteboard and retrieved in real-time. Other decision makers may display alternative routes for use by emergency services that are also posted and viewed by all participants in the session. In asynchronous mode, only the communal information spaces are visible to the group, but interaction is not provided. In either case, management of distributed collaboration requires a client-server computational model with a custom-built interface that recognizes and responds to interaction events within group sessions.
25.5
Discussion
The design and architecture supporting the geovisualization capabilities, discussed at length in w and w illustrate the objectives noted at the end of w The data model underlying the distributed data resources plays a central role in enabling users to drive the visualizations over the multi-dimensional data space. The space, time and thematic dimensions that are abstracted in the ontologies enable users to specify any combination of parameters over these dimensions. Moreover, the level of detail required can be specified by semantically zooming through levels of the ontologies. In the context of distributed resources, the ontologies serve as a repository of commonly accepted terms within a GI research community and provide interoperable access to separate data sources by mapping the ontological terms to individual database schemas. The visualization toolset provides the user with flexible capabilities to specify the computational and statistical algorithms that may determine the visualization as illustrated in w and Figure 25.6. In conclusion, certain observations may be made about collaborative geovisualization over distributed data resources. Effective geovisualization should be tailored to the requirements of application domains. Within an application domain oriented system, users should be able to make a choice from alternative representational forms. Experience seems to indicate that most users show a preference for conventional static cartographic displays unless the application is particularly linked to real time decision making. The interface should also provide the usual direct display handling mechanisms such as zooming, annotation, querying by interactive selection, etc. The aims of the previous point can be achieved by modular design of the GIS software infrastructure to provide flexibility and expandability. A side effect of this would be that geovisualization would become more closely integrated with spatial decision support and datamining. The inclusion of semantic knowledge within the geodatabase augments the information space and encourages group awareness of the context of the application
510
Z. Kemp
domain. The Internet and the World Wide Web are increasingly being used for data sharing and collaboration so the geovisualization interface should be capable of being extended to function within a WWW server using standard exchange protocols. A taskbased perspective on visualization makes it possible to embed some "intelligence" in the system, which itself chooses an appropriate mechanism for presentation depending on the nature of the tasks and the variables involved. The provision of highly interactive, multimedia group work requires substantial computational resources. In this context Grid based technologies have the potential to provide the required infrastructure (see key topic on Grid computing, Brodlie et al., this volume (Chapter 21)).
Acknowledgements The author gratefully acknowledges the contributions of Robert Frank and the Durrell Institute of Conservation and Ecology, University of Kent, Canterbury Christ Church University College and the Marine Laboratory, Universit6 du Littoral, Calais, members of the INTERREG project "Biodiversity and Cartography of Marine Resources in the Dover Straits", (GOSE reference 97/C8/03).
References Aiken, A., Chen, J., Stonebraker, M., and Woodruff, A., (1996) "Tioga-2: A direct manipulation database visualization environment", Proceedings of the 12th International Conference on Data Engineering. New Orleans, LA: ICDE. Andrienko, N., Andrienko, G., and Gatalsky, P., (2000) "Towards exploratory visualization of spatio-temporal data", Proceedings of the Third Agile Conference on Geographic Information Science, Helsinki. Ault, J. S., and Luo, J., (1998) "Coastal bays to coral reefs: systems use of scientific data visualization in reef fishery management", Proceedings of the ICES Conference. Bertin, J., (1983) Semiology of Graphics: Diagrams, Networks, Maps. Madison, WI: University of Wisconsin Press. Borgida, A., (1995) "Description logics in data management", IEEE Transactions on Knowledge and Data Engineering, 7(5), 671-682. Bowker, G. C., (2000) "Mapping biodiversity", International Journal of Geographic Information Science, pp. 739-754. Brewer, I., MacEachren, A. M., Abdo, H., Gundrum, J., and Otto, G., (2000) "Collaborative geographic visualization: enabling shared understanding of environmental processes", IEEE Information Visualization Symposium, Salt Lake City, Utah, October 9-10, 2000: IEEE, pp. 137-141. Buttenfield, B., (1996) "Scientific visualization for environmental modeling", In: Goodchild, M. F., Steyaert, L. T., Parks, B. O., Johnston, C., Maidment, D., Crane, M., and Glendinning, S., (eds.), GIS and Environmental Modeling: Progress and Research Issues. Fort Collins, USA: GIS World Books, pp. 225-229.
A Knowledge-based Collaborative Environment for Geovisualization
511
Chen, H., Smith, T. R., Larsgaard, M. L., Hill, L., and Ramsey, M., (1997) "A geographic knowledge representation system for multimedia geospatial retrieval and analysis", International Journal of Digital Libraries, 1(2), 132-152. Online: http ://ai.bpa. arizona, edu/-~mramsey/papers/gkrs/gkrs.html Cook, D., Majure, J. J., Symanzik, J., and Cressie, N., (1996) "Dynamic graphics in a GIS: exploring and analyzing multivariate spatial data using linked software", Computational Statistics: Special Issue on Computer aided analysis of spatial data, 11(4), 467-480. Dickey, T., (1992) "Physical-Optical-Biological Scales Relevant to Recruitment in Large Marine Ecosystems", In: Sherman, K., Alexander, L., and Gold, B., (eds.), Large Marine Ecosystems: Patterns, Processes and Yields. Washington DC, USA: AAAS Press. EUNIS: European Nature Information System, (1999) EUNIS Habitat Classification, European Environment Agency. Online: http://www.eunis.eea.eu.int/habitats.jsp Fairbairn, D., Andrienko, G., Andrienko, N., Buziek, G., and Dykes, J. A., (2001) "Representation and its relationship with cartographic visualization: a research agenda", Cartography and Geographic Information Sciences, 28(1), 13-28. Fonseca, F. T., and Egenhofer, M., (1999) "Ontology-driven geographic information systems", In: Medeiros, C. B., (ed.), Proceedings of the Seventh ACM Conference on GIS, pp. 14-19. Online: http://www.spatial.maine.edu/-~max/RC39.html Frank, R., and Kemp, Z., (2001) Integrated spatiotemporal analysis for environmental applications, Innovations in GIS 8: Spatial Information and the Environment. Gahegan, M., Harrower, M., Rhyne, T.-M., and Wachowicz, M., (2001) "The integration of geographic visualization with databases, data mining, knowledge construction and geocomputation", Cartography and Geographic Information Science, 28(1), 29-44. Gardels, K., (2000) The Open GIS Approach to Distributed Geodata and Geoprocessing. Online: http://www.regis.berkeley.edu/gardels/envmodel.html (23/10/03). Guarino, N., (1998) "Formal ontology and information systems", In: Guarino, N., (ed.), Formal Ontology in Information Systems (FOIS '98). Amsterdam: ICES, pp. 3-15. Online: http://www.oceanlaw.net/orgs/maps/ices_map.htm Kavouras, M., and Kokla, M., (2002) "A method for the formalization and integration of geographical categorizations", International Journal of Geographic Information Science, 16(5), 439-453. Kemp, Z., and Lee, H., (2000) "A multidimensional model for exploratory spatiotemporal analysis", In: Abrahart, R. J., and Carlisle, B. H., (eds.), Proceedings of the Fifth International Conference on GeoComputation, University of Greenwich, pp. 23-25. Kemp, Z., and Meaden, G., (2001) "Visualization for fisheries management from a spatiotemporal perspective", ICES Journal of Marine Science, 59, 190-202. Kraak, M. J., and MacEachren, A. M., (1994) "Visualization of the temporal component of spatial data", In: Waugh, T., and Healey, R., (eds.), Proceedings of the Sixth International Symposium on Spatial Data Handling, Advances in GIS Research, Edinburgh, Scotland, pp. 391-409.
512
Z. Kemp
Lee, T. K. H., (2000) A multidimensional analytical framework for hierarchical reasoning in space and time Ph.D. Thesis, Computer Science, University of Kent at Canterbury, UK. (British Library). MacEachren, A. M., Wachowicz, M., Edsall, R., Haug, D., and Masters, R., (1999c) "Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods", International Journal of Geographical Information Science, 13(4), 311-334. Mennis, J., Peuquet, D., and Qian, L., (2000) "A Conceptual framework for incorporating cognitive principles into geographic database representation", International Journal Geographical Information Science, 14(6), 501-520. Mesrobian, E., Muntz, R., Shek, E., Nittel, S., La Rouche, M., Kriguer, M., Mechoso, C., Farrara, J., Stolorz, P., and Nakamura, H., (1996) "Mining geophysical data for knowledge", IEEE Expert, 11(5), 34-44. Smith, B., and Mark, D. M., (1998) "Ontology and geographic kinds", In: Poiker, T. K., and Chrisman, N., (eds.), International Symposium on Spatial Data Handling (SDH '98). Vancouver, Canada: International Geographical Union, pp. 308-318. Tufte, E. R. (1990) Envisioning Information, Cheshire, CT: Graphics Press, 126 pp. Tyler, A., (2000) "The new visual marine information systems (VMIS) framework", Oceanology, 337-347. Ungerer, M. J., and Goodchild, M. F., (2002) "Integrating spatial data analysis and GIS: a new implementation using the component object model (COM)", International Journal of Geographic Information Science, 16(1), 41-53. Upson, C., Faulhaber, T., Kamins, D., Schlegel, D., Laidlaw, D., Vroom, J., Gurwitz, R., and van Dam, A., (1989) "The application visualization system: a computational environment for scientific visualization", IEEE Computer Graphics and Applications, 9(4), 30-42. Varma, H., (2000) "Applying spatiotemporal concepts to correlative data analysis", In: Wright, D., and Bartlett, D., (eds.), Marine and Coastal Geographic Information Systems. London: Taylor and Francis, pp. 75-94. Wache, H., Vogele, T., Visser, U., Stuckenschmidt, H., Schuster, G., Neumann, H., and Hubner, S., (2001) "Ontology-based integration of information - a survey of existing approaches", Proceedings of IJCAI-O1 Workshop: Ontologies and Information Sharing, Seattle, WA, pp. 108-117. Wachowicz, M., (2001) "GeoInsight: an approach for developing a knowledge construction process based on the integration of GVis and KDD methods", In: Miller, H. J., and Han, J., (eds.), Geographic Data Mining and Knowledge Discovery. London: Taylor and Francis, pp. 239-259. Woodruff, A., Su, A., Stonebraker, M., Paxson, C., Chen, J., and Aiken, A., (1995) "Navigation and coordination primitives for multidimensional visual browsers", In: Spaccapetra, S., and Jain, R., (eds.), Proceedings of the Third IFIP 2.6 Working Conference on Visual Database Systems. Lausanne, Switzerland: Chapman and Hall, pp. 36-371.
ExploringGeovisualization J. Dykes, A.M. MacEachren,M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 26
Geovisualization Issues in Public Transport Applications David Fairbairn, School of Civil Engineering and GeoSciences, University of Newcastle Upon Tyne, Newcastle Upon Tyne NE1 7RU, UK
Keywords: public transport, journey planning, geovisualization, cartographic representation, transport information systems, transport maps Abstract This chapter is intended to highlight the role of contemporary research in geographic visualization (geovisualization) and its applications to the specific field of public transport information systems. With a significant need for efficient information handling by and for a range of participants and stakeholders, public transportation operations can exemplify the variety of potential contributions by geovisualization researchers to everyday scenarios. A number of problems and issues in the representation of spatial data in transport information systems are outlined. Some of the conclusions from the recently completed Research Agenda in geovisualization (MacEachren and Kraak, 2001) are then examined to determine the extent to which these problems can be resolved and these issues can be tackled. Particular emphasis is given to reconciling the specific data handling requirements of a bus information system with the outcomes of geovisualization research in areas of representation (how displays in the information system might appear), data-, user- and task-dependence (how visualization is affected by externalities), and the impact of modem developments (how the means of information handling may be changed by improvement in technology). The primary goal of this chapter is to examine the nature of geovisualization and its application in public transport information systems. After an examination of data handling in such systems, a number of specific issues in contemporary geovisualization research are discussed and an assessment is made of their impact on the presentation of spatial information for all stakeholders in public transport operations.
26.1
Public Transport Information
Imagine a public transport vehicle, primarily a bus, but possibly also a train, ferry or localized airplane service. In most cases, such vehicles operate to a pre-determined 513
514
D. Fairbairn
timetable and route and their schedules and resource usage are fairly predictable day-today. Notwithstanding such regularity, there is still an enormous amount of primary and derived data that can be associated with the operation of such a service. The collection, use, archiving and presentation of such data are areas of recent interest (Infopolis2 Consortium, undated). It is suggested in this chapter that current research in geovisualization can assist in the data handling inherent in a public transport information system. The category "public transport information systems" is often taken to apply to all information systems associated with travel, including booking systems, on-line timetable enquiries and route-planning software. Examples of each can be given, showing how they address the information needs of passengers before and whilst journeys are being undertaken, and also how they consider the possibilities of data collection and use by drivers and operators, along with extensions into service querying and system analysis. For example, the inventory developed by the Infopolis project exemplifies passenger systems (Infopolis 2 Consortium, undated) whilst Blessington and Tarrant (1995) discuss the engineering of the Romanse system that provides real-time information to operators. It is also clear that much of the data handled in such circumstances, despite having spatial dimensions, is rendered in non-spatial ways (e.g., tabulated timetable information presents names of locations and times of services, but does not represent them in their spatial position). There is considerable scope, therefore, in examining the problems in, and solutions to, transport information handling in a wide range of scenarios employing a variety of data.
26.1.1
Data for public transport information systems
Data is (or could be) collected by bus operators and other stakeholders in public transport (such as local government, consumer groups etc ) from a diversity of sources. The basis of much of this information is an automated vehicle location system, invariably a GPS receiver, carried on each vehicle. The spatial dimension of this data is important, and data can be held, integrated with other information (such as temporal and service provision data) and presented to a range of people, from the transport company Managing Director to the passengers, in a variety of ways. Examples of the data handled are given later in this chapter, in the context of the implications for geovisualization research of such data handling. In addition to such real-time data collection and presentation, there are many uses and users of assembled information about routes, locations and public transport status, which have used mapping for data representation, archive and query. Such data handling will also be considered. Further, an obvious characteristic of public transport information systems is the variable location of the components involved: the monitoring of vehicles may take place in a centralized control station, whilst the data gathering undertaken by the driver/pilot is inevitably undertaken en route; passengers, both actual and potential, are likely to travel across the geographical area of operation, but the transport planner will be in a small team in one office location. Distributed and mobile technologies inevitably impact on the processing of transport data by these varying "players" performing their varying tasks: these will also be addressed.
Geovisualization Issues in Public Transport Applications
26.1.2
515
Data handling requirements in public transport information systems
Public transport information handling can be undertaken at various stages of a journey (home, at interchanges, en-route), by various individuals (passengers, drivers, managers), at different levels of involvement (customer, company accountant, government planner), at different scales (neighbourhood, specific route, entire network) and through varying media. Taking account of such varying scenarios, Franzen (1999) identifies five functional levels exemplifying data handling in the realm of public transport: 1. 2. 3.
4. 5.
the accessibility level addresses data requirements for those planning transport systems (politicians, administrators); the travel level involves preparatory route-planning by potential travellers; the transport level involves real-time data handling, usually by passengers who are checking routes, being customers and requiring current schedule information; the traffic level concerns the operational issues connected with network management, maintenance and control; the motion level is connected with the individual drivers and their vehicles.
These levels are defined by varying criteria - data, users, tasks - but they can each involve the use of spatial data: there is scope to examine the data requirements of each level, to highlight the tasks and the user requirements for spatial data within the level, and also to speculate on the application of new representations for each of these functional levels. Whilst both traditional mapping and analytical GIS tools have been used to address Levels 1, 4 and 5, most developmental effort, from a cartographic viewpoint, has been directed towards Levels 2 and 3. The need to convey spatial data related to the transport network to the potential or actual traveller is perceived to be immensely important: attracting customers and easing their journey is a prime objective of public transport systems. The possibilities and implications of incorporating new representations into these two levels are addressed in this chapter along with the associated data handling issues. In w some of the research issues that have informed these new representations are introduced.
26.2
Contemporary Geovisualization Research for Public Transportation Needs: Background
A recently published research agenda in geovisualization (MacEachren and Kraak, 2001) forms the framework within which this study of public transport information needs has been undertaken. The overall aims of the research agenda are to develop theory to facilitate knowledge construction and to develop visual tools to support the theory and allow the "searching for unknowns" in spatial data sets. To contribute to achieving these aims, the agenda recognizes four main themes of interest. For each, a series of objectives (challenges) are presented. The theme most appropriate to Levels 2 and 3 highlighted above is representation (Fairbairn et al., 2001), which impacts directly on the design
516
D. Fairbairn
and implementation of useful public transport information systems. The specific challenges under this theme are: 9 9 9 9 9
to to to to to
revise existing cartographic theory; develop new representation methods; consider representations in the light of data requirements; consider representations in the light of task requirements; beneficially apply technological advances in both hardware and software.
Further reports within the research agenda addressing spatial databases cover the themes of interfaces, cognition and usability and knowledge discovery. The challenges facing interface designers include multi-modal methods for access to and interaction with information, work on small mobile displays and customization of interface design to prompt creative thinking. Each of these can have impacts on the design of public transport information systems. Related to them are objectives focused on usability (and cognitive issues that underlie usability), investigating how the representations and interfaces work and determine the contexts within which geovisualization is most successful. Knowledge discovery research has objectives of examining visual approaches to datamining and deriving "added-value" from spatial databases, and more theoretical issues related to, amongst others, database handling, tool construction for accessing databases and the nature of spatial data itself. Two challenges highlighted in these further reports - user requirements and interface design, which are closely related to representation - are addressed at the end of w Finally, there are general challenges (briefly considered towards the end of this chapter) affecting visualization as a whole (i.e., across more than one of the four themes raised individually above), some of which pertain to the role of the user or viewer in public transport information systems: 9 9 9 9
to develop experiential technologies; to understand the human dimension of visualization; to incorporate notions of size and complexity of contemporary geospatial data sets; to address the implications of mobile and distributed architectures for accessing and using spatial data.
The research agenda also addressed issues related to developing geovisualization for group working: these are possibly of less relevance in handling public transport information, although Garbis (2002) offers an indication of the role of collaborative activities in information handling.
26.3
Contemporary Geovisualization Research for Public Transportation Needs: Synthesis
The research challenges, although deliberately general in nature, have, to varying degrees, applicability to information systems for transportation. First, the five specific challenges related to representation are addressed.
Geovisualization Issues in Public Transport Applications
26.3.1
517
Portraying spatial data
Existing cartographic theory would suggest that a map, the iconic representation of space, can traditionally portray spatial data graphically in a static, 2D manner. In order to successfully achieve this, data is modelled and abstracted and a rendering of the data is designed. Contemporary forms of spatial data possess dynamism and multi-dimensionality, but there is still the need for modelling and abstraction, particularly to translate the data into models suited to digital and cartographic representations that lead to effective visualization. From a public transport information perspective, we need to be able to successfully abstract the full complexity of a "real-world" public transport system, with its operators, passengers, routes, vehicles, drivers, fixed facilities (such as stops, interchange stations, depots), schedules, financial management, resource usage (including fuel), efficiency monitoring and relationships with the rest of the world. Figure 26.1 illustrates the requirements for representing a small sub-set of such variables (in this case, the physical street layout, the route plan and the terminal details) in the suburbs of a British city. This information-rich graphic is efficiently designed for passenger use, although due to the omission of a large number of operational variables it is clearly not appropriate for management functions. For passengers waiting at bus stops, a prime information requirement is how soon the next bus will appear - at the transport level (Level 3) the data to be handled is inherently temporal. In many cases, representations of such temporal data will not embody any spatial dimension (a display indicating the period until the next bus arrives is likely to be textual). However, the major factor in determining arrival time of the next bus is its position relative to the passenger waiting at the bus stop, and there are possibilities of displaying current position of the next bus at each bus stop. There is a longer-term need to address "process", as well as temporality. For an urban bus operator, for example, an assessment of delays and their causes may need to be undertaken and represented. Further, the analysis of such delays using spatial representations needs to be done effectively so that we can investigate how patterns and processes are scale-dependent. Thus, representing delays and re-routing for temporary (say, one week) road works, may have different requirements to representing shorter term, but more widespread causes of delay such as regional weather events.
26.3.2
New methods of representation
Level 3 of the functional classification covers en-route transport needs, both at bus stops and on-board. Any on-board information component in a bus information system has generally relied on schematic route diagrams. The design of these is governed by the nature of the route (even with circuitous routes, the linear nature of bus travel lends itself well to simplistic diagrams) and by the space available (schematic route diagrams can fit into more restricted on-board display spaces than geographical maps). The research agenda asks: what is the effectiveness of "non-conventional graphics ... superimposition ... multiple views ... morphing ... schematics?" (Fairbairn et al., 2001, p.21). Here we concentrate on the latter: such diagrams are widely used, on-board and also at bus stops. It is recommended
518
D. Fairbairn
that separate diagrams for each service should be displayed and should "include well-known landmarks (e.g., railway station) as well as street names" (Centro, 2001). There is a need to design schematic diagrams to ensure speedy assimilation of information: "bus users do not expect to struggle to understand information at bus stops. Most people are prepared to look at it for about 8 sec. If after this length of time, they have not been able to interpret some useful information, they are very likely to give up trying" (Centro, 2001). To date, there has been little other guidance for cartographic designers on the creation of schematics, beyond the advice to reflect "real-world" locations of landmarks and street names. The recent interest in disseminating schematic route diagrams on the small screens of mobile devices, however, has led to firmer guidance on the construction of such representations; (see Coors et al., this volume (Chapter 27)) and Agrawala and Stolte (2001). Figure 26.2 effectively shows schematic bus route maps with varying levels of fidelity to topographic reality and reflection of real-world landmarks. For a relatively simple, mono-directional bus route (Figure 26.2a), directions and locations can be approximated, such that information content is high, but the other schematic (Figure 26.2b) is considerably less information-rich, with fewer off-route locational details and a topographically distorted circular itinerary. Schematic diagrams have been used to represent spatial data for hundreds of years, but their use is not restricted to Cartography. From the abstractions introduced by gaming metaphors (portraying chess matches graphically relies on schematized representations) to the presentation of precise diagrams in the printed circuit board design process (Prasad, 1997), schematic representations have characteristics and potential which cartographers need to examine more fully; (see Rodgers, this volume (Chapter 7)). Back on board the bus, non-regular passengers have a prime concern about not missing their stop, or ensuring that they are at the correct stop for their eventual destination. Some way needs to be found to match places of interest and likely destinations to the schematic diagram or other on-board information service. A solution common in tourist mapping is to show schematic landmark representations (Deakin, 1996). The information-rich diagrams shown in Figure 26.2a can promote confidence in unfamiliar neighbourhoods, but the most effective method may be by using some form of audio cue (perhaps triggered by the bus approaching a stop). Information about the name or location of the forthcoming stop is often given orally by the driver or is prompted automatically. Such audio cues can enhance the geovisualization display onboard (Denmark, 2000). Following the conceptual exploration of the linkage between sound and visualization by Krygier (1994), a small number of practical systems for assisting visually handicapped travellers have been developed (Parkes, 1988; Strothotte et al., 1996) but the potential of sound is virtually unexplored as a parameter in geovisualization. Contemporary changes in representation including the introduction of dynamism, animation, interactivity and hyper-linking have led to new methods of display. For example, Level 2 of the transport information system (the travel component, which deals with planning) should be able to cope with dynamism and animation - reflecting the temporal nature of bus data; should cope with interactivity - as passengers select or
Geovisualization Issues in Public Transport Applications
519
Figure 26.1. Extractfrom Milton Keynes suburban bus map, showing the bus routes, interchange plans and frequency information along with accurate "real-world" topographic representation of streets, physical features and administrative units. (Copyright Milton Keynes Council.)
enquire about personalized routes; and should handle hyper-linking - as prospective travellers move, for example, from map to timetable enquiry to external tourist information source to ticket purchase form. In all cases Web-based representations are the obvious means of data representation and dissemination. The Web-based metaphor for interacting
Figure 26.2. Schematic representations of varying complexity for bus routes in Paris. (Copyright RATP.)
520
D. Fairbaim
with information is replicated on public interactive terminals (PITs), which can incorporate animation and interactivity and display hot-links (Figure 26.3). In addition, existing and anticipated portable and mobile communications, such as third-generation (3G) mobile phones and personal digital assistants equipped to receive wireless data downloads can represent and handle information in this way; (see Coors et al., this volume (Chapter 27)). More investigative tasks may be required if the enquirer requires interactive tools for exploration and/or controls for playing through multiple views, allowing efficient data extraction and understanding. Perhaps, multiple routes are possible, and route planning may involve following through varying schedules of sequential journeys. Such "analytical animation" may involve the real-time "questioning" of one of a number of buses on a dynamic representation or determining the nearest "dial-a-ride" resource. Even where animated displays are merely spooled to travellers, these requirements for interactivity and customized information search should lead us to move beyond the video-player metaphor for interacting with animations to more responsive displays capable of being queried (Andrienko et al., 2000). In considering novel methods of geovisualization for public transport information systems, cartographers should not create striking and awe-inspiring graphics solely because they can. The intention is to move from visualizations "which give rise to wonder
Figure 26.3. An example Public Information Terminal (PIT), Madrid, summarizing the solution to a route enquiry and embodying hyperlinks (Iogos to the right) to further maps of the areas around the start and destination.
Geovisualization Issues in Public Transport Applications
521
... towards those which achieve new insights and support critical inquiry" (Fairbairn et al., 2001, p.15) and aid the journey planning process by ensuring that effectively designed representations are also capable of yielding added-value and analysis.
26.3.3
Data requirements for visualizing public transport information
It is implied in this chapter that the bus is a mobile laboratory. An enormous range of data can be obtained directly or as added-value from the operation of bus services. Such data is of use at all functional levels of the public information system, from the initial planning process (Level 1) to the control of the bus by the driver (Level 5). It may have differing type (e.g., multi-variate data on bus operation, spatio-temporal timetable data, uncertain data on passenger number predictions); differing properties (e.g., geographical scale, coverage, resolution and attributes); differing methods of collection and storage (e.g., by passenger survey, location by GPS) and other variable characteristics, such as quality or "fuzziness". Bus statistics, internally sourced (such as fuel consumption, speed and passenger numbers), derived (such as revenue), and externally sensed (such as congestion and traffic density) are multi-variate and spatio-temporal. They are less likely to exhibit uncertainty, although they may have variable quality. They may have different scales and resolution, notably in a temporal sense (daily, monthly, annual data) and will be collected in different ways: by questionnaires, ticket machines and driver data loggers for Level 4 functions; by census and demographic means for Level 1 functions; by GPS receivers, odometers and clocks for Level 3 functions; and by bus engine sensors for Level 5 functions. The representation of data of variable scale may need particular attention. The dynamic nature of route planning for motorists, with a seamless link from street level for fine navigation to small-scale nationwide overview of the route, may need to be replicated for passenger transport users. The local bus user may want to link to national bus services and require a full-journey visualization. In addition to scale dependence, geovisualization must address temporal variation. Traffic flow and public transport operations are inherently dynamic. Responses to variations in traffic conditions will come from passengers who wish to avoid notified delays and operators and drivers who are responsible for ensuring adherence to routes and timetables. Real-time information about road conditions is already available in many urban areas through traffic cameras positioned at major junctions: such scenes can be viewed over the Web in some cases, for example, for the German city of Dresden (intermobil Region Dresden, 2003). Creating real-time or summary maps from these images needs to be done to efficiently convey traffic data to a range of stakeholders in public transport. An example of such real-time mapping is available for Paris (Direction R6gionale de l'l~quipement Ile-de-France, 2002). Given the "time-stamped" nature of much transport data, there are clearly priorities in ensuring that data models, interfaces and representations are effective in conveying the varying state, availability and efficiency of the network at any time.
522
D. Fairbairn
It is, therefore, clear that representations are data dependent: the range of design choices available to the contemporary cartographer needs to reflect the diversity of data. Just as clearly, the tasks to which the data are applied are critical to the appearance of the representation.
26.3.4
Task-dependent application of geovisualization in public transport
A major issue associated with interface design is that interaction with geospatial data within a public transport information system may be undertaken by novices. Certainly, much interaction by passengers with public transport information systems may well require the moderation of the enquiry by use of a switchboard or personal contact. However, at times when such assistance is not available, knowledge discovery techniques and artificial intelligence can potentially assist. Intelligent agents, interacting with users, can translate requests (limited, for example, by time, route or transit company), retrieve data and present information (Rauschert et al., 2002). Prompt-led journey planning is wide spread, (Kenyon et al., 2001), although links to graphical representations are limited. It is suggested that geovisualization techniques should become more closely coupled to journey planning and route investigation applications. Some Web-based enquiry systems do yield functional graphic output (Figure 26.4), to a certain extent replicating earlier paper-based products presented individually on request by motoring organizations for their members (Dorling and Fairbairn, 1997).
Figure 26.4. Web-based route enquiry system and resultant schematic representation, German Railways (Deutsche Bundesbahn).
Geovisualization Issues in Public Transport Applications
26.3.5
523
Technological development for geovisualization in public transport
In many cases, real-time handling of bus information will be undertaken using automated data logging, for example, logging of GPS-derived position (Level 5), but in the context of representation, the major need is to develop methods for using mobile communication devices (notably mobile phones and PDA devices). These contemporary mobile communication devices with restricted graphical capabilities pose significant problems for the cartographic designer, but are clearly of importance for users who are on the move. Further consideration of this challenge is given in the next chapter. Here, we can indicate that the potential of transport representations on mobile devices and distributed systems is high. Both passengers (en-route) and operators (monitoring performance "in the field") are likely to have access to such technology and geovisualization must address the peculiar needs and capabilities of these devices. Initial attempts to deliver simplistic map presentations in WAP-based applications failed due to constraints of the devices. New, more sophisticated mobile devices offer cartographers considerably greater potential (Gartner and Uhrliz, 2002). Currently developing protocols, such as General Packet Radio Service (GPRS) or Universal Mobile Telecommunications System (UMTS), offer higher transfer rates, graphic capability and other multi-media functionality. The exploitation of these enhancements, integrated with 3G mobile phones technology seems promising for applications in both en-route journey plan revision and decision making, as well as transport operations management. Such possibilities of data dissemination can be integrated with automated position-fixing; variable-scale display of routes on relatively high resolution PDA screens; textual enhancement by downloading timetable information; graphical, pictorial and audio supplementary data for descriptive and confirmation purposes; routing possibilities; and feedback loops for responding to queries (Gartner and Uhrliz, 2002). Contemporary studies of mobile Cartography have revealed a range of problems and opportunities in the representation and interface with geospatial data on the mobile platforms available today (Gartner et al., 2001; Hardy et al., 2001). For efficient rendering of data on power-hungry and memory-challenged mobile devices with limited storage and processing capabilities, a key challenge is to determine what data is to be disseminated; the goal is to provide exactly what the user needs, and nothing more. To optimize data delivery, user profiles are needed indicating (by communication from client to server) the limitations of the device, the location of the client, the task being undertaken, the required scale and, in the context of a transport information system, supplementary data such as timetables, connection details and the real-time location of services (US National Research Council, 2003). The rendering of this data must be adaptive (Reichenbacher, 2001) such that the wide range of tasks (notably those indicated in Levels 3, 4 and 5) can be addressed by a wide range of users operating a wide range of mobile devices whilst in transit.
524
26.3.6
D. Fairbairn
User-dependent application of geovisualization in public transport
At each of the levels indicated, there are users who have varying requirements. At L e v e l 1, planners and administrators need to access large amounts of spatial data, perform detailed analyses, run simulations and modify operational systems. The planning of a new bus route, the location of bus stops and the determination of service frequency all need real-time GIS using demographic, census and topographic data with well-designed map handling capability. Unfortunately, the current failure of GIS to incorporate exploratory geovisualization methods is limiting its effectiveness in addressing the decision-making needs of the transport industry. At other levels there are user needs for real-time, simple and useful information. Geovisualization should assist all stages of GIS, integrating, navigating, interacting with, analysing and presenting geospatial data effectively. In each of these operations, there is an objective of achieving usability in all representations: interface design is important in this respect.
26.3.7
Interface design for public transport information systems
Research into developing more effective interfaces to geospatial data has a number of purposes: to improve user navigation in "hyperspace", to stimulate creative thinking, to make use of new computer architectures and platforms (including mobile and distributed devices), and to optimize user rapport with data and tasks which can be performed using the data. The "spatialization" of human activity in navigating "virtual space", according to Fabrikant and Skupin, this volume (Chapter 35), can be compared to journey planning and monitoring in the "real world" using public transport services and information. An interface which uses network diagrams to show connections and routes, or which uses a step-by-step approach to the acquisition of information, will be naturally appealing to a passenger who is planning a route from point A to point B. In a similar way, such 2D line diagrams could be useful to represent and allow use of a range of spatial data and other data, for example, in the form of "Web trails" improving and recording access routes to the World Wide Web. It is, of course, possible to engender less focussed (possibly more creative) thinking by interface design which encourages hyperlinking to more marginal and exceptional sources. However, it is unlikely that such an approach could benefit the public transport information system user: planning, managing, querying and navigating real-world transport networks require a task-oriented, sequential approach to interface design and information retrieval. It is in the area of information display, retrieval and query whilst using mobile devices that most potential exists in developing novel, effective interfaces (Clarke, 2001). It is particularly important for such limited devices to ensure the appropriate level of generalization and scale of mapping and an effective interface developed from the tried and tested WIMPs (Windows, Icons, Menus, Pointers) metaphor.
Geovisualization Issues in Public Transport Applications
26.4
525
Wider Geovisualization Challenges
The research agenda cited above (MacEachren and Kraak, 2001) has identified representation, databases, usability and interface issues as being the major areas of geovisualization research. Some issues, however, transcend these distinct areas and require a more integrated approach to public transport information system design. These are highlighted below.
26.4.1
Experiential technologies
As has been indicated, there is considerable scope for interactivity in public transport information systems. Users, in particular, have a need for the assurance and valuable information supplied by a range of feedback mechanisms, from a simple "you are here" display to the complex answer to a multiple-journey route enquiry. In many cases, there is a need for the user to react to data, representations and visualizations. In addition, user feedback can provide an indication of errors and the effectiveness of a journey planner (Kenyon et al., 2001).
26.4.2
Overall human factors
Further human factors are important: how do users react to unconventional media displays on PITs, at bus stops or on-board? Can we cope with different competencies of users, perhaps by changing the representations available to them? How is the decision-making of users affected by representations, whether graphical, numerical or narrative? The bus stop display may merely indicate the time until the next bus arrives, whilst the operations room may have a full topographic map base with real-time updated plotted positions of every bus.
26.4.3
Size and complexity of contemporary transport spatial data sets
It is clear that a public transport information system, at whatever level it is applied, is likely to suffer from data glut, rather than data scarcity The representation of vast amounts of information is a problem common to many such systems, not only a bus information system. The visualization challenge is to render data in ways which reflect such complexity, whilst ensuring the efficient rendering of graphical output.
26.4.4
Mobile and distributed platforms
The convergence of the ICA research agenda with the ideas promoted by the US National Research Council IT Roadmap reveals that "research is needed to develop contextsensitive representations of geospatial information and to accommodate data subject to continual updating from multiple sources ... [and to address] perceptualization issues in connection with the need for small, lightweight, and mobile technologies that can be used in public spaces" (US National Research Council, 2003, p.94). The results of such crosscutting research will affect all transport passengers who rely on mobile devices
526
D. Fairbairn
for journey planning and monitoring, for map representations of their route and the network, and for effective interface to operational information.
26.5
Summary
The geovisualization research agenda has outlined a series of important challenges for the cartographic community (MacEachren and Kraak, 2001). This chapter has identified those challenges which are most urgent and engage from the viewpoint of visually enabled information retrieval, decision-making and geospatial data presentation for a bus information system. They include the need to abstract real-world data, both spatial and temporal, to a suitable degree for graphical display; the necessity of ensuring that schematic representations are used to their full capability; and that supplementary cues, such as audio, are effectively integrated with visual displays. Also, it is suggested that the supply of public information using a range of contemporary means, such as mobile devices and kiosks, be examined to determine the most effective means of representation. The ability of an associated geovisualization system to account for task-, location-, and user-dependence should also be determined, along with the flexibility of design in adjusting the display to meet such criteria. Such flexibility must also be evident in the interface design which connects a user to the information-providing service. Furthermore, it has been suggested that the nature of transport information databases, along with the variety of users and media used for communicating information to a customer, mean further demands on the representation capabilities of such a system. Facing such challenges requires a willingness to cooperate with domain-specific users, such as those providing public transport services, and with specialist data handling experts, such as computer scientists, graphic designers and those familiar with contemporary human-factors research. The responsibilities of, but also the opportunities for, cartographers to supply effective geovisualization methods and resulting displays for the public transport information sector are being tested. Both they and other current researchers in geovisualization have a significant role to play in the efficient rendering, communication and querying of public transport information and meeting the challenges of this important area of human activity.
Acknowledgements Many thanks to Alan MacEachren for informative comments on drafts of this chapter and the identification of a number of useful sources.
References Agrawala, M., and Stolte, C., (2001) "Rendering effective route maps: improving usability through generalization", SIGGRAPH 2001, Los Angeles, pp. 241-249.
Geovisualization Issues in Public Transport Applications
527
Andrienko, N., Andrienko, G., and Gatalsky, P., (2000) "Supporting visual exploration of object movement", Proceedings of the Working Conference on Advanced Visual Interfaces A VI 2000, Palermo, Italy, pp. 217-220. Blessington, H., and Tarrant, D., (1995) "The ROMANSE project", Proceedings of the Institution of Civil Engineers Conference, Birmingham, pp. 78-87. Centro (2001) Getting around the West Midlands. Online: http://www.centro.org.uk/centro587364/Corporate/Infoplan4.htm (01/09/02). Clarke, K., (2001) "Cartography in a mobile intemet age", Proceedings of 20th International Cartographic Conference, Beijing, pp. 1481-1488. Deakin, A., (1996) "Landmarks as navigational aids on street maps", Cartography and Geographic Information Systems, 23(1), 21-36. Denmark, D., (2000) Best Practice Manual: Public Transport Information. Sydney, Australia: NSW Ageing and Disability Dept. Direction R~gionale de l'l~quipement Ile-de-France (2002) SYTADIN - l'6tat du trafic en Ile-de-France en temps reel. Online: http://www.sytadin.tm.fr/ Dorling, D., and Fairbaim, D., (1997) Mapping: Ways of Representing the World. London: Longman. Fairbairn, D., Andrienko, G., Andrienko, N., Buziek, G., and Dykes, J. A., (2001) "Representation and its relationship with cartographic visualization: a research agenda", Cartography and Geographic Information Sciences, 28(1), 13-28. Franzen, S., (1999) Public Transportation in a Systems Perspective - A Conceptual Model and an Analytical Framework for Design and Evaluation, PhD thesis. Chalmers University. Garbis, C., (2002) "Exploring the openness of cognitive artifacts in cooperative process management", Cognition, Technology and Work, 4(1), 9-21, Online: http://www. tema.liu, se/people/chrga/papers/pdf/sl.pdf Gartner, G., and Uhrliz, S., (2002) Maps, Multimedia and the Mobile Internet, Geowissenschaftliche Mitteilungen (Technische Universitat Wien), 60, 143-150. Gartner, G., Uhrliz, S., and Pammer, A., (2001) "Mobile Internet: applying maps to mobile clients", Proceedings of the Workshop on Maps and the Internet in the 20th International Cartographic Conference, Beijing. Hardy, P., Haire, K., Sheehan, R., and Woodsford, P., (2001) "Mobile mapping ondemand, using active representation and generalisation", Proceedings of 20th International Cartographic Conference, Beijing, pp. 3239-3247. Infopolis, (2000). Online: http://www.ul.ie/--infopolis/ Infopolis 2 Consortium (undated) Infopolis 2 - Overview of Existing Systems. Online: http ://www.ul.ie/--infopolis/existing/index.html intermobil Region Dresden (2003) dd-info Stadt und Verkehr Informationssystem Region Dresden. Online: http://www.intermobil.org/doris/net/start/start.xml (23/10/03). Kenyon, S., Lyons, G., and Austin, J., (2001) Public Transport Information Web Sites: How to Get it Right. London: Institute of Logistics and Transport. Krygier, J., (1994) "Sound and geographic visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Oxford, UK: Pergamon, pp. 146-166.
528
D. Fairbairn
MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Science (Special Issue on Geovisualization), 28(1), 3-12 (American Congress on Mapping and Surveying, 80 pp). Parkes, D., (1988) "Nomad, an audio-tactile tool for the acquisition, use and management of spatially distributed information by visually impaired people", Proceedings of the Second International Symposium on Maps and Graphics for Visually Impaired People, London, pp. 24-29. Prasad, R., (1997) Surface Mount Technology. New York, NY: Chapman and Hall, p. 772. Rauschert, I., Agrawal, P., Fuhrmann, S., Brewer, I., Wang, H., Sharma, R., Cai, G., and MacEachren, A. M., (2002) "Designing a human-centered, multimodal gis interface to support emergency management", A CM GIS'02, l Oth A CM Symposium on Advances in Geographic Information Systems. Washington, DC, USA: ACM, pp. 119-124, November, 2002. Reichenbacher, T., (2001) "The world in your pocket - towards a mobile cartography", Proceedings of 20th International Cartographic Conference, Beijing, pp. 2514-2521. Strothotte, T., Fritz, S., Michel, R., Petrie, H., Johnson, V., Reichert, L., and Schalt, A., (1996) "Development of dialogue systems for a mobility aid for blind people: initial design and usability testing", ASSETS '96, Second Annual ACM Conference on Assistive Technologies. CaFnada: Vancouver, pp. 139-144. US National Research Council Computer Science and Telecommunications Board (2003) IT Roadmap to a Geo-Spatial Future. Washington, DC: National Academies Press, p. 119.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 27
Presenting Route Instructions on Mobile Devices: From Textual Directions to 3D Visualization Volker Coors, Stuttgart University of Applied Sciences, Schellingstr. 24, 70174 Stuttgart, Germany Christian Elting, European Media Lab (EML), Schloss-Wolfsbrunnenweg 33, 69118 Heidelberg, Germany Christian Kray, German Research Center for AI (DFKI), Stuhlsatzenhausweg 3, 66123 Saarbrticken, Germany Katri Laakso, Nokia Research Center (NRC), It~imerenkatu 11-13, 00180 Helsinki, Finland
Keywords: route instructions, mobile devices, positional information, 3D maps, multimodal presentations
Abstract In this chapter, we evaluate several means of presenting route instructions to a mobile user. Starting from an abstract language-independent description of a route segment, we show how to generate various presentations for a mobile device ranging from spoken instructions to 3D visualizations. For the latter type, we provide a novel compression algorithm that makes 3D presentations feasible even in a mobile setup, and we report the results of a pilot study using 3D visualizations on a mobile device. In addition, we examine the relationship between the quality of positional information, available resources and the different types of presentations. The chapter concludes with a set of guidelines that can help to determine which presentation to choose in a given situation.
27.1
Introduction
Mobile devices such as portable digital assistants (PDAs) and mobile phones have become tools that we use on a daily basis. Currently, we can observe a convergence: cell phones incorporate more and more functionality, which was once the domain of PDAs and ultra-portable computers, while PDAs and similar devices can be updated with communication abilities or already have them out of the box. The proponents of this new class of devices often cite location-based services (LBS) such as incremental guidance in a foreign city as a key benefit. 529
530
V. Coors, C. Elting, C. Kray & K. Laakso
In this chapter, we present an analysis of this specific service with a focus on how to present route instructions depending on various situational factors such as limited resources and varying quality of positional information. We first review some general considerations for presentations on mobile devices. After a brief definition of positional information, we discuss the process of generating route instructions, presenting route instructions, and technical issues associated with presenting route information in 3D. This is followed by a brief discussion of an empirical assessment of the 3D representation methods devised. Based on this discussion and on empirical evidence, we then present guidelines on when to select which type of presentation. 27.2
Generating Presentations for Mobile Assistance
In order to produce a presentation that is appropriate for the current situation it is necessary to adapt it to situational factors such as the currently available resources or the quality of the available positional information. Especially, when generating a presentation on mobile devices, two types of resources are of special importance: Technical resources include such factors as speed, bandwidth and screen resolution. The power of the underlying hardware not only influences the speed of the generation process but also limits the presentation in terms of complexity and media choice. On a low-end PDA, for example, animated instructions may be unfeasible due to latency of the screen. The network bandwidth is a further limiting factor that has an impact on the transmission of data, e.g., the streaming of movies. Furthermore, the screen resolution influences the layout and the level of detail of a presentation. Consequently, limited technical resources are especially important for presentations on mobile devices. The cognitive resources of the user also impact the way that information should be presented to the user. Depending on what the user is doing at a given moment, her working memory or attention may be severely limited, e.g., when performing two tasks at the same time. Therefore, an electronic tourist guide should for instance avoid computing routes on which inexperienced users are likely to get lost (Baus et al., 2002). Taking into account the cognitive resources of the user is crucial because there is evidence that for many people the use of a mobile device in itself results in a higher cognitive when compared to the use of other, more familiar electronic devices (Elting et al., 2002). In addition to cognitive and technical resources, the user's current position is not only a key factor in determining what situation a mobile user is in but also highly relevant in the context of generating and presenting route instructions. For example, knowing the user's current location is vital if the beginning of a route corresponds to her current location, which frequently occurs in the case of a mobile user. However, we have to analyze and define more precisely what exactly constitutes the user's current position. Figure 27.1 illustrates the different facets that we subsume under the term positional information. As we see it, focusing solely on a person's location in space
Presenting Route Instructions on Mobile Devices
531
Figure 27.1. Positional information.
(e.g., X- and Y-coordinates) means to leave out some important facets: We subsume under the notion of positional information all spatial information that is necessary to uniquely align a human user in space. Aside from the location the following aspects fall into that category: speed, acceleration, heading, body orientation, and viewing direction. In many cases, a system has to know these factors as well in order to provide the user with services that are tailored to her current position. Route directions, for example, often include turning instructions, which depend on a reference direction (i.e., the viewing direction, body orientation, or heading). The user's current speed and acceleration is equally important for timing and designing route instructions. For each facet of positional information there are several types of sensors that can measure it at varying precision and that are prone to various errors such as shielding, reflection, outage, or occlusion (Baus and Kray, 2002). Since there is a limit to the weight a user is willing to carry around, the number of sensors is limited as well. Therefore, the user's current position often cannot be determined precisely or entirely. Consequently, a robust system has to provide means to cope with missing or imprecise positional information, and should also be able to function in case of no sensor readings at all (e.g., in a narrow street where a GPS receiver has no reception). These considerations not only apply to the generation and presentation of route instructions but also to most types of location-based services in general. Ideally, further situational factors should be taken into account when generating and presenting route instructions on mobile devices. For example, the user's age and physical constitution may have an impact in the process, as do the current means of transportation and the weather conditions.
27.3
Generating Route Instructions
Incremental guidance is a service that is frequently associated with the concept of a mobile assistant. In addition, it is a field, where the advantage of a mobile device compared to a stationary one is directly apparent. In order to produce route instructions, several processes have to be completed. In the first step, the origin and target location (and possibly intermediate locations) have to be determined. Depending on how the user relays this information to the system, this may include speech recognition, semantic
532
V. Coors, C. Elting, C. Kray & K. Laakso
parsing, gesture analysis, and frequently the determination of the user's current position as well (i.e., if the origin of the route corresponds to the user's current position). The second step consists of the computation of a suitable route that not only leads from origin to target but ideally takes into account situational factors such as the user's preferences or means of transportation. If the resulting route is to be described incrementally, it is also necessary to divide it into smaller segments. But even in the case of complete route descriptions such segmentation may be necessary, e.g., in order to generate a verbal description of a non-trivial route. Since it is unlikely that the user will always meticulously follow the route, her progress has to be monitored continuously. Should she leave the proposed route, or exhibit a behavior indicating that she might be lost (e.g., moving back and forth several times), a new route or segment has to be computed dynamically. Finally, the route has to be presented to the user, either as a whole or incrementally. In the latter case, several presentations need to be generated that have to be timed according to the movement of the user, and the system has to present them at the fight location. Additionally, it has to choose the appropriate media and presentation mode. Consequently, there is a need for a high level representation of route instructions that allows the generation of complete or incremental directions in various media and modes. The format that we propose addresses these requirements and has been successfully applied within a mobile tourist guide (Malaka and Zipf, 2000). This format includes a preverbal message (pvm) that comprises several relational localizations for key points on a route segment, a path relation (Kray and Blocher, 1999) describing the trajectory, and a turning instruction. Relational localizations are generated for the start and the end point of each segment as well as for the corresponding trajectory as a whole. They consist of a qualitative spatial two-point relation (Cohn, 1996) and a corresponding anchor object (or reference object). All relations are rated using a normalized degree of applicability (Schirra, 2000). Figure 27.2 shows an example pvm as well as the different presentations that can be generated from it.
27.4
Presenting Route Instructions
We describe various ways to present route instructions on a mobile device as well as their respective properties in w These instructions are generated from the same underlying representation format - the preverbal message.
27.4.1
Textual and spoken instructions
Figure 27.2a shows an example of route instruction in natural language. Depending on the situation (e. g., availability of rendering resources) the system can either chose to output it textually, use speech synthesis, or use a combination of both. When generating a natural language utterance, there are several degrees of freedom that result from the information contained within the pvm. First of all, not all information needs to be verbalized. For example, the overall goal of the route should probably only be mentioned
Presenting Route Instructions on Mobile Devices
533
Figure 27.2. A preverbal message and four ways to present it to the user.
at the very beginning of a route (to make sure that it is the right one), or when the route is resumed after an interruption. Depending on the user' s knowledge of the environment, the system may choose to leave out most information (if she is familiar with it) or to always include all available information (if she is on a sightseeing tour in an unfamiliar city). If speed is an issue, for example, in case of a fast car driving quickly approaching an exit on a highway, information may be limited to only a turning instruction given at exactly the right moment. In case of high cognitive load, a combination of a turning instruction paired with the path relation may be advantageous as it would result in a short instruction ("Turn right onto Main street.") and require little memorizing effort. However, as the length of the segment/route increases the user needs to remember spoken instructions longer, which may require constant rehearsal in case of an elaborate sentence such as that shown in Figure 27.2a. Textual instructions in such a case may also require re-reading and may take longer to decode compared to other graphical means (such as 2D route sketches or 2D maps).
27.4.2
2D route sketches
Another way to present route instructions is familiar to users of car navigation systems. It consists of a (mostly qualitative) 2D route sketch such as (Agrawala and Stolte, 2001) or the one shown in Figure 27.2b. In its most abstract form, only an
V. Coors, C. Elting, C. Kray & K. Laakso
534
arrow pointing in the intended direction of motion is shown. The pvm can be used to annotate this with additional information. Since key points of a segment are localized using qualitative spatial relations, it is a straightforward task to add the corresponding annotations to the basic arrow. For example, the start point is localized using the relation (contact, Marktplatz, 1.0), which indicated that the beginning of the segment is the market place. Consequently, we can label the start point of the arrow shown in Figure 27.2b with "Marktplatz". Using the path relation included in the PVM, a label for the arrow may be generated. In the example, the path relation is (follow, Hauptstra~e, 0.8), which results in the body of the arrow being labeled with "Hauptstra[3e" (main street). The selection of which components of a pvm should be realized can be guided by the degree of applicability of the corresponding relation as well as by situational factors. The main advantage of route sketches can also be a key disadvantage: While the high level of abstraction takes away all unnecessary information and may help to focus on the relevant aspects, it may also leave out things that would help a user to find her way, i.e., contextual information about the environment.
27.4.3
2D maps
The most common way to present route instructions graphically is a geographic 2D map, which is annotated with information that is relevant in the context of the route (e.g., landmarks). A 2D map provides a reasonable overview of the surroundings and is well suited to present a general overview of a tour. Compared to dynamic 3D walkthroughs (see below), a 2D map is a more abstract. It is also usually rendered as a static picture, so that the user does not need to focus on the map all the time. The information contained in the pvm can be used to generate maps that are a graphical rendering of a route instruction. The names of the relevant locations of the route instruction (contained in (start), (global-goal), etc.) are included in the map and the path of the user is highlighted. One challenge in generating maps is to produce geographical information that can be properly decoded by the user. Therefore, it is important to annotate a 2D map with additional information that makes it possible for the user to match it with her actual surroundings. In addition, it is necessary to find a proper zoom factor for the map, which provides sufficient details, but does not overload the user with information. If contextual information about the user's orientation is provided it may be useful to rotate the map into her current viewing/walking direction (Baus et al., 2002). This may help to preserve the user's cognitive resources, as she does not have to perform a mental rotation, which is a very demanding cognitive task. If no information about the user's orientation is available it is crucial to include information about landmarks (e.g., Churches) into the map that can be viewed from a distance and serve as orientation points (Deakin, 1996). Finally, it is also necessary to find a proper level of detail for the annotations of the map, e.g., street names or the highlighting of certain buildings or areas.
Presenting Route Instructions on Mobile Devices
27.4.4
535
Pseudo-realistic instructions
A very natural way to present route instructions is to use pseudo-realistic instructions, i.e., 3D maps or animations. There is some evidence that the use of a 3D model facilitates the task of recognizing landmarks and finding routes in cities (compared to the use of a symbolic 2D map). In addition, search and visualization of location-based information seems to be more intuitive with life-like 3D (Rakkolainen ct al., 2000). The 3D visualization has a model character, i.e., the objects of the real world displayed should be represented in a geometrically correct way and at the right position. Furthermore, the visualization as a means of communication demands an adequate degree of readability. A preverbal message contains the most relevant information for route instructions, and hence defines the thematic focus of the resulting 3D map. In order to prepare a graphic abstraction according to a given thematic focus the contribution of the different features within this focus must be specified. For each feature, a decision about its importance for achieving this output goal must be made. The pvm contains several important features in the context of a route: the start, the end, and the global goal as well as the anchor objects of the two-point relations. Other eye-catching buildings can also be used as visual landmarks. These features should be accentuated by a pseudo-realistic representation, for example. Additional buildings can provide a helpful context, but can also confuse and distract from the original aim of the navigation support. Figure 27.3 shows an example sequence of a 3D-route map. Landmarks are visualized in detail with textured models to attract the user's attention. Less relevant buildings are rendered in grey and in a semi-transparent way. In order to specify the abstraction level we relate a dominance value to each feature, similar to the approach presented in Hartmann et al. (1999), which reflects the ranking of this feature in the communication of the reply to the original request. For ease of handling, these dominance values can be classified, according to Krtiger, (2000), into four representation classes: visible, discriminable, classifiable, and identifiable. A dominance function relates a dominance value to each feature f E F according to the pvm E PVM: dom : F X P V M ---, 9~
(1)
Figure 27.3. Some sequences of a 3D-route map to the Fraunhofer Institute of Computer Graphics. Navigational landmarks are visualized in detail with textured models to attract the user's focus while buildings with less dominance are shown in grey-scale and semi-transparent rendering style.
536
V. Coors, C. Elting, C. Kray & K. Laakso
One factor that influences the dominance function is the relevance of a feature in the context of a user specific query. In the case of route instructions this is a spatial query searching all buildings along the route that assigns a relevance value factor to each feature in the underlying 3D model. This relevance factor is calculated based on the approach presented in Keim and Kriegel (1994). First, the distance between an attribute and the corresponding query values is determined for each feature. The distance functions used in this step depend on data-type and application. However, the dominance domf of a feature depends on more than the relevance factor R(pvm). Other factors are the use of a feature as an anchor object O(pvm) in a twopoint relation, and the general use of this feature as a landmark L(user). These influence factors, together with relevance, can be weighted using the parameters a0, a l, and a2: domf : ao • R(pvm) + al • O(pvm) + a 2 • L(user)
(2)
For route visualization the dominance value of a feature is mapped to a level-ofdetail function: the higher the dominance of a feature the more detail will be included in the model used to visualize it. Assigning a0 a low value results in fewer buildings along the route that are neither landmark nor anchor object. A higher value of a0 will lead to more realistic models. However, the resulting visualization will not immediately draw the user's attention to the landmarks and anchor objects.
27.4.5
Multi-modal instructions
There are several possibilities for combining the above-mentioned presentations. However, not each combination produces meaningful results. It is a basic problem of presentation planning to compose multi-modal presentations that are coherent and can be properly decoded by the user (Wahlster, 1998). This is especially true in the case of presentations in which modalities include cross-modal references or coreferences to world objects, such as buildings in route instructions (Andr6 and Rist, 1995). Since 2D cartographic route instructions provide a rather abstract rendering by means of a geographic map it is useful to combine their display with a linguistic output modality, which can add the details of the route instructions (e.g., exactly when to turn on the route) by means of speech or written text. Speech and textual output allow the presentation of nearly any given content on a certain level of detail (Bernsen, 2001). A common problem of presentations involving speech is that many users prefer to explore presentations rather than being a passive consumer (e.g., users might stop and repeat a certain part of the presentation if they did not understand it properly). Therefore, it makes sense to combine spoken and written output so that users can re-read it if they want to (Elting et al., 2002). In order to support explorative presentations, speech can also be attached to objects and accessed as the user explores - similar to Cartwright's sage metaphor in the GeoExploratorium (Cartwright, 1999). However, this has to be done with care as speech is much more intrusive than textual output. In addition, the voice of the speech synthesis is also crucial for the acceptance of the whole presentation since a user is easily annoyed by a voice that she dislikes. Many
Presenting Route Instructions on Mobile Devices
537
implementations of multi-modal dialog systems suffer from rather long processing times. Acoustic speech output can also serve to signal that an answer to the user's request has been calculated and to guide the user's attention. Dynamic presentations may be beneficial in the context of a route that consists of several route elements as the user can be guided step by step through the presentation so that in each step the presentation can focus only on those elements that are necessary for this step (e.g., a certain location on the route or the direction to choose at a crossing). Therefore, it is possible to reduce the amount of details that are incorporated into a single static map. However, contrary to static maps, dynamic presentations demand the user's attention all the time. Therefore, they have to be chosen with care - especially on mobile devices as they may distract the user from more important tasks such as driving a car. We implemented two kinds of dynamic presentations: The first one included a sequence of map-text presentations, in which each step described a route element of the tour that was highlighted on the map. The second one was an alternative with a scrolling 2D map that followed the path of the tour. Text was displayed dynamically below the map whenever a new route element was reached. In both cases the user should be able to stop, continue or reset the presentation. In contrast to dynamic map-text presentations, 2D geographic maps can be combined with sketched route instructions instead of text. Then, the user can focus on the main information of the route instruction, which is displayed by the sketch, but can also explore the surroundings using the context provided by map. In this presentation, it is important that the user can identify her position on the sketch with her position in the map. Therefore, the user's location should be pinpointed in both and the orientation of the sketch and the map should be the same. This is crucial as the evaluation results from the study presented in w suggest that the cognitive demands of matching two maps (in this case 2D and 3D) may be high. However, Rakkolainen et al. (2000) also show that some users prefer a combination of a 2D and a 3D map (if properly aligned) over a presentation with a single map.
27.5
Technical Issues of 3D Maps
In w we outlined various ways to present route instructions using text and speech, route sketches, 2D and 3D maps, and multi-modal combinations of these presentations. While 2D maps on mobile devices are widely used, the deployment of 3D maps is still in an early stage of development. Several technical limitations on PDAs have to be taken into account, especially rendering capability, screen size, and limited bandwidth. Here, we will further discuss these technical issues and propose a compression technique that addresses the bandwidth problem. Currently, only small textured 3D models can be rendered on a PDA at frame rates above 10 fps (which are required, e.g., for interaction). However, the presently low visualization performance will be significantly improved following the rapid development in the range of graphic processors. Therefore, we expect hardware-accelerated graphics on PDAs in the near future.
538
V. Coors, C. Elting, C. Kray & K. Laakso
The small screen size on PDAs limits both 2D and 3D maps. Unfortunately, this will most likely not change much in future as the small size of PDAs is one of their key features. Hence, we have to deal with the screen size during the mapping process, by using interactive tool tips instead of a map legend to save space on the screen for example. However, the screen size problem will be one of the main future challenges in geovisualization on mobile devices. The currently available mobile network standard (GPRS) allows a transfer rate of 57.4 kbps. With third generation mobile communication networks like UMTS the bandwidth will increase significantly and transfer rates up to 384 kbps are expected in urban areas. However, the large amount of data needed to encode 3D maps is a problem even in high bandwidth wireless networks. As the limited bandwidth of current mobile networks is one of the main bottlenecks in the visualization pipeline, specialized compression methods for 3D geometry have to be used to minimize the amount data. Transmitting a 3D map stored as a 3 mb VRML file via UMTS will take about 60 s. A standard compression algorithm like gzip reduces the data volume to 1 mb and transmission time to 20 s. Still, 20 s is a long time for the user to wait. The proposed Delphi compression method, which is specialized on 3D geometry, will compress this 3D map down to 180 kb. Transmission of this file will take less than 4 s.
27.5.1
Triangular meshes
Before going into details of the Delphi compression algorithm, we shortly introduce some fundamental concepts of 3D graphics. Triangle meshes are the de facto standard for exchanging and viewing 3D data sets. This trend is reinforced by the wide spread of 3D graphic libraries (OpenGL, VRML) and other 3D data exchange file formats, and of 3D adapters for personal computers that have been optimized for triangles. A triangle mesh is usually stored as a list of coordinates (vertex list) and a list of triangles that reference these coordinates (connectivity). Figure 27.4 shows a VRML representation of a triangle mesh. For manifold meshes Euler's formula gives the relation between numbers of vertices V, edges E and faces F as V - E + F = 2. In a triangulated manifold mesh each edge bounds two faces and each face is bounded by three edges, or 2E = 3F. As a result, in a large model with the number of triangles F is F -- 2V.
27.5.2
Delphi compression of triangular meshes
The Delphi compression technique for triangle meshes (Coors and Rossignac, 2003) is based on the Edgebreaker algorithm (Rossignac, 1999; Rossignac et al., 2001). Edgebreaker uses a state machine to traverse the mesh and compress its connectivity. The succession of case types produced by this traversal are encoded as a succession of symbols from the set {C, L, E, R, S} , called the CLERS sequence. For zero-genus meshes, the CLERS sequence is sufficient to represent the complete connectivity. These situations and the associated CLERS symbols are shown in Figure 27.5. At each state the state machine moves to a triangle T. It marks all visited triangles and
Presenting Route Instructions on Mobile Devices
539
In VRML 97: IndexedFaceSet { # Geometry (Vertices) coord Coordinate { point [ x0 y0 z0, xl yl zl, ..., x8 y8 zS] } # Connectivity Coordlndex[0 1 5 - 1 , 0 5 4-1,..., 6, 7, 8] } Figure 27.4. VRMLrepresentationof a triangularmesh. their bounding vertices. Let left and right denote the other two triangles that are incident upon T. Let v be the vertex common to T, left, and right. If v has not yet been visited, then neither have left and right. This is case C. If v has been visited, we distinguish four other cases, which correspond to the four situations where one, both, or neither of the left and right triangles have been visited. The arrow indicates the direction to the next triangle in the traversal. Note that in the S case, Edgebreaker moves to the right, using a recursive call, and then to the left. The popularity of Edgebreaker lies in the fact that all descriptors are symbols from the set {C, L, E, R, S}. No other parameter is needed. Because only C triangles introduce a new vertex V and the number of triangles F is 2V, half of the descriptors are Cs. A trivial code (C = 0 , L = l l 0 , E = l l l , R = 1 0 1 , S = 100) guarantees 2 bits per triangle (bpt). To further reduce the costs for storing the mesh connectivity, the Delphi algorithm predicts the location of vertex v of the next triangle in the mesh traversal by using the parallelogram rule (Touma and Gotsman, 1998). Delphi predicts the symbol of the corresponding triangle by snapping the vertex v to the nearest
V 1
L
C
R
S
E
Flgure 27.5. Mesh traversal creates a C L E M sequence.
Edgebreaker
Delphi
CLERS sequence: CRSRLECRRRLE (32 bit) Figure 27.6. Mesh traversal in Edgebreaker creates the CLERS sequence. Delphi performs the same traversal, but predicts the CLERS symbols leading to the Apollo sequence. Correct predictions are shown in green, wrong predictions in red.
Presenting Route Instructions on Mobile Devices
541
already visited vertex, if one lies sufficiently close. If no boundary vertex is close enough, a C triangle is predicted. Otherwise, one of the other four symbols is predicted based on the location of the snapped vertex. If the guess is correct, only a confirmation bit needs to be transmitted. The decompression performs the same prediction and can directly restore the CLERS sequence. If the prediction was wrong, some bits have to be stored in order to rectify the prediction. This sequence of confirmation and rectification bits is called the Apollo sequence, a name inspired by the ancient Greek god Apollo, who is bound to the truth and cannot lie. Figure 27.6 compares the CLERS and the Apollo sequence of a small part of a triangular mesh. Assume that the rest of the mesh is already traversed. Because up to 97% of Delphi's predictions are correct, connectivity information is often compressed to a fraction of a bit per triangle using an entropy encoder. Experimental results lead to a compression rate up to 0.5 bpt for urban models. In order to compress the vertex list, the same prediction of vertex v can be used. For each new vertex during the mesh traversal introduced by a C triangle, only a corrective vector is transmitted to compensate for the error between the correct location and its prediction. In general, the distribution of corrective coordinates has lower entropy than the distribution of the original coordinates (e.g., they spread around zero). Therefore, the corrective coordinates can be compressed with fewer bits on average. Combined with quantization and entropy encoding about 10 to 12 bit per vertex (bpv) are needed to store the vertex list (Touma and Gotsman, 1998). Note that due to quantization the compression of the vertex list is lossy and the quantization parameters have to be chosen with care. Summing up, with Delphi compression connectivity and vertex list can be compressed down to 11-13 bpv. Compared to an internal representation of a mesh where three float values are used to store each vertex (36 bpv) and three integer value to store each triangle (36 bpt or 72 bpv), Delphi' s compression rate is about 98%. A 3D map with 100.000 triangles is compressed down to 80 kb plus additional costs for color and textured coordinates. A detailed explanation of Delphi compression is given in Coors and Rossignac (2003).
27.6
Evaluation of 3D Maps
Since 3D maps are just being introduced as a navigation aid, one of the key unanswered questions is what the relative advantages of a 3D map and a 2D map will be. Therefore, we focused on using mobile 3D maps in a pilot study. Empirical results for the presentation of route instructions in connection with combinations of 2D and 3D maps were collected during the evaluation of the first prototype of a 2D/3D map application for the Nokia Communicator. This evaluation took place in August 2002 in TCnsberg, Norway with a group of ten boat tourists. We summarize the most important results of the evaluation here and further details are published elsewhere (Laakso, 2002).
542
V. Coors, C. Elting, C. Kray & K. Laakso
The main purpose of this pilot study was to collect feedback about using mobile 3D maps in city environments. We plan to follow this up with evaluations of later prototypes.
27.6.1
Participants
There were a total of 10 users participating in the tests (plus one user who participated in a pilot of the test). All users were selected at TCnsberg harbor without strict criteria. Nine of the users were males and one was female. Their ages varied from 33 to 63 years, the average age was 51.6 years. All participants got a voucher worth a free night in TCnsberg harbor (approx. 17 C) for their participation. All users had visited TCnsberg before, but only four of them had seen or used the map of TCnsberg before the tests. The users were quite experienced with maps in general and eight of them said that they used maps often, especially sea maps. Seven of 10 participants rated themselves as being professional or skillful map users.
27.6.2
Apparatus
The tests were performed with an IBM ThinkPad 240 laptop running a mobile phone emulator at a screen resolution of 800 • 600pixels. Figure 27.7 shows a user navigating in the city (top) and a closer look to the emulator window (bottom). We used an emulator in the test because we were interested in a user feedback on 3D maps before optimizing the application for the mobile phone hardware. However, the application including the 3D map was later ported to the Nokia Communicator, a high-end mobile phone featuring a large color screen. As shown in Figure 27.7, the application showed both a 3D view of the city and a 2D map of the same area in a split window. Using that application was one test cage in the experiment. Another case involved using a static 2D paper map (without a laptop).
27.6.3
Procedure
Each test session consisted of three parts: introduction, test tasks, and interview. In the first part, the project and the application were introduced to the user. The user filled out a questionnaire that requested some basic data about age, sex, education and prior knowledge about maps, and about the area. At the end of the introduction phase a rough outline of the test was explained to the user. The test part included six similar tasks. In all of them, the user was asked to go from one place to another. In the first four tasks, the participants were asked to use the application and its combined 2D/3D map. In the remaining two tasks users were asked to use a normal 2D paper map. We decided to emphasize the amount of tasks with combined 2D/3D map, because virtually all users are familiar with normal 2D paper maps and thus already know how to read this kind of map. Starting and target locations were marked in the maps, but no GPS was available to continuously update the user's current position on the map.
Presenting Route Instructions on Mobile Devices
543
Figure 27.7. Evaluating 3D maps: The white house in the middle was found easy to recognize. The color of the arrow was highlighted in red for legibility in print.
Each test session ended with an interview, in which the users' opinions about the application, 2D and 3D maps in general and the test session were collected. In addition, users were asked to fill out another short questionnaire.
27.6.4
Results
In general, the users' attitudes towards the prototype were very positive: three quarters of them would like to use this kind of service rather than 2D paper maps and guidebooks. The 3D map itself was believed to be a good idea, although many experienced map users thought that an electronic 2D map would be sufficient for them. There were two major problems in the tests: the laptop screen was hard to see in the sunlight and the users had to position themselves in the model using the keyboard because no GPS was available. Both of these issues influenced the satisfaction ratings returned in the experiment.
544
V. Coors, C. Elting, C. Kray & K. Laakso
Seven of the l0 users were observed to use the 3D map in the application as a navigational aid in the tasks. All of them used it to recognize buildings, and mostly successfully. Some users claimed that non-textured buildings were hard to distinguish from each other, but textured buildings, especially one white house (see Figure 27.7), were considered easy to recognize. Some users complained that the comparison between the 2D and 3D maps in the application was difficult, because there was no clear correspondence between them. Apart from matching buildings the most common navigation strategy for users was to follow the direction arrow in the 3D view as well as the target location and current location being displayed in the 2D map. The users could also choose the viewing height in the 3D view to switch between walking level (pedestrian view, 1.8 m altitude) and flying level (bird' s-eye-view, 25 m altitude). Interestingly, the flying mode was found to be much easier for navigational purposes. 3D maps were found to be slower to use, both in initial orientation and route finding compared to paper 2D maps. We defined the orientation interval to begin at the moment the user was shown the target location and to end when she started to walk towards it. Then the route finding interval began and lasted until the user reached the target. An average orientation interval lasted 42 s when the users used the 3D map, and 10 s when the 2D paper map was in use. Route finding intervals also depend on the lengths of the routes, so it is more appropriate to measure average travel speed for a route than absolute time intervals. Therefore, we rely on the optimal route length divided by the average time, which was about 1 m/s for the 3D map and 1.5 m/s for the 2D paper map. A t-test returned a p-value of 0.023 for the differences between orientation times and 0.001 for the differences between route speeds. This suggests that both results are relevant. When the users were asked how they would like to improve the application, four things were mentioned frequently. According to the users, the 3D model should be more detailed and realistic and the target should be highlighted in it. Street names should be visible and a zoom function should be included in the 2D map.
27.6.5
Discussion
The purpose of this pilot study was to gain an initial understanding of the factors that influence route planning and following performance with mobile devices. The results will be used to improve future prototypes of our navigation system. The small number of test users and the fact that the choice of participants was not random, suggest that the results of this study cannot be generalized. There might also be a potential order effect in the results, because all users performed first the tasks with the 3D map and only then the tasks with the 2D paper map. However, all users started with different tasks and completed different tasks with 3D and 2D maps (four with a 3D map and two with a 2D paper map). Therefore, we collected 40 samples for 3D maps and 20 samples for 2D maps, which allows us to do some simple statistical analysis to the results. However, the most important outcome of
545
Presenting Route Instructions on Mobile Devices
the experiment was to collect some input about how useful and usable 3D maps would be in city navigation. It should also be noted that the majority of the users were males and all of them were experienced with 2D paper maps. There is some evidence that navigation strategies vary between males and females (Hunt and Waller, 1999; Deakin, 1996) and it may well be that females, who are (perhaps) not as accustomed to using maps, would have found 3D maps with landmarks more useful. Despite the observation that experienced male users preferred the familiar 2D maps to the new 3D maps, the results were promising. The users were able to recognize real world objects from the 3D model and to use these landmarks as navigational aids. Many users also said that even though a 3D map would not give them much additional value, it was more fun to use. However, there was certainly a novelty factor involved, and further studies are needed to determine whether this judgment will prevail after many uses. Another interesting result was that users generally preferred the flying mode to the walking mode when using the 3D map. The flying mode gave them a better overview of the surroundings and helped them in building recognition. The results also indicate that the contrast of the 3D model (especially the 3D arrow) has to be improved in order to make the application usable in sunlight. There should also be a better correspondence between the 3D and the 2D map in order to assist users in switching between both systems of reference as discussed by Fuhrman and MacEachren (2001).
27.7
Selecting a Presentation
We can evidently generate a broad range of different presentations from a single preverbal message. However, it is also important to derive some guidelines for which presentation to choose for a given situation. The field test described above certainly shows that there are differences that are not only related to performance but also to perception of the users: Even though most candidates were much faster using 2D maps, they all noted 3D visualizations were "more fun". Table 27.1 lists the different presentation types that we reviewed in this chapter, and highlights their relationship to positional information and technical resources. Obviously, the more sophisticated a presentation is, the more technical resources it Table 27.1. Relationship among presentation type, positional information, and resources -
an overview. Presentation
Location
Orientation
Technical resources
Text Speech 2D sketches 2D maps 3D visualization
Required Required Required Not required Not required
Required for turns Required for turns Not required Not required Not required
Low Low-medium Low-medium Medium-high High
546
V. Coors, C. Elting, C. Kray & K. Laakso
requires. However, the same does not necessarily apply in the case of cognitive resources. While textual and spoken instructions may be easy to understand for a human user (depending, e.g., on the complexity of the content), they include little context. Furthermore, to include additional context (such as information about the surrounding area) either means to add more text, which may quickly increases the cognitive load, or to generate a multi-modal presentation. Due to the lack of context, the current location of the user has to be known rather precisely in order to generate verbal instructions. Turning instructions, for example, do rely on the precise knowledge about the user's current orientation. The same is true for other directional information. While it is possible to compensate for varying quality of positional information using induced frames of reference (Baus and Kray, 2002), the resulting utterance may be more demanding in terms of cognitive resources, as the listener has to perform one or more mental (or physical) rotations/translations. 2D sketches are similar to verbal presentations as they facilitate understanding since they leave out the "unnecessary" and focus on information that is immediately relevant to the current task. Effective sketches require sophisticated algorithms to be generated on the fly (Agrawala and Stolte, 2001). Once generated, only limited technical resources are necessary to display the sketches, as only simple geometrical forms have to be drawn. If the presentation is constantly aligned to the user' s current view direction, the continuous redrawing entails a higher consumption of technical resources. Generating 2D maps is a process that consumes significantly more technical resources than verbal output or 2D sketches. Depending on the implementation, a map has to be generated from a large dataset (or to be clipped from a larger image), which may include the selection of an appropriate zoom factor, of a level of detail, and of which objects to depict. Placing labels on the corresponding map is also a computationally demanding task. And if the map is aligned to the user' s current view direction, it has to be rotated continuously (with label positions recalculated on the fly). However, a map does provide much more context than verbal instructions as it naturally includes nearby objects. Consequently, maps can help to compensate for imprecise or missing positional information. If the view direction of the user is unknown, the map can be rendered using canonical directions. If her location is imprecise, the depicted area can be enlarged. The map also allows the user to compensate for missing positional information, albeit at the price of a potentially higher cognitive load (as anyone knows who has tried to navigate in a foreign city with nothing but a paper map). 3D visualization can be even more demanding in terms of technical resources than 2D maps. The results from our field test do not indicate that they lead to better navigation performance. However, since 3D maps are just being introduced as a navigation aid and 2D maps have been used for centuries, it may just be a design/symbolization difference - 3D might be better, if the design is as good as the best current 2D maps. But, of course, arriving at the ideal design for dynamic 3D maps to support navigation might take some years of experimentation. One main advantage of 3D maps is that they successfully support the recognition of landmarks if they are sufficiently detailed. Additionally, there may be an aesthetic factor that should not be underestimated.
Presenting Route Instructions on Mobile Devices
547
From the above discussion, we can draw several guidelines concerning the selection of presentations for route instructions. Spoken instructions have the unique advantage of not requiring visual attention, but rely on precise positional information. Written instructions are the least demanding as far as technical display resources are concerned, and can be combined with all other presentation types. Both written and spoken instructions are not well suited to providing context. This is especially true for audio, due to its sequential nature. In time critical applications also the complexity of algorithms to generate verbal instruction on the fly has to be taken into account. Route sketches are a good compromise in terms of resource consumption and dependency on positional information. Since only a few very simple forms have to be drawn, they are well adapted to the small display size of a mobile device. They may be easily understood, but do offer little context, and providing additional context comes at the cost of increased complexity. 2D maps on the other hand naturally incorporate contextual information, and are therefore a good means to address situations, where only imprecise positional information is available and the user needs support in the orientation stage. Additionally, they are a well-known tool for navigation, and may help an untrained user to familiarize herself with the system. However, during the route following stage, 2D maps are less helpful than verbal or sketched presentations. In addition, due to high demands on technical resources, 2D maps are not well suited for situations in which a fast response is required. The same applies even more in the case of 3D visualizations. The more realistic the visualization, the more demanding it is in terms of technical resources. But the resulting realism may also enable the user to recognize objects from the rendering in the real world, which is a distinct advantage over all other means of presentation. This makes 3D visualizations well suited for situations where time and technical resources are not an issue, and where the available positional information is somewhat imprecise: The realistic presentations of 3D allow the user to search her environment visually for specific objects, and then to align herself accordingly, thereby compensating for the imprecision. Additionally, our field test hints at another good application for 3D visualizations: entertainment. On a leisurely sightseeing tour, it may add to the enjoyment of a tourist, and it may be beneficial when planning such a tour (e.g., over the internet).
27.8
Conclusions
So far, we have only discussed technical and cognitive resources. However, the user's preferences and abilities are further important factors in the context of selecting a presentation. For example, there is empirical evidence that the age and gender of the user have an impact on the way spatial problems such as wayfinding are solved (Fontaine and Denis, 1999; Jenkins et al., 2000). Currently, we are taking these into account only during the generation of the pvm, but we plan to include them in the process of generating a presentation as well. Route instructions can take various forms, and we reviewed several of them for a mobile user. We introduced an abstract format that allows the generation of many
548
V. Coors, C. Elting, C. Kray & K. Laakso
different presentations including 3D visualizations. For the latter ones, we introduced a novel compression algorithm that makes the use of 3D models on mobile device more feasible. Based on a field test and a thorough analysis of the relationship among presentation type, positional information, and technical and cognitive resources, we presented several guidelines for the selection of a presentation in a given situation. In future, we intend to apply this knowledge within the design-evaluation cycles of the different prototype systems described in this chapter.
Acknowledgements The authors were supported by the Klaus Tschira Foundation (projects SISTO and VisualMap), the German Federal Ministry for Education and Research (project Embassi, grant 01 IL 904 D/2) and the European Union (project TellMaris, IST-2000-2824). This work is based on an earlier work: Presenting Route Instructions on Mobile Devices in International Conference on Intelligent User Interfaces, Conference Proceedings 2003, Miami: ACM Press, 9 ACM 2003 (see Kray et al., 2003).
References Agrawala, M., and Stolte, C., (2001) "Rendering effective route maps: Improving usability through generalization", SIGGRAPH 2001, Los Angeles, pp. 241-249. Andr6, E., and Rist, T., (1995) "Generating coherent presentations employing textual and visual material", AI Review, 9, 147-165. Baus, J., and Kray, C., (2002) "Frames of reference, positional information, and navigational assistance", Proceedings of Flairs 2002, Pensacola Beach, FL. Baus, J., Krtiger, A., and Wahlster, W., (2002) "A resource-adaptive mobile navigation system", International Conference on Intelligent User Interfaces IUI02, San Francisco, January 13-16, 2002. Bernsen, N. O., (2001) "Multimodality in language and speech systems - from theory to design support tool", In: Granstr6m, (ed.), Multimodality in Language and Speech Systems. Kluwer: Academic Publishers. Cartwright, W., (1999) "Extending the map metaphor using web delivered multimedia?", International Journal of Geographical Information Systems, 13(4), 335-354. Cohn, A. G., (1996) "Calculi for qualitative spatial reasoning", In: Calmet, J., Campbell, J. A., and Pfalzgraf, J., (eds.), Artificial Intelligence and Symbolic Mathematical Computation. Berlin: Springer, pp. 124-143. Coors, V., and Rossignac, J., (2003). "Delphi encoding: improving edgebreaker by geometry based connectivity prediction", GVU Technical Report GIT-GVU-03-30, Georgia Institute of Technology, Atlanta, USA. Deakin, A., (1996) "Landmarks as navigational aids on street maps", Cartography and Geographic Information Systems, 23(1), 21-36.
Presenting Route Instructions on Mobile Devices
549
Elting, C., Zwickel, J., and Malaka, R., (2002) "Device-dependant modality selection for user-interfaces - an empirical study", International Conference on Intelligent User Interfaces IUI02, San Francisco, (January 13-16, 2002). Fontaine, S., and Denis, M., (1999) "The production of route instructions in underground and urban environments", In: Freksa, C., and Mark, D. M., (eds.), International Conference on Spatial Information Theory: Cognitive and Computational Foundations of Geographic Information Science. Berlin: Springer, pp. 83-94. Fuhrman, S., and MacEachren, A. M., (2001) "Navigation in desktop geovirtual environments: usability assessment", 20th International Cartographic Conference, Beijing, China, pp. 2444-2453. Hartmann, K., Krtiger, A., Schlechtweg, S., and Helbing, R., (1999) "Interaction and focus: towards a coherent degree of detail in graphics, captions and text", In: Lorenz, P., and Deussen, O., (eds.), Simulation and Visualization '99, SCS Europe, Erlangen, Ghent, pp. 127-138. Hunt, E., and Waller, D., (1999) "Orientation and wayfinding: a review", ONR Technical Report N00014-96-0380. Arlington, VA: Office of Naval Research. Jenkins, L., Myerson, J., Joerding, J. A., and Hale, S., (2000) "Converging evidence that visualspatial cognition is more age-sensitive than verbal cognition", Psychology and Aging, 15(15). Keim, D. A., and Kriegel, H.-P., (1994) "VisDB: database exploration using multidimensional visualization", Computer Graphics and Applications, 6, 40-49. Kray, C., and Blocher, A., (1999) Modeling the basic meanings of path relations, Proceedings of the 16th IJCAI. San Francisco, CA: Morgan Kaufmann. Kray, C., Laakso, K., Elting, C., and Coors, V., (2003) Presenting route instructions on mobile devices, International Conference on Intelligent User Interfaces 2003. Miami, Florida: ACM Press, pp. 117-124. Krtiger, A., (2000) Automatische Abstraktion in 3D-Graphiken, Dissertationen zur Kfinstlichen Intelligenz, Vol. 232, Berlin: Aka Akademische Verlagsgesellschaft Aka Gmbtl, 232 pp. in German. Laakso, K., (2002) Evaluating the use of Navigable Three-Dimensional Maps in Mobile Devices, Master Thesis, Department of Electrical and Communications Engineering, Helsinki University of Technology. Malaka, R., and Zipf, A., (2000) "DEEP MAP - challenging IT research in the framework of a tourist information system", Proceedings 7th International Congress on Tourism and Communication Technologies in Tourism (ENTER 2000), Barcelona, Spain. Rakkolainen, I., Timmerheid, J., and Vainio, T., (2000) "A 3D city info for mobile users", Proceedings of the 3rd International Workshop in Intelligent Interactive Assistance and Mobile Multimedia Computing (IMC'2000), Rockstock, Germany, pp. 115-212, (November, 9-10, 2000). Rossignac, J., (1999) "Edgebreaker: connectivity compression for triangle meshes", IEEE Transactions on Visualization and Computer Graphics, 5(1), 47-61, Jan-Mar 1999.
550
V. Coors, C. Elting, C. Kray & K. Laakso
Rossignac, J., Safonova, A., and Szymczak, A. (2001) "3D compression made simple: Edgebreaker on a corner-table", Shape Modeling International Conference, Genoa, Italy, May 2001. Schirra, J., (2000) "Bildbeschreibung als Verbindung von visuellem und sprachlichem Raum", Eine interdiszipliniire Untersuchung von Bildvorstellungen in einem H6rermodell, Infix, St. Augustin, Online: http://www.computervisualistik.de/--schirra Touma, C., and Gotsman, C., (1998) "Triangle mesh compression", Proceedings Graphics Interface 98, pp. 26-34. Wahlster, W., (1998) "Intelligent user interfaces: an introduction", In: Maybury, M., and Wahlster, W., (eds.), Readings in Intelligent User Interfaces, Morgan Kaufman Press.
Section E Making Useful and Useable Geovisualization Design and Evaluation Issues
28. Making Useful and Useable Geovisualization: Design and Evaluation Issues Fuhrmann et al ..........................................................................................................................................553 29.3D Geovisualization and the Structure of Visual Space Colin Ware, Matthew Plumlee ................................................................................................................... 567 30. Applications of a Cognitively Informed Framework for the Design of Interactive Spatio-temporal Representations Robert M. Edsall & Laura R. Sidney ........................................................................................................ 577 31. User-centered Design of Collaborative Geovisualization Tools Sven Fuhrmann & William Pike ................................................................................................................ 591 32. Towards Multi-variate Visualization of Metadata Describing Geographic Information Paula Ahonen-Rainio & Menno-Jan Kraak ............................................................................................... 611 33. Evaluating Self-organizing Maps for Geovisualization Etien L. Koua & Menno-Jan Kraak ........................................................................................................... 627 34. Evaluating Geographic Visualization Tools and Methods: An Approach and Experiment Based upon User Tasks Carolina Tob6n ...........................................................................................................................................645 35. Cognitively Plausible Information Visualization Sara Irina Fabrikant & Andr6 Skupin ........................................................................................................ 667
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 28
Making Useful and Useable Geovisualization: Design and Evaluation Issues Sven Fuhrmann, Department of Geography, GeoVISTA Center, The Pennsylvania State University, 302 Walker Building, University Park, PA 16802, USA Paula Ahonen-Rainio, Institute of Cartography and Geoinformatics, Department of Surveying, Helsinki University of Technology, PO Box 1200, Espoo, FIN-02015 HUT, Finland Robert M. Edsall, Department of Geography, Arizona State University, PO Box 870104, Tempe, AZ 85287, USA Sara I. Fabrikant, UC Santa Barbara, Department of Geography, 3611 Ellison, Santa Barbara, CA 93106, USA Etien L. Koua, Department of Geo-Information Processing, International Institute for Geo-Information Science and Earth Observation, PO Box 6, 7500 AA Enschede, The Netherlands Carolina Tob6n, University College London, Centre for Advanced Spatial Analysis, 1-19 Torrington Place, Gower Street, London WC1E 6BT, UK Colin Ware, Center for Coastal and Ocean Mapping (and Computer Science Department), University of New Hampshire, 24 Colovos Road, Durham, NH 03824, USA Stephanie Wilson, Centre for HCI Design, City University, Northampton Square, London EC 1V 0HB, UK
Keywords: usability, usefulness, user-centered design, human-computer interaction, geovisualization
Abstract The design of geovisualization tools is not only a technical research question. For many years geovisualization tool design was largely technology driven, where system designers and final users were mostly one and the same. Nowadays geovisualization tools are applied in and developed for a broader geosoftware market with the goal of providing useful and usable geovisualization. Sometimes this goal is not reached for many reasons, resulting in frustrated users and unsolved tasks. The aim of this overarching chapter is to give an introduction into methods and research questions on user-centered geovisualization tool design, bridging the gap between developers 553
554
S. Fuhrmann et al.
and users. In order to stimulate the development of geovisualization theory, the authors of this chapter contribute their views and discuss issues from Computer Science, Information Visualization, Geoinformation Science, Geography and Cartography.
28.1
Introduction
Influenced by recent developments within the Human-Computer Interaction (HCI) community developers of geovisualization environments are becoming increasingly concerned with the usability of their tools. Some of the key questions are whether geovisualization approaches are indeed effective for spatial problem solving, and if the novel tool designs are actually usable and useful for knowledge discovery and decision making? From a research perspective, geovisualizers have become interested in borrowing HCI approaches and applying them to their visualization efforts, as to ensure the usability of their geovisualization tools before they are released (Slocum et al., 2001; Fuhrmann, 2003). It has become clear, however, that constructing effective geovisualization tools and designing novel graphic displays is not just a simple matter of knowledge transfer from HCI to geovisualization. The questions "what can the relatively new field of geovisualization learn from HCI research?" and "how to design useful and useable geovisualization?" reveal a range of multidisciplinary research issues that will be highlighted in this and the following chapters. The International Organization for Standardization (ISO) defined "usability" as "the extent to which a system can be used by specified users to achieve specified goals with effectiveness [the extent to which a goal is reached], efficiency [the effort to reach goals], and satisfaction [the user's opinion on system performance] in a specified context of use" (ISO 9241-11, 1998). This definition may be of benefit when identifying assessment measures of system usability (e.g., how fast a user is able to perform a task), but it might be too vague when assessing whether and how a tool can help solving a particular research problem. The ISO usability definition is mostly grounded in HCI and ergonomic workstation design research (Dix et al., 1998). In HCI, primary attention is often given to the optimal modeling of a system. In geographic, statistical, and information representation, the focus is more on the design of a representation to support the analysis of phenomena represented. Mark and Gould (1991) cited this distinction over 10 years ago, just as designers of GIS began considering HCI research: "Instead of interacting with a computer peripheral or its user interface, GIS users should be able to interact more directly with geographic information and geographic problems. A focus on human-problem or human-phenomenon interaction will better enable design and implementation of optimal user interfaces for GIS and related software". Over the years, usability engineering has provided a wealth of usability assessment methods for components and tools (Nielsen, 1993). Currently, the application of these methods is intensified in geovisualization design and developers
Making Useful and Useable Geovisualization: Design and Evaluation Issues
555
need to be aware that these often system-focused HCI methods might not distinguish between useful and usable. On the one hand, a geoscientist may argue for the parallel coordinate plot (PCP) as a useful approach to extract discrete multi-variate structures from a multi-variate geographic dataset. On the other hand, it may not be usable for a novice, because of its apparent semantic complexity, due to its novelty, and because of its graphic limitations such as overplotting. Although usability engineering partly borrows empirical principles and approaches from cognitive psychology (Lewis and Rieman, 1994; Landauer, 1995), its goals are typically of pragmatic nature. Usability evaluation is often restricted to an assessment of how well users may master a series of known or defined tasks with a particular interface component or tool, and/or are able to understand the conceptual model of a system to achieve the goal. It has been shown that user effectiveness, efficiency, and satisfaction measurements retrieve important usability information if the tasks for reaching a goal are well defined (Nielsen, 1993; Lindgaard, 1994). Operating airline booking systems or ATMs are examples of well-defined task structures. The user needs to proceed in predefined steps to retrieve a ticket or money from a system. Thus, usability evaluation uses the characteristics of the defined tasks and often applies detailed scenarios to measure users' successes in working with computer-based tools. In geovisualization, however, abductive data exploration and knowledge discovery use scenarios are typically ill defined, thus goal achievement becomes difficult to measure. We need to assess additional (mostly qualitative) information and ask: Is this user interface or tool useful? does it support the users' ability to understand the characteristics of the data represented? Does it allow new information to be extracted or spatial problems to be solved by interacting with the data? Thus, usefulness is often hard to measure quantitatively. In geovisualization, it expresses how well, for example, a geovisualization tool supports users in generating an appropriate model of the geospatial structure or phenomenon being investigated and to solve a research problem. The relationship of users' individual differences, such as their cognitive abilities, their socio-demographic profile, their individual knowledge base (e.g., background and training), and their understanding of the underlying depiction framework embedded in geovisualization tools is often not systematically assessed with usability engineering techniques during tool development (Slocum et al., 2001). When evaluating geovisualization tools it is sometimes difficult to clearly distinguish between usability engineering for improving the design of a tool, from formally testing a theoretical framework employed for depiction. Evaluating existing geovisualization tools or components on their usefulness and usability can only be considered as one part in the geovisualization tool design process. Often, "last minute" evaluations of software tools reveal major flaws that might bring a project back to its early conceptual stages. In order to avoid timely and costly tool developments, approaches to user-centered design are undertaken that utilize usability evaluations at an early stage. Efforts towards user-centered geovisualization design and elevation methods that can accompany the complete development cycle to ensure usable and useful geovisualization tools are introduced in w and w
556
28.2
S. Fuhrmann et al.
User-Centered Design Approach to Geovisualization
The development of geovisualization tools has often been limited to the domain of research, and frequently the system designer and the final user have been the same person: the innovator. Norman (1998) notes that in any domain each of five possible users categories - innovators, early adopters, pragmatists, conservatives and skeptics - have specific preferences and goals that need to be considered when designing software. Currently, geovisualization tools are evolving from the instruments developed by innovators and used by early adopters to the broader audience of pragmatists and conservatives and the range of possible user domains and tasks should be reflected in their design. Geovisualization designers are aware of some of the issues this may raise and have began to address them as discussed above and in Andrienko et al., this volume (Chapter 5). In retrospect, geovisualization tool development, as well as other interactive software development, has been largely technology driven. Software engineers have defined concepts for tools following the latest possibilities of technology. With finished concepts in mind, they approached users in order to study their tasks and requirements that could be met with the tools to be designed. More recently, a paradigm shift towards user-centered design has occurred and methods have been developed involving user participation from the concept design stage of interactive software development, for example, (see Fuhrmann and Pike this volume (Chapter 31)), Ahonen-Raino and Kraak this volume (Chapter 32), Tob6n this volume (Chapter 34), etc. Modem user-centered design approaches of usability engineering integrate user domain and task reflections, aiming at usable and useful systems. Most user-centered design approaches are built on theories of cognitive psychology and social sciences
Plan the user centered process
Specify the context of use Specify user and organizational requirements
Evaluate design Produce design solutions Product/Prototype requirements
meets
Figure 28.1. The user-centered design process (Bevan and Curson, 1999).
Making Useful and Useable Geovisualization: Design and Evaluation Issues
557
(Hackos and Redish, 1998; Dix et al., 1998), and are under continuing development (Hassenzahl, 2001; Lewis, 2001). Usually, a user-centered design process (Figure 28.1) involves principles that can be described as: 9 9 9 9
28.2.1
set an early focus on users and tasks; apply iterative and participatory design; measure the product usage empirically through user testing; modify the product repeatedly (Gould and Lewis, 1987; Rubin, 1994).
Focus early on users and tasks
One way of setting an early focus on domain users to gather more information is by utilizing the user analysis - an "activity of getting to know the characteristics of people who will later use the software" (Henry, 1998). A user analysis determines several characteristics, for example terminology, task expertise, disability and computer literacy of users and integrates domain expertise into the design process. Methods of learning about users and their needs that could be applicable to geovisualization tool design include unstructured and structured interviews about work situations and attitudes (questions and their sequences are either predetermined or not) and participant observations where users are monitored while archiving a particular working goal (Beyer and Holtzblatt, 1998). Besides user characteristics, the range of domain-specific geovisualization goals and tasks needs to be considered. In general, geovisualization goals can be broken down into four categories: data exploration, analysis, synthesis and presentation (MacEachren and Kraak, 1997; Gahegan et al., 2001; (see also Gahegan, this volume (Chapter 4)). These geovisualization goals can be achieved through a series of tasks, subtasks, decisions, and constraints. For presentation purposes, tasks and goals can often be predicted during system design whereas the range of tasks and their application in exploratory geovisualization are often unpredictable (ill-defined), requiring more flexible systems. Thus, geovisualization tool design ideas should be based on context of use rather than on what is technically possible. This is particularly relevant when the instrument designed is intended for domain users rather than as an innovator's proof-of-concept in order to assure that the design concepts effectively support users' work processes. Context and tasks of domain users are usually assessed with a task analysis. A task analysis is "the breakdown of overall tasks, as given, into their elements, and the specification of how these elements relate to one another in space, time and functional relation" (Sheridan, 1997). It is a multi-disciplinary method that supports evaluating HCI in terms of actions and cognitive processes in relation to user specific goals. About 25 different techniques can be applied during a user task analysis (Kirwan and Ainsworth, 1992). Here we cannot highlight the techniques in great detail, but can list and characterize the most common used for: 9
task data collection (techniques that are used for collecting data on humansystems-interactions) and;
558
S. Fuhrmann et al.
9
task description (techniques that structure the information collected into a systematic format).
Informational and survey interviews are two inexpensive interviewing methods for task data collection (Kirwan and Ainsworth, 1992). Informational interviews are set up to collect a wide range of information on a task situation, while survey interviews have a more specific objective; they may for example review just one task in detail. The main advantage of the method is its natural and direct approach towards the user. In comparison to questionnaires, interviews are flexible. Important information can be documented quickly and later analyzed (Kirwan and Ainsworth, 1992). Since the participant can be highly influenced by those conducting the experiment, the social, interpersonal interaction also contains some limitations (Dix et al., 1998). An alternative method of querying the user is to administer a questionnaire. A questionnaire usually contains a set of predetermined questions and is typically answered in a fixed sequence (Kirwan and Ainsworth, 1992). Questionnaires can be applied during different stages in user-centered design: for example, to find out about tasks of a user group or to measure user satisfaction at the final stage in the designing process (Nielsen, 1993). In addition, questionnaires can be used to reach a wide user group but they are inflexible when compared to interviews since the questions, and more importantly most answers, are fixed in advance. These answers might be restricted to the knowledge of the researcher and diverge from users needs. However, answers of closed questions can be analyzed more rigorously and allow the processing of many responses (Nielsen, 1993; Dix, et al., 1998). In order to include a more user- and use-oriented perspective during geovisualization tool design, scenarios (Kuutti, 1995; Hackos and Redish, 1998) have become a popular method during user-task analysis. Carroll (2000) describes scenarios as "stories - about people and their activities". Usually these stories consist of userinteraction narratives, which are descriptions of what users do and experience as they try to make use of hardware and software (Kuutti, 1995). These user-interaction scenarios are a sophisticated medium for representing, analyzing and planning ways in which new hardware or software might impact user's tasks and experiences (Carroll, 1997). Most importantly, the vocabulary in these narratives is rich in actions, objects and metaphors, supporting their identification and incorporation into user interface design (Fuhrmann et al., 2001).
28.2.2
Describing tasks and concepts
Hierarchical task analysis (HTA) takes the results of the above task data collection techniques and describes the identified tasks and goals, placing emphasis on human abilities and system usability. HTA is directed towards decomposing a process into a hierarchy of operations and plans with instructions and constraints. Operations describe the basic tasks and subtasks of users, while plans display the condition statements that are necessary to execute operations (Dix et al., 1998; Hackos and Redish, 1998). The HTA can be graphically represented as a hierarchical diagram or in tabular form (Shepherd, 1995). In user interface design, HTA has many advantages because it is an economical
Making Useful and Useable Geovisualization: Design and Evaluation Issues
559
method for gathering and organizing processes. In addition, it focuses on specific tasks within the context of an overall goal (Kirwan and Ainsworth, 1992). Since HTA identifies known tasks, ill-defined tasks, such as the goals of data exploration, might not get recognized. This limitation needs to be accounted for when applying the HTA during a user interface design process. The shortcoming of describing ill-defined tasks for exploratory geovisualization might be overcome with the help of more participatory design methods. In order to achieve this goal, close contact between users and designers is emphasized in understanding the future context of tool use. Descriptive information acquired from users tends to generalize details that may prove to be important in design. Methods such as story telling have been developed to address this (Hackos and Redish, 1998; Erickson, 1995). In story telling, users are asked to recount and describe critical incidents that they recall related to the particular phenomenon under study. Stories give subjective, ambiguous and individual user views but as such they can be a valuable consideration in any design process of an exploratory nature. They may reveal more information about users and their sophisticated and sometimes abstract needs than generalized, objective descriptions (Erickson, 1995). One of the challenges in designing geovisualization tools through user-centered methods is often the genetic nature of the tools. The motivation for designers to invest in user requirement studies may be limited by the fact that intended users of the tools are frequently an ill-defined set of individuals. Even in this scenario, a designer would be advised not to operate at the "general user" level but to sample different geo-domain users in order to obtain different elements of input to the context of use and user needs. However, user-centered design methods are often criticized for being time consuming (Nielsen, 1993), since involving users in the early design process stages increases the complexity of the design task. The benefits of an early emphasis on usability evaluation are discussed in w as various methods can be used and adapted to obtain information about a broad range of system and user aspects.
28.3
Dimensions of Geovisualization Evaluation
As part of the aim to make users the focus of the design cycle, the geovisualization community recognizes the need to evaluate its artifacts, yet the goals of such endeavors are not always clear. A fundamental driver of any evaluation activity is to identify aspects of a system that are less than optimal and have the potential to be improved in a redesign effort. Hence the importance of asking ourselves "why are we evaluating?" and "what is the purpose underlying the evaluation?" Within the field of HCI, this type of formative evaluation is commonly carried out by usability practitioners as part of ensuring the usability of interactive systems. Usability evaluation allows us to obtain data, often quantitative, about aspects of a system or the users' performance with that system that may be used for identifying aspects that are problematic for the user and to highlight potential fixes. These methods can also be used for comparison purposes, for example, against established benchmarks or against alternative designs or products, in order to
560
S. Fuhrmann et al.
identify which is easier to use or to learn, or to identify their relative advantages and disadvantages. Usability evaluation, however, can also be more exploratory, reflecting the nature of geovisualization. This purpose is to understand more about users' tasks and goals - how and why they are employing geovisualization. Hence, evaluation can contribute to the understanding of research questions such as the type of tasks geovisualization systems should support. Clearly, the eventual goal here is to provide better support for the users' work, so this is closely related to the improving of systems. In practice, evaluations may be multi-facetted, serving several of the purposes listed above.
28.3.1
Dimensions of usability evaluation
The aim of evaluation may vary depending on the stage of the design and system lifecycle at which the evaluation is conducted. For instance, usability evaluation can be conducted to investigate a concept to be embedded in the design or it can be part of the implementation process. In general, usability evaluation investigates the functionality of the tool in terms of its ability to support user tasks, examines the interface in terms of how its features support user tasks and needs and assesses the way the tool accommodates different user operations. But it can also involve other aspects of the design and use of the artifact such as its effectiveness or perceived user satisfaction, which may be assessed against some level of expectation. These three dimensions of usability evaluation are considered here: the stage of the development cycle where the evaluation is conducted, the artefact to be evaluated, and the approach used to evaluate the tool.
The stage where usability evaluation takes place One dimension of usability evaluation is the stage at which it is conducted. Depending on the stage in the system lifecycle, the evaluation can be carried out on a design concept, a design specification, a prototype or a fully functional system. It can be conducted as part of the design and development cycle, as a test during implementation or as a final assessment to understand the behavior of the tool and the users. In the early stages, the main purpose of an evaluation is often to examine the effectiveness of preliminary design concepts. For example, while designing geovisualization tools for a specific user domain, the contextual limitations may be revealed at the concept design stage through scenarios or paper prototypes. These fairly simple techniques allow communicating design concepts to users at a draft level. Erickson (1995) states that working prototypes should be at a level of robust drafting to encourage users to give feedback to designers at an early stage. The "lowfidelity" prototypes may be constructed from standard office materials and then used where a member of the design team manipulates the prototype in response to user actions in order to convey the interactivity of the system. This technique elicits user feedback at low cost and without interference from detailed considerations such as graphic design.
Making Useful and Useable Geovisualization: Design and Evaluation Issues
561
In later stages of the development cycle, usability evaluation can be conducted to assess the usability of the product itself. This type of usability test may again be formative, focusing on usability problems, such as effectiveness and efficiency that can be detected by expert users through techniques such as heuristic evaluation (Nielsen, 1993), or it may assess how the product compares to some predetermined usability standard. In the latter case, the test objectives are related to performance criteria such as speed and accuracy, how well and how fast the user can perform various tasks and operations, and how well all the components of the product work together. Techniques such as logging, which involves the automatic collection of statistics about actual system use, are helpful for gathering detailed data of how users perform their work once a system or prototype has been developed.
The artifact to be evaluated The second dimension, related to the above, is the nature of the artefact to be evaluated. It is not necessary to have an interactive, fully functional system to conduct an effective evaluation (see in "The stage where usability evaluation takes place" in w Rettig, 1994). In geovisualization, researchers typically need to evaluate a concept or a prototype implementation. These prototypes can be used during system development for communicating with users. In order to evaluate interactivity (e.g., navigation through a virtual environment) and to obtain accurate performance measures m such as time taken by a user to complete a specific task, their success or error rates, evaluation with a functioning software prototype is required (Andrienko et al., 2002). An artifact under evaluation may be a full system (either as a prototype or a functioning system) or a component of it. Evaluation of a full system allows us to consider interaction between various components but effectiveness of each component may be easier to assess by evaluating them individually.
Approaches to usability evaluation Usability evaluation can be undertaken using a number of approaches according to whether it is user, design/system expert, or theory based (Sweeney et al., 1993). A userbased evaluation involves users completing tasks in the environment whereas expertbased evaluation involves evaluators using the system in a more structured way in order to determine whether the system corresponds to predefined design criteria and some general human factor principles. These techniques commonly referred to as "usability inspection methods" (Nielsen and Mack, 1994), are not widely used for geovisualization at present. In a theory-based evaluation, a designer or evaluator can assess the match between user tasks that need to be supported and the system's specification to generate quantitative values on learnability or usability. The methods used also depend on the type of data that needs to be collected either to improve particular aspects of a system or for research purposes. There are usability evaluation methods to gather both qualitative and/or quantitative information and they are commonly combined in order to obtain complementary data. For instance, performance measurements are recorded and commonly analyzed using a statistical
562
S. Fuhrmann et al.
method to detect trends or usability problems with the system. These measurements, however, are often of greater value when accompanied by supplementary information from users about their perceptions of the usefulness of the system. Subjective data of this form is typically gathered using techniques such as interviews or questionnaires. An example for an important assessment method is the "thinking aloud study". The method was developed in order to investigate which human cognitive processes occur during problem solving. The technique was transferred into HCI research where it is often applied. While working with a software tool, participants are asked to verbalize their thoughts as they try to solve a particular task. Participants usually do not only report how they solve a particular task but also include information about their perceptions and feelings, such as fear and anger (Weidle and Wagner, 1994). Additionally, participants often subjectively comment on the prototype, which supports the identification of flaws and errors in the user interface. Thinking aloud results in data describing cognitive processes of which the participant is aware. Other processes might not be identified using this technique, since not all mental processes can be verbalized directly (Kirwan and Ainsworth, 1992; van Someren et al., 1994). Therefore, a clear understanding of the purpose and aim of an evaluation helps in determining the methods to be employed and the data to be collected. However, practical considerations, such as the cost in time or money of particular techniques, can also be influential when choosing specific methods and planning an evaluation. The broad range of techniques discussed in w and w are but a sample of what is available. Furthermore, these methods can be customized so that we can address particular research questions through a well planned evaluation.
28.4
Discussion: Do We Need a Geovisualization Theory?
Currently, most geovisualization is still arrived at through a design process, based on accumulated experience codified in procedures, written design rules and unwritten individual and group knowledge. However, more formal theory can contribute to design guidelines, and the long-term payoff is design that is more likely to be valid across different applications of geovisualization and across culturally different user groups. Theory development is a long-term process; it takes enormous effort to carry out human studies to answer small questions about some part of the user interface to information. Nevertheless it can be worth the effort because the results are potentially lasting. Geovisualization theory can be divided into two broad categories: that which comes from other disciplines such as Perceptual Science, Cognitive Science, or applied disciplines such as Human-Computer Interaction. There is also theory developed specifically in the context of geovisualization. Although theory may originate from some other disciplines, the role of geovisualization researchers will be to extend it in ways that are specific to geovisualization. A good example of this is the large body of work that has been carried out on pseudo-color sequences. This concerns the way in which information variables, such as temperature, population density and the like can be best expressed on a map using color. The theory of perception suggests that using a luminance scale will express
Making Useful and Useable Geovisualization: Design and Evaluation Issues
563
a monotonic (continuously increasing) variable most successfully, although saturation (vividness) of color will also work. The theory also suggests that a representation of the physical spectrum (relying upon color hue) will not be perceived as monotonic. These predictions have been shown empirically to apply to the problem of pseudo coloring. The result is a body of theory and a set of design guidelines specifically tailored to the needs of geovisualization (Ware, 1988; Brewer, 1994). A key area of cognitive theory concerns the way that people use external imagery as a support in decision making (Zhang, 1997). Instruments for geovisualization can be regarded (amongst other things) as being cognitive decision support tools. People can cognitively operate much more effectively with an external artefact, such a map, than with a purely mental image. For example, maps are often essential tools in planning. We can cognitively use the map for rapid "what if?" scenarios when planning travel, for example, "what if I were to take this route rather than that?". With geographic information systems behind the geovisualization, interactive maps can provide far more powerful cognitive support tools that take advantage of all sorts of sophisticated data manipulations. Cognitive theory can provide insights on how to better design the interfaces used for geovisualization. For example, it is known that visual and verbal working memory have very limited capacity and this can be a major bottleneck to the process of ideation. Modeling this, along with the system characteristics can be useful in answering questions such as whether and when multiple views of data are likely to be useful, for example, (see Ware, this volume (Chapter 29)) and Roberts, this volume (Chapter 8). Ultimately we may hope to develop a kind of extended cognitive theory that encompasses both human cognitive systems and the external computer-based geovisualization system and the twoway flow of information between the two. Such a theory could ultimately help guide the early stages of system design. The evaluation methodologies that are appropriate to theory development are generally more rigorous than those required for usability design. Most methods used in visual science, cognitive science, and social science are potentially applicable. This level of effort is justified because the goal is to generate theories that will endure, rather than the quality of a single system. Because of this, it behooves researchers to be as rigorous as possible and to try to take a long-term perspective. For example, a study of the utility of stereoscopic display (Kirschenbauer, this volume (Chapter 18)) should ideally be undertaken with a very high-resolution display because stereoscopic depth perception is capable of taking into account very small differences in images. Doing the study with a low-resolution display may lead to useful insights, but we can confidently predict that displays will be better in the future. Geovisualization is in its beginnings in terms of the development of a body of established theory. As indicated, much of the theory upon which we currently draw may have origins in other disciplines, but it will have been extended and refined in ways that make it specific to geovisualization. The related disciplines of Scientific Visualization, Information Visualization, Human-Computer Interaction and Cartography certainly have plenty to offer (MacEachren, 1995; Dix et al., 1998; Card et al., 1999; Chen, 1999; Ware, 2000).
564
S. Fuhrmann et al.
References Andrienko, N., Andrienko, G., Voss, H., Bernardo, F., Hipolito, J., and Kretchmer, U., (2002) "Testing the usability of interactive maps in CommonGIS", Cartographic and Geographic Information Science, 29(4), 325-342. Bevan, N., and Curson, I., (1999) "Planning and implementing user-centered design", CHI 1999, Adjunct Proceedings, Pittsburgh, ACM Press, New York, NY, USA. Online: http://www.usability, serco.com/research/publications.htm Beyer, H., and Holtzblatt, K., (1998) Contextual Design - Defining Customer-Centered Systems. San Francisco: Morgan Kaufmann Publishers, p. 472. Brewer, C. A., (1994) "Color use guidelines for mapping and visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography, Vol. 2. Oxford: Elsevier Science Ltd., pp. 123-148. Card, S. K., Mackinlay, J. D., and Shneiderman, B., (eds.), (1999) Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann Publishers. Carroll, J. M., (1997) "Scenario-based design", In: Helander, M., Landauer, T. K., and Prabhu, P., (eds.), Handbook of Human-Computer Interaction. Amsterdam, North Holland: Elsevier, pp. 383-406. Carroll, J. M., (2000) Making Use - Scenario-Based Design of Human-Computer Interactions. Cambridge, MA: The MIT Press, p. 368. Chen, C., (1999) Information Visualization and Virtual Environments. London: SpringerVerlag, p. 223. Dix, A. J., Finlay, J. E., Abowd, G. D., and Beale, R., (1998) Human-Computer Interaction. Englewood Cliffs: Prentice Hall. Erickson, T., (1995) "Notes on design practice: stories and prototypes as catalysts for communication", In: Carroll, J. M., (ed.), Scenario-Based Design, Envisioning Work and Technology in System Development, John Wiley & Sons, Inc., New York, NY, USA, pp. 37-58. Fuhrmann, S., (2003) Facilitating wayfinding in desktop geovirtual environments. Muenster: Institut ftir Geoinformatik, Westf~ilische Wilhelms-Universit/it, p. 177. Fuhrmann, S., Schmidt, B., Berlin, K., and Kuhn, W., (2001) "Anforderungen an 3Dinteraktionen in geo-virtuellen visualisierungsumgebungen", Kartographische Nachrichten, 51 (4), 191-195. Gahegan, M., Harrower, M., Rhyne, T.-M., and Wachowicz, M., (2001) "The integration of geographic visualization with databases, data mining, knowledge construction and geocomputation", Cartography and Geographic Information Science, 28(1), 29-44. Gould, J. D., and Lewis, C., (1987) "Designing for usability: key principles and what designers think", In: Baecker, R. M., and Buxton, W. A. S., (eds.), Readings in Human-Computer Interaction: A Multidisciplinary Approach. San Mateo, CA: Morgan Kaufmann Publishers, Inc., pp. 528-539. Hackos, J. T., and Redish, J. C., (1998) User and Task Analysis for Interface Design. New York: Wiley, p. 488.
Making Useful and Useable Geovisualization: Design and Evaluation Issues
565
Hassenzahl, M., (2001) "The effect of perceived hedonic quality on product appealingness", International Journal of Human-Computer Interaction, 13(4), 481-499. Henry, P., (1998) User-Centered Information Design for Improved Software Usability. Boston: Artech House, p. 250. International Standards Organsiation (1998) "ISO 9241-11Ergonomic requirments for office work with visual display terminals (VDT)s - Part 11", Guidance on Usability. Gen~ve: International Organization for Standardization, p. 22. Kirwan, B., and Ainsworth, L. K., (eds.), (1992) A Guide to Task Analysis. London: Taylor & Francis, p. 417. Kuutti, K., (1995) "Work process: scenarios as a preliminary vocabulary", In: Carroll, J. M., (ed.), Scenario-Based Design: Envisioning Work and Technology in System Development. New York: Wiley, pp. 19-36. Landauer, T. K., (1995) The Trouble with Computers - Usefulness, Usability, and Productivity. Cambridge, MA: The MIT Press, p. 425. Lewis, J. R., (2001) "Current issues in usability evaluation", International Journal of Human-Computer Interaction, 13(4), 343-349. Lewis, C., and Rieman, J., (1994) Task-centered user interface design. A practical instruction. Boulder, CO: Computer Science Department, University of ColoradoBoulder, Online: ftp://ftp.cs.colorado.edu/pub/cs/distribs/clewis/HCI-Design-Book Lindgaard, G., (1994) Usability Testing and System Evaluation - A Guide for Designing Useful Computer Systems. London: Chapman & Hall, p. 393. MacEachren, A. M., (1995) How Maps Work: Representation, Visualization, and Design. New York: The Guildford Press, p. 513. MacEachren, A. M., and Kraak, M. J., (1997) "Exploratory cartographic visualization: advancing the agenda", Computers & Geosciences, 23(4), 335-344. Mark, D. M., and Gould, M., (1991) "Interacting with geographic information: a commentary", Photogrammetric Engineering and Remote Sensing, 57(11), 1427-1430. Nielsen, J., (1993) Usability Engineering. Boston: AP Professional. Nielsen, J., and Mack, R. L., (eds.), (1994) Usability Inspection Methods. New York: Wiley, p. 413. Norman, D. A., (1998) The Invisible Computer. Cambridge, MA: MIT Press. Rettig, M. (1994). Prototyping for tiny fingers. Communications of the ACM, 37(4), 21-27. Rubin, J., (1994) Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: Wiley, p. 330. Shepherd, A., (1995) "Task analysis as a framework for examining HCI tasks", In: Monk, A. F., and Gilbert, N., (eds.), Perspectives on HCI - Diverse Approaches. London: Academic Press, pp. 145-174. Sheridan, T. B., (1997) "Task analysis, task allocation and supervisory control", In: Helander, M., Landauer, T. K., and Prabhu, P., (eds.), Handbook of HumanComputer Interaction. Amsterdam: Elsevier, pp. 87-105. Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., and Hedley, N. R., (2001) "Cognitive and Usability Issues In Geovisualization", Cartography and Geographic Information Science, 28(1), 61-75.
566
S. Fuhrmann et al.
Sweeney, M., Maguire, M., and Shackel, B., (1993) "Evaluating user-computer interaction: a framework", International Journal of Man-Machine Studies, 38(4), 689-711. van Someren, M. W., Barnard, Y. F., and Sandberg, J. A. C., (1994) The Think Aloud Method - A Practical Guide to Modelling Cognitive Processes. London: Academic Press, p. 208. Ware, C., (1988) "Color sequences for univariate maps: theory, experiments and principles", IEEE Computer Graphics & Applications, 8(5), 41-49. Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, p. 384. Weidle, R., and Wagner, A. C., (1994) "Die Methode des Lauten Denkens", In: Huber, G. L., and Mandl, H., (eds.), Verbale Daten. Weinheim: Beltz - Psychologie-Verl. Union, pp. 81-103. Zhang, J., (1997) "The nature of external representations in problem solving", Cognitive Science, 21 (2), 179-217.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter 29
3D Geovisualization and the Structure of Visual Space Colin Ware & Matthew Plumlee, Data Visualization Research Lab, Center for Coastal and Ocean Mapping, University of New Hampshire, Durham, New Hampshire, USA
Keywords: space perception, cognitive modeling, zooming, visualization, multiple views, cost of navigation
Abstract The problems of creating geovisualization interfaces are analyzed by characterizing space in three distinct ways. The first considers the nature of the perceptual structure of space. Here the primary reference system is the egocentric coordinate reference frame, centered on the user and oriented with respect to the user' s view direction. The second characterization is in terms of the cost of gaining extra information by navigation. This navigation is fastest when it consists of simple eye movements, and is much slower with virtual flying interfaces. Zooming interfaces lie somewhere between. The third characterization is in terms of the cognitive mechanism we use to make visual queries on displays containing geographic information. Limits of visual working memory impose severe restrictions on our ability to inter-relate information from different parts of large geographic data spaces. We show how understanding these three aspects of space can be used to make specific predictions about which types of navigation interface will be most suitable for specified tasks.
29.1
Introduction
When scientists and engineers think of space, it is often in terms of uniform Cartesian coordinates, with x, y, and z values defining 3D position. The human perception of space is very different; the visual system supports a number of coordinate systems, including retina-based, as illustrated in Figure 29.1, and egocentric head/torso-based. Humans are also, to a very limited extent, capable of constructing a view of space that approximates Cartesian space, but individuals vary greatly in their ability to mentally imagine 3D structures in this way. In general it is the egocentric space that dominates our visual thinking. 567
568
C. Ware & M. Plumlee
Figure 29.1. Perceptual space relative to an observer. Dimensions x and y (the visual field) are perceived very differently to dimension z (depth). A second way of thinking about space is in terms of the cost of navigation. To visually sample something that is in the visual field takes only an eye movement (a fraction of a second). To sample something in another room will take half a minute or more of navigation time. To sample something in another city takes hours of travel. But computers give us the opportunity to go beyond real-world navigation - in virtual 3D spaces we have the option of creating artificial navigation techniques. For instance, one could grow and shrink like Alice in Wonderland by using a zooming interface to navigate, or one could teleport from place to place like Captain Kirk, just by following a hyperlink. Whatever form navigation takes, however, it has a time cost and a cognitive cost that both affect our ability to make decisions and synthesize information. A third way of thinking about space is in terms of how it is represented in memory. We generally remember the layout of objects in space with respect to our vantage point when we observed them, using egocentric coordinates (Shelton and McNamara, 1997). But in terms of larger scale geographic information, we can either learn by navigation, as we walk and drive from place to place, or we can acquire spatial layout by viewing a map. Navigation knowledge builds up slowly: first we acquire route knowledge, then a map-like cognitive representation eventually forms that allows us to estimate distances between remote points "as the crow flies" (Seigel and White, 1975). The use of maps can dramatically shorten this learning process. A study by Thorndyke and Hayes-Roth (1982) found that 20 min studying a map can result in better distance estimation performance than a year navigating through a building, even though the map was not available during task execution. Memory for map information is generally systematically distorted. For instance, people tend to straighten paths and boundaries, and make intersections more perpendicular (Lloyd and Heively, 1987; Milgram and Jodelet, 1976). There is also evidence for a hierarchical clustering effect. An example of this is that people tend to think of cities in California as being due west of cities Nevada, even though some (e.g., Los Angeles, with respect to Reno) are almost due south (Stevens and Coupe, 1978). This chapter builds on what we know about both perceptual processes, cognitive maps and the navigation costs of visual space for 3D GIS interfaces. It is organized as
3D Geovisualization and the Structure of Visual Space
569
follows. We first present some of the properties of visual space with reference to perceptual mechanisms, and we discuss design implications for interactive GIS. We then factor in the cost of acquiring knowledge through navigation.
29.2
Perceptual Issues
A simplified 3-stage model of the visual system is useful in discussing issues of visual space (Ware, 2000). It is illustrated in Figure 29.2 and described below. Stage 1" Feature processing in parallel. The signals sent up the optic nerve represent space in a very non-uniform way. The center 5% of the visual field has about 50% of the neurons in the primary visual cortex devoted to its analysis (Drasdo, 1997). Signals arriving in the brain undergo massive parallel processing by more than 2 billion neurons. These neurons simultaneously process every part of the visual field to extract features such as edges, colors, textures, elementary motions, and the precursors of depth. If symbols are pre-attentively distinct, they can be immediately identified in a field of other symbols (Triesman and Gormican, 1988). This means that they must differ from all other symbols on a simple dimension, such as size, motion direction, or color. A red symbol on a field of black symbols immediately "pops out" and can be identified. Implications: The non-uniformity of the field means that information should, wherever possible, be clustered so that it can be acquired in a single fixation. In interactive mapping applications, it is important to use highlighting techniques that are pre-attentive. Suppose we ask a GIS to reveal all high-income communities, while retaining all other information. If highlighting is done through a pre-attentive cue (such as
Figure 29.2. A simplified schematic model of the human visual system. At different levels of visual processing, features, patterns, and objects become the focal targets of processing. Visual space has different characteristics at each level.
570
C. Ware & M. Plumlee
a strong color that is not otherwise used, or blurring the background items (Kosara et al., 2002)) then the required information will be much easier to read. Stage 2: Pattern and depth. In the middle stages of visual processing, patterns are extracted from the low-level features. Space is segmented into regions based on color, texture, and border detection operations (Grossberg, 1997). Simultaneously, very different pattern-finding operations based on depth cues are applied to find the relative distances of objects in the third direction, (z) towards and away from us (see Figure 29.1). Depth cues such as occlusion, linear perspective, stereopsis, motion parallax, eye vergence, and others are applied to help us make judgments in this direction. Occlusion is the single most important depth cue, over-riding others and because of occlusion we can see far less information in the z direction. Generally, for perceptual pattern finding operations, such as region segmentation, the 2D visual field is more important than information about the z direction (Ware, 2000). Implications: The different dimensions should be treated differently in interface design. This means that even if we construct GIS interfaces with 3D, we should design them with the viewpoint and 2D visual field in mind. We should use the z direction sparingly in information spaces. For example, it is generally only possible to show two overlapping surfaces simultaneously, and then only to a limited extent and with careful use of transparency and texture. Conversely, it is possible to layout complex, detailed patterns using only the x and y dimensions. In addition, care must be taken to avoid hiding commonly referenced visual objects through occlusion, especially interface controls such as navigation widgets. Stage 3: Object perception and visual working memory. Visual working memory and verbal working memory, together with cognitive instruction execution mechanisms, constitute our core cognitive decision-making engine. We reason with externalizations by constructing visual queries. Often the act of perceiving a pattern is the way we understand a solution to a problem, for example when we perceive a route on a map. However, visual working memory is extremely limited. Studies suggest that it can hold only about three quite simple objects or patterns, and multiple attributes such as shape and color can be composed into a single object (Vogel et al., 2001). We can load new information into visual working memory either by making an eye movement, or accessing internal long-term memory. Problem solving with GIS typically involves bringing together patterns from both internal (long-term memory) and external (display) sources. However, the three-object limit appears to be at least somewhat elastic. A critical issue in this analysis is what constitutes a visual working memory object, and whether an object could be a "chunk" consisting of other objects. Research suggests that chunks can be created with expertise so that, for example, chess experts can see more complex patterns of game pieces at a glance than non-experts (Chase and Simon, 1973). Also, the well-known gestalt laws of pattern perception provide a set of chunks that would be common to all observers, in the automatic grouping of objects (Ware, 2000). Implications: An efficient problem-solving system should be based on patterns that are easily held in visual working memory. In addition, we must pay great attention to navigation techniques: if the navigation itself takes a long time and consumes significant perceptual and cognitive resources, this will leave fewer resources for decision-making.
3D Geovisualization and the Structure of Visual Space
29.3
571
Navigation
A navigation mechanism should afford rapid and simple navigation in order that maximal cognitive resources are retained for decision-making. The navigation mechanism should afford context as well as focal information. Focal information is that which is the immediate focus of attention, and hence most frequently loaded into visual working memory. Context information is task-relevant information that should be available for rapid access if needed. With these criteria in mind, we therefore turn our attention to different navigation mechanisms that can be used in a 3D GIS. 9
9
9
9
Eye movements are the most rapid way of bringing new information into visual working memory. Our eyes jump from point to point with saccadic eye movement, constantly sampling the world for relevant information. Studies in which subjects were asked to scan maps for particular symbols have concluded that we can search between two and three symbols (or map labels) per second (Dobson, 1977; Phillips, 1981). However, this only works for objects in the visual field. There are a number of design implications. If possible, we need to place all relevant information simultaneously in the field of view. Large screens are useful but they must also be ultra high resolution, otherwise when we look directly at something we get little or no new information. Another implication is that in order to make an effective map interface, even if it is 3D, special consideration should be given to layout in the 2D visual field because sampling through eye movements is essentially a 2D operation. W a l k i n g : Some researchers have implemented virtual reality (VR) geographic spaces that afford walking (Darken et al., 1998). Walking to a location helps us build a mental model of a 3D space, but so does giving an overview map, and the overview map is likely to be more effective (for some empirical results and a discussion (see Coors et al., this volume (Chapter 27)). Moreover, in practical terms, creating a virtual space where people can actually walk a significant distance is not currently possible. Typically, it takes tens of seconds to walk to a new location and distance is limited. F l y i n g : Many VR environments have implemented some form of flying interface for navigation. Whereas real-world flying takes hours, VR environments can have non-linear controls that enable rapid navigation over large distances, with reasonable controls at small distances (Ware and Osborne, 1990). Thus, virtual flying takes from a few seconds to tens of seconds depending on the distance. Hyperlinks: Computer Science has produced a number of navigation techniques that go beyond the simulation of real-world navigation. With hyperlinks, some object is selected using the mouse, causing an abrupt change in the information displayed. A mouse selection typically takes one to two seconds. One of the problems with hypeflink navigation is loss of context. Typically, the old information is simply replaced, a problematic effect when the goal is the synthesis of information.
Eye movements:
572
C. Ware & M. Plumlee
Zooming: The advantage of zooming in and out is that it can enable scale changes of many orders of magnitude in a few seconds (Bederson and Hollan, 1994; Ware et al., 2001). Navigation is achieved by zooming out to get an overview, then zooming back in to obtain new focal information. Context is provided by the zoom out, but users must still remember the content of successive views. Linked windows: Linked windows can show detailed information and context at the same time (Ware and Lewis, 1995; Plumlee and Ware, 2002). Manipulating the view in each window typically requires two or three hand movements and may occupy a few seconds. There can also be perceptual and cognitive costs involved in setting up and manipulating extra windows. As a general design principle, context-giving overviews may be best presented as an exocentric view (not viewer centered) whereas an egocentric view will be better for control tasks such as steering a vehicle (Lasswell and Wickens, 1995; McCormick et al., 1998).
29.4
Cognitive System Models to Evaluate Alternatives
For the purposes of cognitive modeling, we can consider that the purpose of a 3D GIS display is to get relevant information into visual working memory for decision-making. We can therefore analyze the value of different navigation techniques by considering visual working memory capacity as a key resource. We have already argued that eye movements are the most efficient means for navigation, but in many cases it is not possible to have all relevant information in a single view. As previously discussed, zooming and coupled windows are alternative ways of obtaining detailed views while providing rapid access to a larger spatial context. But the question remains, when is each technique most appropriate? To answer this question, we have constructed a kind of cognitive cybernetic model that takes both human memory and system characteristics into account (Plumlee and Ware, 2002). Consider the task of finding similar or identical patterns spaced far apart in a large geographical space, as illustrated in Figure 29.3. With a zooming interface it is necessary to zoom into and look at one pattern, then hold that pattern in visual working memory while zooming out to seek other patterns. The pattern in visual working memory is then compared to new patterns seen during the search process. If a possible match is found, it may be necessary to zoom back and forth to confirm details of the match. An alternative method is to use extra windows to magnify parts of the main display. Each window has a dragable proxy (illustrated in Figure 29.3) that enables the focus to be changed through direct interaction. When two such windows are in position, it is possible to simply make eye movements between them to confirm or disconfirm the match far more rapidly. The critical resource here is visual working memory capacity, because it determines how many visits are required to make the comparison. It is known that visual working memory has a very limited capacity - about 3 simple items (Vogel et al., 2001). If the target pattern is simple enough to be held in visual working memory, then zooming will often be more efficient, because it avoids the overhead of setting up multiple
3D Geovisualization and the Structure of Visual Space
573
Figure 29.3. Subwindows show a magnified view together with the source of the information in the background overview.
windows. If more than 3 items are in the target pattern, then it will be necessary to zoom back and forth between them; the more the items, the more zooming is required. When there are many items in a pattern, the window solution will often be faster: once the
Figure 29.4. Model predictions are shown on the left. Measured task performance is shown on the right. Multiple windows speed performance relative to the use of a zooming interface when the number of objects to be compared is larger. 9 2002 (Plumlee and Ware, 2002).
574
C. Ware & M. Plumlee
windows are set up over the patterns, rapid eye movements are all that are required to make the comparisons. We have modeled two user interfaces (simple zooming vs. multiple windows). The model generally predicts the average time required to complete a task that requires moving from one view to another. It takes into account the requirements of the task, the speed of the interface, and the cognitive limitations of the user. When applied to a pattern-matching task, it specifically accounts for the fact that only limited information from part of a scene can be retained in visual working memory as a basis for comparison with another part of a scene. An interface that requires zooming to be used to do the comparison will perform poorly compared to one that provides multiple views, because the user will have to zoom back and forth many times. When extra windows are allowed, much more rapid eye movements can be made back and forth to compare the two patterns. The model predicts that when more than 3 or 4 component items are in a patternmatching task, extra windows allow the task to be performed faster. We have carried out an experiment using various numbers of simple colored objects such as cubes, cylinders, spheres and cones. These objects were grouped into widely separated clusters, and the subject's task was to determine which of the clusters matched a designated target pattern. The predictions of the model are shown in Figure 29.4a, modeled for capacities of visual working memory at 2, 3, and 4 items (the heavy lines indicate a 3-item limit on visual working memory). As can be seen, the model predicts that zooming will have an initial benefit because extra windows take more time to set up. However, as the number of objects increases, the extra window interface will be beneficial. The measured results, as shown in Figure 29.4b, closely matched the prediction in overall trends. We believe that our failure to find a clear stepwise pattern has to do with the fact that the number held in visual working memory is somewhat random, varying from trial to trial and person to person. The model and experiment are both described in detail in Plumlee and Ware (2002).
29.5
Summary Implications
To summarize, the purpose of this chapter has been to describe the structure of space in terms of both perception and action, and discuss the implications for 3D GIS interfaces The two dimensions of space orthogonal to the line of sight (x and y in Figure 29.1) are extremely non-uniformly represented at a single glance, but can be explored by rapid eye movement. In contrast, the third direction along the line of sight (z) is understood in terms of depth cues and has far more limited capacity for conveying information. A core resource in visual problem solving is visual working memory, and we have shown that this can be used to determine when extra windows in an interface are needed. A number of design implications have been drawn which we briefly summarize here. The display space should be treated as non-uniform with most of the information laid out in the x and y directions. Only sparing use should be made of the third (z) direction.
3D Geovisualization and the Structure of Visual Space 2.
3.
4.
5.
575
Navigation controls should generally be in the visual field and never be hidden through occlusion, in order to minimize the cognitive cost of navigation. It should not be necessary to use cognitive resources simply to find controls, although expert users may use non-visual controls, such as keyboard control keys. For rapid navigation, for tasks involving simple patterns, zooming is better than flying, which is better than walking. The more rapid the navigation technique, the shorter is the time between critical visual comparisons, and the less likely will the information held in visual working memory be lost in the transition. Extra windows and/or views should be provided whenever it is necessary to compare objects or patterns that have more than a minimal degree of complexity. This enables rapid eye movements as a basis for comparison. Overview maps in 3D GIS can enable users to gain knowledge of the relative layout and distances between objects. This suggests that combining overviews with more focused views will be useful if they can easily be linked.
Overall, even though we may require a 3D GIS system, it is still necessary to pay special attention to 2D design issues such as the layout of objects on the picture plane when a scene is viewed from a particular vantage point.
Acknowledgements We gratefully acknowledge financial support from the National Science Foundation ITR #0081292 and NOAA.
References Bederson, B. B., and Hollan, J. D., (1994) "PAD+ +: a zooming graphical user interface for exploring alternate interface physics", Proceedings User Interfaces Software and Technology '94. New York: ACM, pp. 17-27. Chase, W. G., and Simon, H. A., (1973) "Perception in chess", Cognitive Psychology, 4, 55-81. Darken, R. P., Allard, T., and Achille, L., (1998) "Spatial orientation and wayfinding in large-scale virtual spaces: an introduction", Presence: Teleoperators and Virtual Environments, 7(2), 101-107. Dobson, M. W., (1977) "Eye movement parameters in map reading", American Cartographer, 4, 29-37. Drasdo, N., (1997) "The Neural representation of visual space", Nature, 266, 554-556. Grossberg, S., (1997) "Cortical dynamics of three-dimensional figure-ground perception of two-dimensional pictures", Psychological Review, 10(3), 618-658. Kosara, R., Miksch, S., and Hauser, H., (2002) "Focus + context taken literally", IEEE Computer Graphics & Applications, 22(1), 22-29, (Jan/Feb 2002). Lasswell, J. W., and Wickens, C.D., (1995) The Effects of Display Location and Dimensionality on Taxi-way Navigation, Tech Report, (ARL-95-5/NASA-95-2). Savoy, IL: University of Illinois.
576
C. Ware & M. Plumlee
Lloyd, R., and Heively, C., (1987) "Systematic distortions in urban cognitive maps", Annals of the Association of American Geographers, 77, 191-124. McCormick, E., Wickens, C. D., Banks, R., and Yeh, M., (1998) "Frame of reference effects on scientific visualization subtasks", Human Factors, 40, 443-451. Milgram, P., and Jodelet, D., (1976) "Psychological maps of Paris", In: Proshansky, H. M., Itelson, W. H., and Revlin, L. G., (eds.), Environmental Psychology. New York: Holt Rinehart & Winston, pp. 104-124. Phillips, R. J., (1981) "Searching for a target in a random fixation of names", Canadian Journal of Psychology, 35(4), 330-346. Plumlee, M., and Ware, C., (2002) "Zooming, multiple windows, and visual working memory", Proceedings of the Working Conference on Advanced Visual Interfaces, Trento, Italy, pp. 59-68. (May). Shelton, A. L., and McNamara, T. P., (1997) "Multiple views of spatial memory", Psychonomic Bulletin & Review, 4, 102-106. Siegel, A. W., and White, S. A., (1975) "The development of spatial representations of large-scale environments", In: Reese, H. W., (ed.), Advances in Child Development and Behaviour. London: Academic Press, pp. 9-55. Stevens, A., and Coupe, P., (1978) "Distortions in judged spatial relations", Cognitive Psychology, 10, 422-437. Thorndyke, P. W., and Hayes-Roth, B., (1982) "Differences in spatial knowledge acquired from maps and navigation", Cognitive Psychology, 14, 560-589. Triesman, A., and Gormican, S., (1988) "Feature analysis in early vision: evidence from search asymmetries", Psychological Review, 95(1), 15-48. Vogel, E. K., Woodman, G. F., and Luck, S. J., (2001) "Storage of features, conjunctions and objects in visual working memory", Journal of Experimental Psychology, Human Perception and Performance, 27(1), 92-114. Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, p. 384. Ware, C., and Lewis, M., (1995) "The DragMag image magnifier", A CM CHI'95 Conference Champion, 407-408. Ware, C., and Osborne, S., (1990) "Exploration and virtual camera control in virtual three dimensional environments", A CM SIGGRAPH Computer Graphics, 24(2), 175-183. Ware, C., Plumlee, M., Arsenault, R., Mayer, L. A., Smith, S., and House, D., (2001) "GeoZui3D: data fusion for interpreting oceanographic data", Oceans 2001 Proceedings, Hawaii, pp. 1960-1964.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter 30
Applications of a Cognitively Informed Framework for the Design of Interactive Spatio-temporal Representations Robert M. Edsall & Laura R. Sidney, Department of Geography, Arizona State University, Arizona, USA
Keywords: interaction, usability, cognitive science, schemata, tool design, animation Abstract In dynamic geographic information and visualization systems, the ways in which a user is allowed to manipulate the map and the data represented (through various interaction capabilities) are just as important as the ways the data are presented (as marks on the screen). This chapter describes strategies that will help cartographers design geovisualization interfaces and environments. We present a framework, informed by cognitive science, for designing and developing modes of interaction for use in geovisualization environments. We also review some applications of that framework in the context of representing the temporal component of geographic data. Consideration of the factors that guided us in these applications may assist both geovisualization users in understanding the limitations and opportunities presented by the tools they are using and geovisualization developers in the creation of such tools.
30.1
Introduction
Cartographers have considered the role of the design of. cartographic symbols for decades. Symbolization strategies for maps, like design strategies for other useful everyday objects, are guided by empirical and theoretical studies that link the form of the symbols to their function. With the increasing prevalence of interactive cartographic products, the attention of cartographers is expanding to include both the design of marks on a page or screen (traditional "symbols" of maps) and the design of the ways that users may manipulate those symbols. Indeed, in a geovisualization context, maps (and other cartographic products, such as graphs and charts) are meant to be manipulated, for their function is to build knowledge in (interactive) conjunction with the existing expertise and knowledge base of the analyst using the maps. 577
578
R.M. Edsall & L.R. Sidney
To facilitate this knowledge construction, or ideation (see Gahegan, this volume (Chapter 4), and Andrienko et al., this volume (Chapter 5)), modes of interaction need to be designed just as carefully as the visual symbols on the map display. This chapter describes strategies that will help cartographers design geovisualization interfaces and environments. We advocate keen attention to cognitive aspects of the design of tools for map manipulation, and describe several efforts that serve as case studies for the approach. We argue that, in many respects, the design of interaction forms in a geovisualization environment is analogous to the design of symbols on a static map. Just like symbols, the form of the interactive tools can suggest interpretations of the data, and inform the knowledge that is gained from the display. With this in mind, the case studies described in this chapter include some novel tool designs that encourage users to understand their data in ways that would have been more difficult with other designs. Consideration of some of the same factors that guided us in these case studies may assist not only other geovisualization developers in the creation of tools but also geovisualization users in understanding the limitations and opportunities presented by the tools they are using.
30.2
Interaction and Symbolization
With static maps or with a highly automated (i.e., minimally interactive) system of graphic representations, the ability to examine information from a variety of perspectives is quite limited, and visual displays of information are effective primarily for communication of the designer' s - and not the end user' s - ideas. This is contrary to the basic paradigm shift in Cartography away from an approach that favors the passive communication of ideas to the map user (with a design emphasis on the single-best-solution map) to an approach that enables and encourages multiple perspectives on data, with a goal of active exploration of information for the discovery (not communication) of knowledge. However, design strategies to incorporate this shift into real-world interactive environments are yet to be fully developed. Modes of interaction signify operations that can be applied to the data in a manner similar to the way in which modes of symbolization signify characteristics of the data. A pseudo-3D button in a Web map interface, for example, signifies to the user that there is more information available upon pressing the button. Tweedie et al. (1996) described such an interface as "opportunistic". Different visual designs of this button may signify different opportunities to the user. A hotlink without a 3D effect such as a drop shadow may fail to indicate the opportunity to gain more information (van den Worm, 2001). Though static maps could be considered interactive in that they can be turned or sketched on, here we will use a definition developed by Crampton (2002, p. 88), that interactivity involves "a system that changes its visual data display in response to user input". Crampton proceeds to create a useful typology of interaction in geovisualization based on interactivity tasks, in much the same way that Krygier et al. (1997) categorized dynamic representation ("resources") by their form and function. Such a task-oriented approach, combined with the consideration of the types of users and the types of data (see Andrienko et al., this volume (Chapter 5)), should drive interactive tool design.
Design of Interactive Spatio-temporal Representations
579
The design of interactive elements of geovisualization environments, therefore, should be treated in much the same way as the design of symbols on a static map: in many ways, interaction itself is a form of symbolization in dynamic visualization applications.
30.3
Utilization of Mental Models in Spatio-Temporal Interface Design
Card et al. (1998) propose several mechanisms by which "visualizations" - that is, dynamic computer-generated representations of data - aid in cognition, extending the notions of Larkin and Simon (1987), who focus on static diagrams. Visualizations, they argue, (i) reduce the search for relationships by grouping related information visually, and (ii) "offload" inferences from the cognitive to the perceptual system (both of which apply to static as well as dynamic representations). In addition, dynamic visualizations facilitate the representation of large amounts of data, primarily because they are manipulable, making it possible for the user to restrict attention to only a portion of the data at a time (for an examination of some of the limitations of this approach (see Ware and Plumlee, this volume (Chapter 29)). In diagrammatic representations (both static and dynamic), location is used as an organizational variable for the information, and patterns and relationships in the data are observed using knowledge structures, known as schemata, about space. Examples relevant to design in geovisualization include locational schemata, including the abstract notion that "closer in space equals closer in value." Information Visualization researchers have exploited this particular schema in map-like representations of abstract information, representations which some cartographers have termed "spatializations" (Fabrikant and Buttenfield, 2001; Fabrikant and Skupin, this volume (Chapter 35)). Dynamic representations aid in the monitoring of processes, and enable the use of temporal as well as locational schemata to be utilized for the recognition of patterns. It is for this reason that research into how spatial and temporal relationships are obtained and stored in the mind is a key requirement for understanding how such representations should be designed. Schemata represent elements of long-term memory. They are activated from memory when necessary, and integrated - like building blocks - to form coherent representations of a given problem or phenomenon. Particularly important among these building blocks in the design of geovisualization interfaces, it seems, are anticipatory schemata, which set up conceptual expectations and guide thoughts and emphasizing elements in a scene, perhaps through their graphical (or linguistic) representation (Peuquet, 2002). More temporary internal representations are formed as the elements of the graphic prompt these building blocks and activate them in conjunction with one another (Johnson-Laird, 1983). In highly interactive and animated representations and environments, the impacts of a designer's choices of interaction methods - and their effect on building mental models of the phenomena represented - remain unexamined. It is reasonable to expect that giving users the ability to adjust parameters of the display will result in users discovering different features and model concepts than they would with static representations. It is also reasonable to expect that the level and type of interactivity
580
R.M. Edsall & L.R. Sidney
provided will have an impact on this process. Enabling users to interact with displays not only alters what they see, but also how they see it; a highly interactive environment actually encourages the formation of multiple perspectives on data and discourages the notion that there is one "objective" and "optimal" means with which to examine data visually (MacEachren and Ganter, 1990). Certainly, some traditional conventions and assumptions about visual data symbolization apply to dynamic representations on a computer display. However, interpretations of the represented phenomena and the consequent understanding are now a combined function of symbolization and the facility to interact with the representation. For example, though animation has been justifiably cited as an important method for visualizing dynamic information (Moellering, 1976; Acevedo and Masuoka, 1997), Dorling (1992) notes that the animation without interactivity and other supplemental visualization methods is incomplete. Dorling' s informal testing of an elaborate animation of British electoral results yielded a salient point about animation: "... almost everyone who was shown this video wished to control it personally, rewinding, pausing, and fast-forwarding to enable them to grasp what was being shown. As 'film' it would not have worked. As 'video' - shown to well-informed individuals - it revealed facts they had not previously appreciated (Dorling, 1992, p. 219)". Extending these now-commonplace "VCR style" controls that Dorling is advocating to designs that enable more creative interaction with animations or other dynamic displays will likely lead to more creative insight. Important features, say, of a thunderstorm in an animation could go unnoticed if the user is not allowed to adjust the speed of the animation, not allowed to turn variables (like temperature) off and on, or not allowed to reorder the frames or inspect only every third frame (DiBiase et al., 1992; Kraak and Klomp, 1996). In those cases, the visual symbolization (colors, glyphs, camera angle, etc.) of the display is not at issue. Instead, the cartographic researcher is interested in the different insights about the data that are generated when different means of interacting with the data (e.g., different means of selecting, analyzing, or symbolizing data) are provided. The ability to change the variables represented or the symbolization methods used dynamically (e.g., animation pace, classification breaks, color scheme, scale) will itself dictate the potential for insight from a graphic display. Assessing the potential for insight through the use of tools of various interactivity styles and levels is a current priority in geovisualization research. Confirming or refuting some of the (presently speculative) hypotheses outlined above, or others like them, would represent significant contributions to the geovisualization literature. As presented in the introduction to this section (Fuhrmann et al., this volume (Chapter 28)), this type of research is fundamental to understanding what makes geovisualization environments useful. Because geovisualization tools are meant to be used in conjunction with one another, designing tools through testing them in isolation is often impractical (though necessary where possible). Therefore, designing tools is most successfully and efficiently accomplished using a sound theoretical framework, informed by cognitive science and usability engineering, like that described here.
Design of Interactive Spatio-temporal Representations
581
To summarize, the design of effective geovisualization systems should be based on the encouragement of creative and multiple perspectives on the data represented. This can be achieved by recognizing that users can be prompted to such perspectives with tools that conform to, but also alter, their internal representations of space, time, and the represented data. Users of interactive visualization environments are now allowed to make symbolization choices previously left to cartographers. The new task of the cartographer (as a geovisualization designer) is the creation of tools and representations that afford opportunities for insight. These tools for interaction - just like the marks on a static map - must thus be designed to maximize the possibilities for insight.
30.4
Applications: Strategies for Representing Time in Geovisualization
Our approach can be applied to geovisualization tools, maps, graphs, and environments in general. The following case studies utilize the concepts above to facilitate the understanding of the temporal component of geographic information. Many of the data sets of interest to geographers are by nature temporal in character (in addition to being spatial); indeed, much of today' s most vital and pressing geographical problems are those of temporal process, from climate change to spatial diffusion of disease to urban sprawl (Kite, 1989; Bloomfield and Nychka, 1992; Groisman and Legates, 1995; Acevedo and Masuoka, 1997). It is in these contexts that the potential for geovisualization and geographic information systems (GIS) that integrate time into their representation, both in terms of representation through database and display, is the greatest.
30.4.1
Interface design in temporal GIS
Cognitively informed representation of temporal data has recently been addressed in the context of GIS. Peuquet (1994) writes that the overall goal of a temporal GIS is: "... to represent stored spatio-temporal data in a way that conforms to human conceptualizations of the world in space-time and geographic theory and to technical demands for accuracy and flexibility in computer-based analysis and visual presentation (p. 442)". Again, this represents a call for customizable and flexible representation systems for the display and analysis of geographic data. But a well-reasoned design of those representations must take into account the existing internal (mental) representations of space and time to be most effective (Couclelis, 1993). From a temporal point of view, this might mean conforming to a model of time as linear and irreversible. However, for true flexibility, the design may also demand the ability to develop alternate models of time, including cyclical or chaotic models. As Peuquet (2002, p. 295) notes: "... there can never be a perfect single mode of communication that is both formally specified and allows for all the complexities of the world.
582
R.M. Edsall & L.R. Sidney
Like the human exploring his or her real environment, the task requires a combination of modes working in concert". Creative investigations of the temporal character of spatio-temporal information might therefore be encouraged by the provision of tools that match this variety of mental conceptions of time. Allowing a user, for example, to interact with the time component of an animation solely via a timeline representation (such as that offered by a QuickTime movie interface) may limit insights from that animation to trends rather than cycles. A more complete understanding of the temporal nature of geographic phenomena might be enabled by the representation of time in alternate ways. A clock-like circular representation, for example, might prompt temporal schemata dealing with the periodic character of phenomena. One implementation of this principle occurred in the construction of an interface for the temporal GIS Tempest (Peuquet and Qian, 1996). We designed a temporal querying tool that could alternate between a time line and a "time wheel" at the user' s request (Edsall and Puequet, 1997). The line representation (Figure 30.1a) is designed for querying observed raw data, with no presumption or imposition of temporal rhythms or patterns. For example, a query about the gradual change of an undisturbed meadow to old-growth forest land over many decades would likely be linear and continuous (for example, "how has land cover changed between 1950 and 1990?"), and would likely be best represented graphically (and mentally) by a time line. The wheel representation (Figure 30.1 b), on the other hand, is useful when querying spatio-temporal data that may have a known or anticipated cyclical nature. The user may specify the period of the cycle represented and choose to query only those dates that correspond to a particular duration within the cycle. For example, suppose a researcher were interested in the variation of rainfall each monsoon season over several years, he or she would customize the time wheel to a yearly period and then select the days, weeks, or months of interest to limit the investigation. A prototype query tool is conceptual combination of linear and cyclical representations of time: the "time coil." The coil of time resembles neither a line (implying linearity) nor a clock face (implying periodicity). Rather, the coil resembles a single-helix - a telephone cord - with linearity along its length, but cyclical behavior in its internal structure (Figure 30.2). Looking at the time coil from the "side" prompts a concept of linearity; yet, looking from its end (showing only a circle) prompts a concept of periodicity. This tool could be used in contrast to (or in complement with) the existing temporal query and display tools to facilitate creative interaction and the development of multiple temporal perspectives. The flexibility of investigative methods designed into these tools encourages creative exploration of the databases for the formulation of new hypotheses as well as the proof of existing ones.
30.4.2
Legend design for animated maps
A pair of projects described below were designed to assess the degree to which changes in the representation of the temporal indicator (or "temporal legend") on an animated map influences the understanding of the mapped phenomena. An interactive animated map,
Design of Interactive Spatio-temporal Representations
583
Figure 30.1. (a) The Tempest timeline query tool (Edsall and Peuquet, 1997). (b) The Tempest timewheel query tool (Edsall and Peuquet, 1997). whether or not it is linked to a database representation of the data displayed (as in a temporal GIS), graphically represents spatio-temporal information as a sequence of images within which changes, patterns, and processes can be identified visually. As discussed above, interface design and data representation techniques have a profound impact not only on what information is communicated by the representation but also on how that information is interpreted and explored. A wide variety of different methods currently exist for representing the variable that serves as the basis for the animation (this is often time, but may be any ordered variable). In initial work assessing the relative effectiveness of these types, Kraak et al. (1997) refer to these indicators as temporal legends, and classify them as linear, cyclical (a similar distinction as the timeline and timewheel query tools described in w and text, where the location in time of the animation is given simply by letters and numbers that change with the animation. Our general hypothesis was that different characteristics of space-time data would be enhanced using one legend form over another. Classifying characteristics of space-time phenomena can be accomplished in a variety of ways, no single one of which is sufficient to describe all forms of data: for example, we distinguish a spatio-temporal phenomenon by its linearity, stability, and regularity. In that particular study, we investigated whether there is a differential influence of legend design depending on the focus (or type) of the task at hand when using the animated map. We asked carefully designed questions of human subjects (in this case, a mix of college students - both geography and non-geography majors - and college faculty) to assess their understanding of an animated satellite weather image. The legends tested were of the linear, cyclical, and text varieties, and are illustrated in Figures 30.3 and 30.4. The questions reflected tasks dealing with the linearity, stability,
Figure 30.2. The (prototype) Tempest time coil tool (Edsall and Peuquet, 1997).
584
R.M. Edsall & L.R. Sidney
Figure 30.3. The linear legend type (Kraak et al., 1997). and regularity of the phenomena in the image, and were asked while users were provided only one of the three legend types described above. The results of the particular tests of that study failed to show significant differences among the legend types in either of the dependent variables that were recorded: accuracy and response times of answers to all questions. In addition, the hypotheses that different question types were more accurately or more quickly answered using different legends were also rejected. In retrospect, several aspects of this usability assessment prevent broad generalization from the result. 1 Even with these limitations, the evidence suggests that legend types might not have the hypothesized impact on performance, at least for narrowly defined tasks of the kind used here. It remains an open question, however, whether legend design has an impact on more broadly defined exploration tasks more typical of geovisualization. However, as discussed in the introduction to this section, assessing, understanding, and generalizing about geovisualization use for such tasks can be challenging (see Fuhrmann et al., this volume (Chapter 28)). A second study to assess similar temporal legend design issues is currently being investigated at Arizona State University. Designing for alternative conceptualizations of time may also involve cultural factors, and a cognitive approach to interaction design should consider a variety of cultural influences that result in alternative conceptions of time and of temporal phenomena. For example, a particularly intriguing and relevant cultural factor for geovisualization design is reading pattern directionality. Reading pattern directionality refers to the direction one learns to read across a page, i.e., horizontally from left-to-right, horizontally from right-to-left, or vertically from right-to-left.
1 Specific limitations to the study, included here as a caveat to those wishing to perform similar assessments, included the following. First, the number of multiple-choice answers (some questions had four answers, others had three or five) varied among task types, and made it impossible to compare percent correct or reaction times directly among tasks. The limited interactivity of the text legend case (in comparison to the others) put that legend at a disadvantage (but one that matched standard practice). Finally, order effects (that the linear and cyclic legend were first for one group each, but the linear legend was last for two groups while the cyclic was last for only one group) may have given the linear legend a slight advantage.
Design of Interactive Spatio-temporal Representations
585
Figure 30.4. (a) The cyclical legend type (Kraak et al., 1997). (b) The text legend type (Kraak et al., 1997).
This factor, it is hypothesized, will influence one's perception of time. Findings in cognitive psychology and the neurosciences have determined that motion is more readily detected in either the right or the left visual field where reading begins Morikawa and McBeath, 1992; Newman and Lamb, 1986; Evitar, 1995). Cultures that employ a left-to-right reading pattern may perceive motion more quickly and more accurately if the motion begins on the left and proceeds into the right visual field, referred to as a leftward bias (Barrett et al., 2002; Evitar, 1995). Arabic cultures that read Hebrew or Jewish scripts tend to favor the left visual field, which means they have a rightward bias where motion begins on the right and proceeds to the left (Newman and Lamb, 1996). The nominal amount of research conducted on Asian cultures that utilize a vertical, right to left reading pattern has produced no lateral bias (Barrett et al., 2002). Currently, most depictions of time on geographic animations progress left-toright. This convention may hinder understanding of a temporal animation for nonWestern users. If an Arabic reader, for example, is observing motion, and the motion is representing changes in time, it may be to the user's benefit to display the motion according to his/her native reading pattern for ease of understanding. Many interfaces of popular computer applications, such as those of Microsoft, Apple, and Netscape, allow custom interactors to be employed. Commonly called "skins," they are components that allow users to customize elements such as backgrounds, title bars, buttons, and other graphical interfaces. Skins allow personalized interaction modes that ease the manipulation of the data contained within the interface. Providing analogous customization capabilities in geovisualization environments, informed by theoretical and empirical study, would expand the power of geovisualization environments and reduce potential effects of cultural (or other) differences that would influence understanding. The first phase of a human-subjects experiment has recently been completed to define the degree of influence of native reading pattern directionality on a user's understanding of a dynamic map and of the represented spatio-temporal phenomenon. The experiment evaluates three temporal legends, oriented according to the three major reading pattern directions in existence today: left-to-right lateral, right-to-left lateral,
586
R.M. Edsall & L.R. Sidney
and right-to-left vertical (Figure 30.5). Subjects were drawn from the diverse student body of Arizona State University, and each subject's native reading direction was noted. Each subject was presented with an animation containing one of three temporal legend types depicting change in a hypothetical dataset. The subjects were grouped by the legend orientation to which they are exposed, with each group comprised of subjects from Western, Arabic, and Chinese cultures. Accuracy, response times, and personal preference were recorded and are being analyzed. We hypothesize that users will perform fastest and most accurately for the temporal legend that moves according to the subject's native reading pattern. However, the evidence from the 1997 study described above indicates that there may not be significant differences in performance measures among the temporal legend movements. The eventual goal of this research is to establish guidelines for the representation of time in a dynamic map. Such guidelines would account for the possibility that internal representations of time (learned through language) might have a real effect on the comprehension of a map and the data contained within it.
Figure 30.5. Three temporal legends used to assess potential cultural differences in temporal conceptualization.
Design of Interactive Spatio-temporal Representations
30.5
587
Summary
The applications above outline a variety of approaches of representing and interacting with time in a geovisualization environment. Many more remain to be examined. The design of these approaches, and their subsequent investigation, exemplify a framework for the creation of tools for geovisualization environments. The methods described above for the investigation of the temporal component of geographic phenomena encourage creativity by tapping into users' mental representations of time. By allowing time to be represented in a variety of ways, multiple perspectives on the temporal nature of a geographic problem are facilitated. These tools are examples of how the design of the interactive components of a geovisualization environment can be a crucial factor in the ability of that system to enable and support insight.
Acknowledgements Gratefully appreciated financial support for projects described herein came from the US Environmental Protection Agency (grant #R825195-01-0) and the National Science Foundation (grant #9983451).
References Acevedo, W., and Masuoka, P., (1997) "Time-series animation techniques for visualizing urban growth", Computers and Geosciences, 23(4), 423-436. Barrett, A. M., Kim, M., Crucian, G. P., and Heilman, K. M., (2002) "Spatial bias: effects of early reading direction on Korean subjects", Neuropsychologia, 40, 1003-1012. Bloomfield, P., (1976) Fourier Analysis of Time Series: An Introduction. New York, NY: Wiley, p. 258. Bloomfield, P., and Nychka, D., (1992) "Climate spectra and detecting climate change", Climatic Change, 21,275-287. Card, S., Mackinlay, J., and Shniederman, B., (1998) "Information visualization", In: Card, S., Mackinlay, J., and Shniederman, B., (eds.), Readings in Information Visualization. San Francisco: Morgan Kaufmann, pp. 1-34. Couclelis, H., (1993) "People manipulate objects (but cultivate fields): beyond the vector-raster debate in GIS", In: Frank, A. U., Campari, I., and Formentini, U., (eds.), International Conference on GIS: From Space to Territory: Theories and Methods for Spatiotemporal Reasoning. Berlin: Springer, pp. 65-77. Crampton, J. W., (2002) "Interactivity types in geographic visualization", Cartography and Geographic Information Science, 29(2), 85-98. DiBiase, D., MacEachren, A. M., Krygier, J. B., and Reeves, C., (1992) "Animation and the role of map design in scientific visualization", Cartography and Geographic Information Systems, 19(4), 201-214, see also 265-266.
588
R.M. Edsall & L.R. Sidney
Dorling, D., (1992) "Stretching Space and Splicing Time: From Cartographic Animation to Interactive Visualization", Cartography and Geographic Information Systems, 19(4), 215-227, see also 267-270. Edsall, R., and Peuquet, D., (1997) "A graphical user interface for the integration of time into GIS", Proceedings of the 1997 American Congress of Surveying and Mapping Annual Convention and Exhibition, Seattle, WA, pp. 182-189. Evitar, Z., (1995) "Reading direction and attention: effects of lateralized ignoring", Cognition, 29, 137-150. Fabrikant, S. I., and Buttenfield, B. P., (2001) "Formalizing semantic spaces for information access", Annals of the Association of American Geographers, 91(2), 263-290. Groisman, P. Y., and Legates, D. R., (1995) "Documenting and detecting long-term precipitation trends: where we are and what should be done", Climatic Change, 31, 601-622. Johnson-Laird, P. N., (1983) Mental Models. Cambridge, UK: Cambridge University Press, 513 pp. Kite, G., (1989) "Use of time series analysis to detect climatic change", Journal of Hydrology, 111,259-279. Kraak, M. J., and Klomp, A., (1995). "A classification of cartographic animations: towards a tool for the design of dynamic maps in a GIS environment", Proceedings on the Seminar on Teaching Animated Cartography, Madrid, Spain, August 30September 1, ICA Commission on Multimedia, ICA, 1996, pp. 29-36. Kraak, M. J., Edsall, R. M., and MacEachren, A. M., (1997) "Cartographic animation and legends for temporal maps: exploration and or interaction", Proceedings of 18th International Cartographic Conference. Stockholm: IGU, pp. 253-262, June 23-27, 1997. Krygier, J. B., Reeves, C., DiBiase, D. W., and Cupp, J., (1997) "Design, implementation and evaluation of multimedia resources for geography and earth science education", Journal of Geography in Higher Education, 21(1), 17-39, 10/27/1999. Larkin, J. H., and Simon, H. A., (1987) "Why a diagram is (sometimes) worth ten thousand words", Cognitive Science, 11, 65-100. MacEachren, A. M., and Ganter, J., (1990) "A pattern identification approach to geographic visualization", Cartographica, 27(2), 64-81. Moellering, H., (1976) "The potential uses of a computer animated film in the analysis of geographical patterns of traffic crashes", Accident Analysis and Prevention, 8, 215-227. Morikawa, K., and MacBeath, M. K., (1992) "Lateral motion bias associated with reading direction", Vision Research, 32(6), 137-141. Newman, S., and Lamb, R., (1986) "Eye movements and sequential tracking: an effect of reading experience", Perception and Motor Skills, 63, 431-434. Peuquet, D. J., (1994) "Its about time: a conceptual framework for the representation of temporal dynamics in geographic information systems", Annals of the Association of American Geographers, 84(3), 441-461. Peuquet, D., (2002) Representations of Space and Time. New York, NY: Guilford, p. 379.
Design of Interactive Spatio-temporal Representations
589
Peuquet, D., and Qian, L., (1996) "An integrated database design for temporal GIS", 7th International Symposium on Spatial Data Handling, pp. 2.1-2.11. Tweedie, L., Spence, R., Dawkes, H., and Su, H., (1996) "Externalising abstract mathematical models", In: Bilger, R., Guest, S., and Tauber, M. J., (eds.), CHI'96: Conference on Human Factors in Computing Systems. Vancouver, BC: ACM/ SIGCHI, Online: http://www.acm.org/sigs/sigchi/chi96/proceedings/papers/Tweedie/lt 1txt.htm van den Worm, J., (2001) "Web map design in practice", In: Kraak, M.-J., and Brown, A., (eds.), Web Cartography. London: Taylor & Francis, pp. 87-108.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 31
User-centered Design of Collaborative Geovisualization Tools Sven Fuhrmann & William Pike, GeoVISTA Center, Department of Geography, The Pennsylvania State University, 302 Walker Building, University Park, PA 16802, USA
Keywords: collaborative geovisualization, user-centered design, usability, computersupported cooperative work, video conferencing, Delphi method
Abstract The design of geovisualization tools that support collaboration among multiple users has become a major research focus in recent years. Designing and assessing interfaces for collaborative tools is more difficult than developing single-place and single-user interfaces, since the differing aims of concurrent users and their different work situations make it difficult to conduct controlled studies using traditional methods for user-centered design (UCD). Nonetheless, UCD is more important for collaborative tools, since they must support interaction between users. This chapter outlines three approaches to facilitate UCD between distributed users and developers of collaborative tools: video conferencing, desktop sharing, and asynchronous discussion. Additionally, we introduce a case study that illustrates how these approaches have been implemented to aid remote collaboration in the environmental sciences. Our goal is to engage users in the geovisualization tool design and evaluation process by merging online and traditional (face-to-face) usability assessment techniques.
31.1
Introduction
Collaborative geovisualization - the process of approaching geovisualization tasks through group effort - has become a major research focus and part of an international research agenda in recent years (MacEachren, 2002; MacEachren and Kraak, 2001). Many geospatial visualization tasks, especially those that are exploratory in nature, such as oil field exploration or land use planning, require the contribution of numerous people with diverse expertise (Mark, 1999; MacEachren and Brewer, 2001). Often, it is difficult for all collaborators to meet at the same time in the same place, and if experts are needed for special consultation, collaboration can be all the more difficult to achieve. These 591
592
S. Fuhrmann & W. Pike
Table 31.1. Properties of collaborative systems (Ellis et al., 1991). Same time
Different time
Same place
Face-to-face interaction (group meetings)
Asynchronous interaction (poll about a landscape planning design proposal)
Different places
Synchronous distributed interaction (video conferencing)
Asynchronous distributed interaction (e-mail, list servers)
situations generate the need for synchronous and/or asynchronous collaborative geovisualization tools that support work in distributed locations. Despite the clear need for collaborative tools, the geovisualization community has not adequately addressed either (i) methods to support analysis and knowledge construction activities for teams of analysts and decision makers (Slocum et al., 2001) or (ii) software design and evaluation techniques suited to develop such collaborative tools. This chapter is concerned with the second of these needs, emphasizing the possibilities for novel means of communication between developers and users during the design process. The field of Computer-Supported Cooperative Work (CSCW) has tackled the problem of distributed synchronous and asynchronous collaboration for many years (Preim, 1999). The aim of CSCW is to design and evaluate tools that support domain-specific tasks and social aspects of work, often among distant partners (Shneiderman, 1998). Ellis et al. (1991) organize the types of CSCW in a time-space matrix (Table 31.1). The design process for tools that support one or more of the different styles of interaction in this matrix is more complicated than that for single-user interfaces (Shneiderman, 1998), in part because the number of users (and the unique work practices each brings) makes it difficult to design one tool that supports the different ways that people work together. As a result, user interfaces that support collaborative geovisualization have often been designed from the system developer's view of the visualization problem, rather than that of the user or the domain expert. Until recently, human cognitive, communicative and usability issues were frequently ignored. To remedy this shortcoming, Shneiderman (1998) suggests that traditional assessment methods from psychology, behavioral sciences and sociology can support the design and evaluation of collaborative tools. We believe this is true, but also hold that developers can leverage online assessment techniques that are themselves collaborative. A number of research questions are motivated by the need to increase users' involvement in the design process: How can we develop and use new, effective assessment methods for synchronous and/or asynchronous distributed collaborative geovisualization tools? When should different assessment techniques be applied? How much usability assessment is enough? This chapter attempts to respond to these questions. We begin in w by outlining the basic principles of user-centered software design. Then, w introduces some common collaboration techniques that can be
User-centered Design of Collaborative Geovisualization Tools
593
applied to the development and assessment of collaborative tools. In w we propose one scheme for integrating these assessment techniques into the user-centered design (UCD) process. A case study that illustrates how the communication techniques discussed in w have been applied in a real-world setting is presented in w Recognizing that not all geovisualization researchers need to be specialists in usability engineering, we aim to create some general guidelines for the design of collaborative tools.
A Basis for User-centered Collaborative Geovisualization Tool Design
31.2
For many years, software developers used so-called style guidelines for product design that often went no deeper than look-and-feel specifications (the appearance and behavior of a set of graphical user interface components). These guidelines were sometimes an amalgam of approaches from different application domains and were usually unconcerned with how the target audience of users actually worked. The products built under these guidelines often failed to match domain users' abilities and requirements, making many of the tools unsuitable for their intended tasks (Helander et al., 1997). Like CSCW, the field of Human-Computer Interaction (HCI) aims to increase the effectiveness and efficiency of interaction between humans and their computer tools (Preim, 1999), and offers research that can benefit geovisualization tool design; see Fuhrmann et al., this volume (Chapter 28). Multi-disciplinary research in HCI addresses all aspects of interactive systems design, implementation and evaluation (Helander et al., 1997; Mayhew, 1999). Here, we focus on one component of the HCI field that has relevance to collaborative geovisualization development: UCD. UCD is concerned with developing and assessing usable and useful interaction between humans and computers by understanding the work practices, tasks, and goals of domain users (Rubin, 1994). Other terms for UCD include usability engineering, human-factors engineering, and software ergonomics (Nielsen, 1993). The basic tenets of UCD have been summarized as five attributes of usability (Nielsen, 1993; Shackel, 1991). Software must (i) be easy to learn, (ii) be efficient to use, (iii) be easy to remember, (iv) limit user errors and (v) be pleasant to use (Nielsen, 1993). These attributes influenced the ISO standard definition of usability, where "usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use" (ISO, 1998). Usability can be seen as an overall property of a system, where: 9 9
e f f e c t i v e n e s s defines the extent to which the intended goals are achieved; efficiency describes the time, money and mental effort put into reaching these
goals; 9
s a t i s f a c t i o n expresses a user's opinion of a system's performance (ISO, 1998).
Usability and usefulness cannot be achieved easily by waiting until the end of the design process to involve users, as might be done when domain users are brought in to
594
S. Fuhrmann & W. Pike
Figure 31.1. Stages in the user-centered software engineering cycle (after Torres, 2002). test prototypes or finished products. While such testing is vital, it is equally vital to involve users from the very beginning of the application design process, that is, it is necessary to center the design on the users. To this end, Hix and Gabbard (2002) describe UCD as a "process by which usability is ensured for an interactive application at all phases in the developing process". UCD attempts to enhance usability through a fourfold method: 1. 2. 3. 4.
place an early focus on users and tasks; apply iterative and participatory design; measure a product empirically through user testing; modify the product repeatedly (Gould and Lewis, 1987; Rubin, 1994).
These four principles have been expanded to a set of seven (often iterative) steps toward software deployment (Figure 31.1). These steps (Dix et al., 1998; Hackos and Redish, 1998; Torres, 2002) include: 9
Planning
-
discuss deliverables, technical approaches, milestones and
timetables. 9
9 9 9
User-task analysis - identify the user domain and its associated physiological, psychological social requirements; create a complete description of tasks, subtasks, and actions required to use a system. Conceptual d e s i g n - identify the features of the tool under development. Prototyping - create a model or simulation of the intended tool. Expert guidelines-based evaluation - use expert heuristics or comparison of new and existing tools to identify potential usability problems.
User-centered Design of Collaborative Geovisualization Tools
9
9
595
u s a b i l i t y e v a l u a t i o n - formative evaluation involves domain users in task-based scenarios, discussing and testing a prototype. Qualitative and quantitative results allow the refinement of the prototype. Summative evaluations are often conducted at the end of the development cycle, when the design tool is tested against other comparable tools. I t e r a t i o n a n d d e p l o y m e n t - the above design process might require several iterations before a tool meets domain requirements sufficiently that it can be deployed to users. Formative and summative
These basic steps in user-centered application development should also be followed during the design of collaborative geovisualization tools; UCD is all the more necessary when tools must support interaction among collaborators as well as between users and computers. Interfaces to visualization tools that support cooperative, social interaction will generally be quite different from the single-user interfaces with which designers are usually more familiar. When planning or executing activities collaboratively, users must be able to present and defend their beliefs to each other, so understanding both the domain and the practices of users is critical to successful systems. The novelty of the user interfaces and the distributed nature of collaborative tools might require new or modified assessment methods in many stages of the product development cycle. A basis for such new or modified methods is found in well-established user, task and usability assessment methods, such as interviews, focus groups, user observations, interaction logging and performance measurements. Focus groups are a common example of traditional assessment techniques others are described in the in the introduction to this section Fuhrmann et al., this volume (Chapter 28). Focus groups are a low cost, informal and efficient qualitative method for user interface development, and are often conducted during formative usability evaluation. Focus groups were originally used exclusively for group product and service assessment, but are now commonly applied in HCI research (Nielsen, 1993). The overall goal is not to reach consensus, or to solve a specific problem, but to identify user preferences, emotions, novel ideas and errors. In geovisualization research, focus groups have been applied to several aspects of interface design (Monmonier and Gluck, 1994; Harrower et al., 2000; Fuhrmann and MacEachren, 2001). Focus groups often consist of 6 - 1 0 domain-specific users and one moderator. In the course of a session, users are introduced to the topic of interest and provided with a prototype (Nielsen, 1993). The most important aspect of a focus group is the discussion. Here, users should be able to formulate ideas and discuss issues freely while the moderator leads and focuses discussion. Usually a set of predetermined questions is prepared, including some probes to encourage everyone to express different views and to stimulate the discussion among all participants (Krueger, 1998). Focus groups were developed as an alternative to interviews where closed questions limited the responses, and focus groups that consist of individuals who do not know one another are considered to be more vivid and free from hierarchical burden (Monmonier and Gluck, 1994). While focus groups seem to be the ideal method to assess collaborative geovisualization tools, as they themselves involve collaborative interaction, they are
596
S. Fuhrmann & W. Pike
difficult to implement because collaborators frequently do not work in the same place. Today's large interdisciplinary research projects often involve domain experts who are distributed throughout a country, or even around the world. Bringing these domain experts together for focus group usability sessions is time consuming and expensive. Moreover, in-person focus groups might not be the best method to design collaborative tools that demand remote interaction. The distributed nature of collaborative work suggests assessment methods that simultaneously enable and measure collaboration between participants. Online synchronous and asynchronous interaction methods can be used to identify the preferences and work practices of domain experts involved in usability testing, both before and during their work with the distributed collaborative geovisualization tools under development. In w we discuss techniques from CSCW that can be applied to geocollaborative usability assessment.
31.3
Achieving User-centered Design through Online Collaboration
The design process we described above emphasizes early and frequent interaction between software developers and potential users. However, implementing and assessing UCD principles for collaborative visualization tools can be complicated by the physical distance between users. As a result, the development of such tools benefits from a suite of interaction techniques suited to distributed use. Here, we outline three approaches to facilitate interaction among distributed users and developers of collaborative tools during the design process: video conferencing, desktop sharing, and asynchronous discussion.
31.3.1
Video conferencing
Same-time, different-place interaction usually takes one of several forms: telephone conferences, online chat, and video conferences. Of these, video conferencing offers the greatest similarity to the face-to-face meetings commonly used in the software design process; video conferences also afford enhancements to verbal communication that allow the geovisualization tools under development to be integrated into design discussions. The most straightforward style of video conferencing involves transmitting audio and video signals between two or more locations. Such conferences enable designers to conduct interviews or focus groups with users without convening the users in one place. Video conferences transmit a speaker's image and voice to other participants over the Internet, and most video conferencing systems will automatically switch the audio and video signal displayed on a participant's desktop to that of the person who is speaking. Conference participants can also tile the video signals from all other participants on their desktop, enabling them to detect body language or other subtle forms of communication collaborators display while another is speaking. Tool designers can use this tiled view of the user group to scan the audience much as they would scan the room during a focus group.
User-centered Design of Collaborative Geovisualization Tools
597
Video conferencing systems can involve high one-time costs for purchasing and installing cameras and support equipment, although the conferences themselves can be inexpensive to run. High-quality video cameras for desktop PCs can cost several hundred dollars, while the server hardware and software to manage conferences can run into the tens of thousands of dollars. A less expensive alternative is to rely on commercial providers of conference server time; in this case, participants need only to be provided with cameras, although commercial conference services charge a per-minute fee. Typical network speeds are sufficient for smooth video and audio signals, especially with compression technology available on higher-end cameras, although dial-up connections (e.g., 56 kbps) generally do not suffice. Studies that have explored the merits of video communication over face-to-face meetings and audio-only conferences suggest that video tools entail little degradation in the quality of interpersonal communication. In the context of software design, trust between designers and study participants (as well as among participants themselves) is a key to elicite useful feedback; Muhlfelder et al. (1999) found that trust levels among video conference participants were similar to those in audio or face-to-face meetings. Compared to audio-only discussions, video meetings can provide increased discussion fluency by allowing participants to provide both visual and auditory cues to "turn-taking" (Daly-Jones et al., 1998).
31.3.2 Desktop sharing Desktop sharing enables collaborators to transmit the contents of their workstation desktops, including all of the files and applications accessible from the desktop. Each user in a desktop sharing session can view and manipulate resources on other users' machines. As a synchronous collaboration technique, desktop sharing can be used in conjunction with video conferences; when combined, such conferences can offer functionality that is difficult (and often impossible) to replicate in same-place usability assessment sessions. Desktop sharing can provide a real-time view into how users manipulate a visual display. Desktop sharing requires little infrastructural investment; many video conferencing systems have built-in desktop sharing functionality. Even without simultaneous video or audio streams, inexpensive or free utilities (such as Microsoft NetMeeting) allow participants in the design process to simultaneously engage in remote desktop manipulation, text chat and whiteboard sketching. Because desktop sharing utilities allow users to manipulate each other's software remotely, tool-sharing utilities do not necessarily need to be built into collaborative geovisualization environments. There may be cases, however, where standard desktop sharing packages do not provide sufficient functionality to suit a design team's needs; for example, most packages do not keep track of which user is manipulating a display or maintain a history of each user's interactions. In such cases, designers can integrate support for data sharing protocols into their geocollaborative tools, allowing them to build custom usability assessment applications. By logging the interactions of multiple users engaged in desktop sharing, designers can mine usability data even after the software has been deployed.
598
S. Fuhrmann & W. Pike
Desktop sharing supports all phases of the UCD process; it is a two-way technology that allows both usability engineers and test participants to share software tools. For instance, rather than deploying collaborative tools at test participants' remote sites, a software designer can install a single copy of a prototype on his or her workstation and observe distributed test subjects as they remotely manipulate it. The engineer can observe how readily subjects find controls, whether or not they apply them correctly, and how they interpret the outcomes. Alternatively, a usability assessment participant might show other participants or tool designers how he or she uses new or existing software, including the visualization tasks performed by the participant. Desktop sharing also allows designers to see how their products are integrated into a subject's workflow; that is, how does the subject apply the product in context with the other tools used to complete his or her tasks? Even before prototypes have been developed, usability engineers can perform remote task analyses of study participants by requesting that they share their desktops for a period of time as they complete their work. Qualitative observation of the participants' mouse movements, choice of tools, and order of operations can reveal much about how the subject performs his or her work. When desktop sharing is done as an auxiliary to an audio/video signal, participants can narrate their tasks and designers can ask clarifying questions. Collaborative geovisualization tasks often require that participants are able to manipulate a common visual display while maintaining a private workspace and that the common visualization distinguishes between the actions performed by different users. Engaging in collaborative visualization tasks through desktop sharing supports both of these aims. It is difficult to offer private areas of a tool where participants can experiment with ideas before presenting them to others when the tool centers on a single large display around which participants are gathered. Participants engaged in remote collaboration, however, already have a local workstation that serves as a personal workspace. Users can choose which applications on their desktop to share and which to keep private, and can thus maintain a second instance of the visualization tools running that only they can see. When the user has created a visualization to share, he or she can move it from the private to public realm by switching on desktop sharing for that instance, or by copying it into an existing shared window. Distinguishing among the participants in a collaborative activity is also much more straightforward when users connect remotely than when they all work at a single display, since their machines are already identified by unique addresses. Customized desktop sharing applications could change the color or text label on a cursor depending on the address from which the instructions are being received. If logfiles associate each software event with the address of the user whose actions fired it, users or software designers can filter views of the workspace by different participants. A primary advantage of the synchronous conferencing model, whether desktop sharing or video conferencing, is its spontaneity. Tool designers can convene user groups more frequently and get more rapid feedback on prototypes than is often possible with traditional meetings. When usability evaluations are performed by convening subjects at a single location, there is usually a limited amount of time for which the subjects are available. Given these time constraints, it is difficult to make changes to tools under development and gain feedback on these changes while the subject group is still
User-centered Design of Collaborative Geovisualization Tools
599
convened. By removing many of the barriers of distance, video conferencing enables rapid prototyping. Video conferencing and desktop sharing can speed iteration over the seven steps outlined in w and can potentially improve the quality of software products by bringing usability testing participants together more quickly, and more often.
31.3.3
Asynchronous computer-mediated communication
Not all stages in the design process require (or even benefit from) real-time communication. In some cases, schedules do not allow frequent live conferences; in others, video conferences can be susceptible to the same problems as face-to-face meetings, including personality conflicts that allow particularly vocal participants to dominate discussion. Test participants may be uncomfortable or unwilling to share their views openly, whether because of personality, status relations, intimidation, or anonymity and privacy concerns. For example, participants in a task analysis session may include both managers and employees of an organization. The employees may feel uncomfortable sharing their work practices with usability engineers while their supervisors are listening in. It may also be common for employees to devise their own problem-solving strategies that differ from accepted practice in an organization; subjects may not share openly if they fear reprisal from managers. Lastly, synchronous communication often does not give participants time to reflect at length on their work practices. Whenever synchronous communication is impossible or undesirable, asynchronous alternatives are available. Different-time, different-place collaboration should allow participants to contribute to stages in a usability assessment on their own schedule and should take advantage of the "breathing room" offered by asynchronous communication. Participants can be asked longer or more detailed questions and can spend more time composing answers. They should also be able to contribute whenever an idea comes to mind - not just when a conference is scheduled. A popular planning technique that is well-suited to asynchronous electronic usability testing is the Delphi method (Linstone and Turoff, 1975). This method provides guidelines for group exploration of complex problems and is usually conducted via traditional mail or e-mail. The Delphi method is appropriate for collaborative tasks where a group of experts iteratively explore opinions and recommendations to arrive at a goal (Turoff and Hiltz, 1995). This goal is not necessarily consensus, but a refined set of beliefs that reflect the breadth of expert opinion. The Delphi method has often been used to structure communication among participants in a software design activity, where a designer moderates a series of Delphi procedures with user groups. Bourque (2002), for instance, engaged software engineers in a series of Delphi activities to define best practices for the tool development process. There, the Delphi method served as a means to elicit, refine, and evaluate a set of software design principles from a panel of experts. Other approaches have used strategies similar to the Delphi method, such as "Issue-Based Information Systems" that support the process of argumentation between participants in the design process (Conklin and Begeman, 1989). More recent work has focused on developing Web-based Delphi tools (Pike, 2001; see w
600
S. Fuhrmann & W. Pike
Applying the Delphi method to UCD involves convening a panel of test participants (perhaps, the same participants who also take part in video conferences) and devising a series of questions that guide the group through the exploration of a problem. In these respects, the Delphi method can be seen as a specialization of conventional faceto-face focus groups; the differences lie in the Delphi method's emphasis on structured communication, anonymity of participants, and iterative refinement of ideas. The questions posed through Delphi activities are generally not those to which there are easy answers. A question such as "Describe the major tasks involved in your work" might ask participants who share common job titles to first brainstorm lists of their job-related activities, then iteratively (and collaboratively) refine those lists to create a set of welldefined tasks that might be the basis of later desktop-sharing sessions. Delphi techniques are also appropriate for interface evaluation. A moderator might ask a general question such as "What are the strengths and weaknesses of this software product?" or more specific ones such as "What tools or operations are missing from this product?" Rather than producing simple lists of responses as a traditional survey might, the Delphi method allows the group, under the guidance of a designer-moderator, to fully explore the breadth and depth of the question to generate richer sets of opinions and their justifications. The results of these activities are often more well-developed (and participants more confident in them) than when subjects respond to simple surveys, because Delphi activities encourage participants to discuss, synthesize, and refine their beliefs. While face-to-face meetings and video conferences can be influenced strongly by group dynamics, a central tenet of the Delphi method is the preservation of participants' anonymity. By stripping responses of personally identifying information, moderators can increase the willingness of participants to share information that they would otherwise withhold, such as work strategies, personal preferences, or disagreements with other participants.
31.4
Applying Online Collaboration Tools in the Design Process
The synchronous and asynchronous collaboration techniques discussed above facilitate interaction between tool designers and user communities involved in the design process. We now discuss how different styles of interaction are suited to different stages of UCD, based on the development cycle illustrated in Figure 31.1.
31.4.1
Planning
Planning for collaborative tool development is generally the purview of the design team. In addition to planning for the software development itself, designers should decide on their collaboration needs, including technical equipment and schedules. In addition, in any collaborative activity (and especially those in which participants are not collocated) the quality of a group's output depends in large part on the quality of the group's interaction. Even the best collaborative tools cannot overcome poor group dynamics.
User-centered Design of Collaborative Geovisualization Tools
601
It may be useful to allocate time during the project-planning phase to establishing social familiarity among participants and software designers (Turoff et al., 2002). Once a potential user group has been identified, for instance, software designers could consider holding an initial face-to-face meeting or video conference. During this meeting, designers might consider simple activities that allow participants to explore each other's background, interests, and expertise. Low-pressure team building activities at the start of collaboration will familiarize participants with each other and with the design team, encouraging better communication once deeper collaboration begins.
31.4.2
Task analysis
To properly engage the domain user, software designers should perform thorough task analyses of user groups (Kirwan and Ainsworth, 1992). Both in-person and remote collaboration techniques can be applied at this stage. The most thorough analyses can be performed in person, when designers fully immerse themselves in the work environment (Hackos and Redish, 1998). However, when much of a potential user's work is completed at a computer workstation, remote desktop sharing can transmit a user' s work patterns to a designer's location. When the user shares his desktop with a designer, the designer can view the user's mouse movements, how he or she balances simultaneously running programs, what software operations are used and in what order. For collaborative activities, task analysis need not be one-on-one; a group of test subjects can all share a common desktop as the designers observe how they might work together to achieve common goals using existing software. Through complementary audiovisual feeds, the designer can ask questions that encourage users to narrate their tasks. Since desktop sharing also allows participants to use sketch programs and electronic whiteboards, they can also collaboratively support creating organizational or task hierarchy charts.
31.4.3
Conceptual design
During the conceptual design phase, tool developers and users translate the tasks identified earlier into functionality specifications for the visualization systems under development. This stage aims to bridge the understanding of users' work environments gained during task analysis and the formal requirements of visualization software to support this work. Conceptual design activities might employ asynchronous Delphi exercises that focus communication between designers and participants toward a core set of work practices and corresponding visualization tasks. For example, a user or design group might devote time over the course of several weeks to identify, refine and agree on a set of key features for the tool under development. Moderated Delphi discussions can provide the structure necessary to negotiate software specifications.
31.4.4
Prototyping and expert guidelines-based evaluation
User-centered activities during the prototyping phase may include video conferences or other synchronous communication tools that enable designers to gain feedback on proposed functionality and visualization schemes. The actual process of software development may also reveal problems that require immediate responses from test
602
S. Fuhrmann & W. Pike
subjects. Using synchronous communication (perhaps, with data sharing features enabled), designers can demonstrate these problems to users, facilitating rapid prototyping. Users can manipulate prototypes on the designer's desktop and provide video commentary. By integrating utilities that track mouse movements and event firing, designers can augment their qualitative observation of usage patterns with quantitative data. Asynchronous Delphi-style techniques could be applied for expert guidelines-based evaluations and longer-term evaluative discussions, including structured discourses on the fundamental design issues that arise during users' long-term tests of prototypes.
31.4.5
Formative and summative usability evaluation
The final stages of software evaluation may benefit most from face-to-face interaction. Nonetheless, there are situations for which remote collaborative usability assessment can be effective. Working through the task-based scenarios outlined during formative evaluation can be accomplished through desktop sharing and video conferencing. Summative evaluation procedures can employ similar sentinel utilities as those used during task analyses, allowing designers to compare the new tool's ease of operation and time-to-completion for common tasks with those of existing software tools.
31.5
Geocollaboration Case Study: The HERO Intelligent Networking Environment
The online collaboration techniques introduced above have been deployed in a realworld setting. The Human-Environment Regional Observatory (HERO) project at Penn State's GeoVISTA Center is developing a collaborative environment that enables communication among researchers distributed around the US and the world. This environment includes synchronous components for real-time conferencing as well as a Web-based Delphi method application used for longer-term planning tasks. The HERO tools are presented here as a case study to illustrate ways in which teams of domain experts and novices can interact remotely to achieve common goals. This experience can inform the integration of such tools into the geovisualization software design process. HERO's video conferencing implementation employs desktop video cameras in participants' offices. By placing hardware in personal spaces, rather than in centralized conference rooms, participants feel more in control of the technology. This control can enhance the naturalness of online meetings and the spontaneity of discussions, since each participant is able to initiate a meeting easily. Subjects may also produce more accurate descriptions of their work when in a familiar, work-related environment ~ than when in an anonymous conference facility; their familiar surroundings can serve as visual prompts when asked to describe parts of their work. Each camera-equipped computer also includes desktop-sharing software that allows participants to remotely view and control each other's machines. This sharing functionality is frequently used to demonstrate software features or analytical methods. Figure 31.2 shows two conference participants engaged in a desktop-sharing activity
*I_
8 0 5. zc -. N
g. 0 3
Figure 31.2. Participants in a synchronous video conferencing and desktop sharing activity can simultaneously engage in audiovisual interaction while viewing how each manipulates software components to complete tasks.
604
S. Fuhrmann & W. Pike
involving the visualization of classes derived from remotely sensed data; their audiovisual interaction is augmented by the ability to view how each manipulates the visualization tools. Undergraduate students working on the HERO project at universities around the country have become adept at asking and answering usability questions through desktop sharing, and the technology has been particularly effective for the collaborative creation of prototype map visualizations in GIS. User-centered tool development takes on a new meaning when users themselves can initiate and complete informal usability assessments using technology provided by project leaders. Test participants may occasionally wish to explore a product or task description on their own, without the guidance (and sometimes interference) of usability engineers. Placing control over some or all of a collaborative assessment in the hands of users might be a key to next-generation assessment tools (Turoff et al., 2002). HERO's experience with video conferencing has revealed several shortcomings in the technology. First, participants are often unfamiliar and initially uncomfortable with the conferencing technology. Participants must learn when and how to speak, and may be so distracted by learning the technology that they spend little time in substantive communication. Second, conversations in video conferences can be stilted, even after practice. There is often a lag time between when participants speak and when their voice is heard over the network, causing frequent interruptions and calls to repeat what was just said. Nonetheless, the ability of these conferences to support remote task analysis and rapid prototyping make them tremendous assets to the UCD process. HERO has also created an electronic Delphi method tool (e-Delphi) that allows team members to explore group thought asynchronously over the Web. The e-Delphi system (Figure 31.3) enables participants to engage in structured discussions that take place over a time period much longer than a face-to-face meeting or video conference. Through iterative argumentation, Delphi participants express their own opinions while gradually understanding those of others. The e-Delphi system stresses a user-centered approach by allowing any registered user to create and moderate a new activity. Participants in usability assessments can start their own internal discussions. The e-Delphi implementation also affords various styles of voting that allow participants to express support or rejection of ideas, and moderators can choose the level of anonymity to preserve (no identifiers, pen names, or real names) during an activity. Access to e-Delphi activities is controlled by membership in groups, and participants are free to create new groups or subgroups to explore a particular topic in more detail. These groups may be divided according to geographic location or domain expertise, and later can be merged when subgroups wish to share the results of their local discussions in an online plenary session. A goal of assessment tools, including e-Delphi, should be to offer a flexible framework that user groups can customize to suit their domain and their working style (Turoff et al., 2002). A key feature of the e-Delphi implementation that supports geovisualization tool development is the ability to post images of, for example, views on data generated by the tools being tested. These samples can serve as the focus of discussion as participants comment on their strengths. System designers can also gather feedback on snapshots of interfaces or tool palettes in the flank manner that anonymous collaboration allows, while
User-centered Design of Collaborative Geovisualization Tools
605
Figure 31.3. The e-Delphi asynchronous collaboration system provides a variety of utilities that can aid the creation of a long-term discourse on software specifications and usability.
making substantially fewer demands on time and schedules than a real-time conference requires.
31.6
Summary
In this chapter, we propose the adoption of synchronous and asynchronous online interaction techniques that can extend and enhance current usability assessment methods for the UCD of collaborative geovisualization tools. Most current UCD assessment methods are derived from recommendations made for single user workspaces
606
S. Fuhrmann & W. Pike
(Nielsen, 1993; Hackos and Redish, 1998). Although these methods provide a basis for creating new collaborative geovisualization tools they do not fully support: 1. 2.
collaboration between groups of usability test subjects and software designers who are often distributed in space; usability measurements of geocollaborative tools running in a "real-world" distributed mode.
While the traditional framework for UCD remains, the usability engineering process could be enhanced for geocollaborative tools. Face-to-face assessment methods will continue to play a vital role in software design, but synchronous and asynchronous CSCW techniques can augment the UCD process for collaborative geovisualizations. We described three common online techniques that can be implemented to achieve the above goal. Videoconferencing and desktop sharing utilities are widely available and can be inexpensive to run in light of rising travel costs. They also enable test subjects to participate in situ, allowing them to share their work practices with noncollocated designers. Asynchronous computer-mediated communication tools such as electronic Delphi activities can augment the discussions and observations that take place synchronously by encouraging participants to reflect on task descriptions, software specifications, and design recommendations at greater depth. The argument for incorporating these and other CSCW methods into collaborative geovisualization application development is supported by the iterative nature of UCD. Electronic collaboration enables much more rapid iteration, as it becomes easier and faster to convene user groups. Studies that have explored the quality of group interaction and satisfaction of participants in online collaborative activities have suggested that electronic collaboration offers similar decision quality to face-to-face meetings (Fjermestad and Hiltz, 1999), with participant satisfaction the greatest when participants were able to mix asynchronous and face-to-face collaboration (Ocker and Fjermestad, 1998). It is not known whether similar satisfaction measures would be obtained if face-to-face meetings were replaced with synchronous Web conferences. Future work on usability assessment methods for collaborative tool design should consider integrating CSCW techniques into all phases of the design process. In particular, the geovisualization software development community should outline a research agenda that considers: 9 9 9
the suitability of various online interaction techniques to each stage of the UCD cycle; the quality of usability information that can be gained remotely compared to inperson interviews, demonstrations, observations or focus groups; the integration of usability assessment utilities such as mouse tracking, event listeners, or time-to-completion clocks for common tasks that help developers collect usability information remotely;
User-centered Design of Collaborative Geovisualization Tools
607
the development of advanced remote collaboration techniques that enhance the sense of presence among collaborators. Research on collaborative geovisualization tool design has just begun and offers many interesting research challenges. We have introduced an initial set of online usercentered assessment techniques that can be expanded into a more formal methodology that merges electronic usability assessment with traditional techniques. The HERO project can serve as a testbed for developing and deploying both geocollaborative tools and online usability assessment methods. Our goal is to integrate the results from our research in usability guidelines for collaborative geovisualization tool design. Such guidelines would allow software engineers to select specific methods of evaluation at each stage of the development process, thereby aiding the production of usable collaborative tools for domain experts.
Acknowledgements This material is based upon work supported by National Science Foundation under Grant No. BCS-0113030 and on work supported by the US Geological Survey. The HERO project is supported by the National Science Foundation under Grant No. SBE-9978052. Many colleagues have contributed to development of these ideas; in particular we would like to acknowledge Chaoqing Yu, Alan MacEachren and Isaac Brewer.
References Bourque, P., Dupuis, R., Abran, A., Moore, J. W., Tripp, L. L., and Wolff, S., (2002) "Fundamental principles of software engineering - A journey", Journal of Systems and Software, 62(1), 59-70. Conklin, J., and Begeman, M. L., (1989) "glBBI: a tool for all reasons", Journal of the American Society for Information Science, 40(3), 200-213. Daly-Jones, O., Monk, A. F., and Watts, L., (1998) "Some advantages of video conferencing over high-quality audio conferencing: fluency and awareness of attentional focus", Int. J. Hum.-Comput. Stud. 49(1), 21-58. Dix, A. J., Finlay, J. E., Abowd, G. D., and Beale, R., (1998) Human-Computer Interaction. Englewood Cliffs: Prentice Hall. Ellis, C. A., Gibbs, S. J., and Rein, G. L., (1991) "Groupware: Some issues and experiences", Communications of the A CM, 34(1), 680-689. Fjermestad, J., and Hiltz, S. R., (1999) "An assessment of group support systems experimental research: methodology and results", Journal of Management Information Systems, 15(3), 7-149. Fuhrmann, S., and MacEachren, A. M., (2001) "Navigation in desktop geovirtual environments: usability assessment", 20th International Cartographic Conference Mapping the 21st century, Beijing, pp. 2444-2453.
608
S. Fuhrmann & W. Pike
Gould, J. D., and Lewis, C., (1987) "Designing for usability: key principles and what designers think", In: Baecker, R. M., and Buxton, W. A. S., (eds.), Readings in Human-Computer Interaction: A Multidisciplinary Approach. San Mateo, CA: Morgan Kaufmann, pp. 528-539. Hackos, J. T., and Redish, J. C., (1998) User and Task Analysis for Interface Design. New York: Wiley, p. 488. Harrower, M., MacEachren, A. M., and Griffin, A. L., (2000) "Developing a geographic visualization tool to support earth science learning", Cartography and Geographic Information Science, 27(4), 279-293. Helander, M., Landauer, T. K., and Prabhu, P., (eds.), (1997) Handbook of HumanComputer Interaction. Amsterdam: North Holland/Elsevier, p. 1582. Hix, D., and Gabbard, J. L., (2002) "Usability engineering of virtual environments", In: Stanney, K. M., (ed.), Handbook of Virtual Environments - Design, Implementation, and Applications. Mahwah: Lawrence Erlbaum Associates, pp. 681-699. International Standards Organisation (1998) "ISO 9241-11Ergonomic requirements for office work with visual display terminals (VDT)s - Part 11", Guidance on Usability. Gen6ve" International Organization for Standardization, p. 22. Kirwan, B., and Ainsworth, L. K., (eds.), (1992) A Guide to Task Analysis. London: Taylor & Francis, p. 417. Krueger, R. A., (1998) Moderating Focus Groups. Thousand Oaks: SAGE Publications, p. 114. Linstone, H., and Turoff, M., (1975) The Delphi Method: Techniques and Applications. Reading: Addison-Wesley. MacEachren, A. M., (2002) "Moving geovisualization toward support for group work", ICA Commission on Visualization and Virtual Environments Workshop, September 11 - 14, London, p. 10. MacEachren, A. M., and Brewer, I., (2001) "Kollaborative Geovisualisierung zur Wissensgenerierung und EntscheidungsunterstiJtzung", Kartographische Nachrichten, 51(4), 185-191. MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Science, Special Issue on Geovisualization, 28(1), 3-12. Mark, D. M., (ed.), (1999) NSF Workshop Report - Geographic Information Science: Critical Issues in an Emerging Cross-Disciplinary Research Domain. Washington, DC: NSF. Mayhew, D. J., (1999) The usability engineering lifecycle - A practitioner's handbook for user interface design. San Francisco: Morgan Kaufmann Publishers, p. 542. Monmonier, M., and Gluck, M., (1994) "Focus groups for design improvement in dynamic cartography", Cartography and Geographic Information Systems, 21(1), 37-47. Muhlfelder, M., Klein, U., Simon, S., and Luczak, H., (1999) "Team without trust? Investigations in the influence of video-mediated communication on the origin of trust among cooperating persons", Behaviour and Information Technololgy, 18(5), 349-360. Nielsen, J., (1993) Usability Engineering. Boston: AP Professional.
User-centered Design of Collaborative Geovisualization Tools
609
Ocker, R., and Fjermestad, J., (1998) "Web-based computer-mediated communication: an experimental investigation comparing three communication modes for determining software requirements", Proceedings of the 31st Hawaii International Conference on Systems Sciences, pp. 88-97. Pike, B., (2001) The HERO e-Delphi System: Overview and Implementation, State College: GeoVISTA Center, Department of Geography, The Pennsylvania State University, p. 6 (October 2001). Preim, B., (1999) Entwicklung lnteraktiver Systeme. Berlin: Springer, p. 557. Rubin, J., (1994) Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: Wiley, p. 330. Shackel, B., (1991) "Usability-context, framework, definition, design and evaluation", In: Shackel, B., and Richardson, S. J., (eds.), Human Factors for Informatics Usability. Cambridge: Cambridge University Press, pp. 21-37. Shneiderman, (1998) Treemaps for space-constrained visualization of hierarchies. Online: http://www.cs.umd.edu/hcil/treemaps (23/10/03). Slocum, T. A., B lok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., and Hedley, N. R., (2001) "Cognitive and usability issues in geovisualization", Cartography and Geographic Information Science, 28(1), 61-75. Torres, R. J., (2002) Practitioner's Handbook for User Interface Design and Development. Upper Saddle River: Prentice Hall PTR, p. 375. Turoff, M., and Hiltz, S. R., (1995) "Computer-based Delphi processes", In: Adler, M., and Ziglio, E., (eds.), Gazing into the Oracle: The Delphi Method and its Application to Social Policy and Public Health. London: Kingsley, pp. 56-88. Turoff, M., Hiltz, S. R., Fjermestad, J., Bieber, M., and Whitworth, B., (2002) "Computermediated communications for group support: past and future", In: John, C., (ed.), Human-Computer Interaction in the New Millenium. Boston: Addison-Wesley, pp. 279-302.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 32
Towards Multi-variate Visualization of Metadata Describing Geographic Information Paula Ahonen-Rainio, Institute of Cartography and Geoinformatics, Department of Surveying, Helsinki University of Technology, P.O. Box 1200, FIN-02015 HUT, Finland Menno-Jan Kraak, Department of Geolnformation Processing, ITC, International Institute of Geoinformation Science and Earth Observation, P.O. Box 6, NL-7500 AA Enschede, The Netherlands
Keywords: geographic metadata, multi-variate visualization, geovisualization Abstract A visual environment is proposed for the exploratory use of metadata relating to geographic datasets. Metadata has a prominent role when acquiring geographic data for solving spatial problems. Users need metadata only occasionally but when they do, they should be able to use metadata to decide how well the available datasets meet the needs of the intended use. This decision can be supported by interactive exploration of metadata in such a way that different characteristics of datasets can be studied simultaneously. Multi-variate visualization techniques such as the parallel coordinates plot, the scatter plot matrix, star plots and Chernoff faces convey different aspects of the metadata. A working prototype of a visualization environment for metadata is drafted. It combines the multi-variate visualization methods with a map of the region of interest, browse graphics and textual metadata in multiple linked views.
32.1
Introduction
Metadata are useful for searching for relevant data in large geographic data collections. Increasingly, the provision of geographic datasets encourages the supply of metadata and the development of related services, especially now that (national) geographic information infrastructures have come of age (Groot and McLaughlin, 2000) and online GIS is developing (Green and Bossomaier, 2002). Intensive use of metadata requires consistency, availability and usability of dataset descriptions. The ISO 19115 standard (ISO 19115, 2003) has been developed in order to ensure the consistency of the semantics and the structure of metadata elements for describing geographic information. Metadata 611
612
P. Ahonen-Rainio & M.-J. Kraak
services provided via the WWW allow users to access the metadata available. Usability of metadata depends heavily on who the users are and what are the tasks of using metadata. Tasks and users are the key factors in determining how the metadata are expressed and provided. Geographic metadata services typically provide metadata in a fixed form using the hypertext mark-up language. Examples of national geographic metadata services are the Danish metadata service (Danish National Survey and Cadastre, 2003) and the NCGI metadata service in the Netherlands (NCGI, 2003). Presentation of metadata tends to be achieved on a dataset by dataset basis. Users have very limited opportunity to select the metadata elements in their area of interest and to study metadata of several datasets at a time; metadata development up until now has focused tightly on the logical and technically correct description of data and the rigid requirements of data experts and suppliers. Exploratory use of metadata has been considered only recently (Albertoni et al., 2003). Automated searching methods for geographic metadata under development (Schlieder and V6gele, 2002; Bucher, 2003) can facilitate efficient use of metadata to a certain extent, but in parallel there are situations when individual users need to access and explore metadata. Users looking for geographic datasets that are fit for intended use may neither be able to express their needs explicitly nor formally. They may lack knowledge of data resources available and want to get an overview of supply before detailing their own preferences. Data fitness is often based on multiple criteria with different importance. As long as the criteria and their significance are implicit, the use of metadata cannot be automated, but is based on human processing. Therefore, users should be able to explore and compare a number of datasets at any particular time as well as select the variables from among the metadata elements and change the set of variables during the exploration in order to gain insight into the geographic data resources available. Mechanistic metadata do not meet these needs. Therefore, we propose a visual environment allowing users to explore geographic metadata. Our approach to a visual environment for geographic metadata brings together three research lines: geovisualization, multi-variate visualization and human-centered design of visualization. Users of geographic metadata can be expected to be familiar with maps and diagrams, which represent the spatial data they work with on a daily basis. They are experienced in using such graphic representations to achieve an overview and to gain insight into spatial patterns in an exploratory manner (Kraak and Ormeling, 2002). Metadata frequently have a geo-component and using a similar visualization metaphor with metadata in an exploratory context seems reasonable. Metadata, like complex data used in knowledge construction applications, is highly multi-variate, and thus multivariate visualization techniques can offer some assistance, such as those applied in Information Visualization (Card et al., 1999; Spence, 2001). The aim of these techniques is to reduce the number of dimensions into a comprehensible level and to summarize all the data in a single graphic. Shneiderman (1997) presents the process of seeking visual information as "overview first, zoom and filter, then details on demand". By combining various visualization methods with the technique of multiple linked views different aspects of
Towards Multi-variate Visualization of Metadata Describing Geographic Information 613
data can be conveyed (Buja et al., 1991). Interactive techniques such as brushing and filtering help the users to interpret the graphics and enable them to construct the knowledge and insight following their own thinking. These techniques have been demonstrated in geovisualization environments (Haslett et al., 1990; MacDougal, 1992; Dykes, 1995). The use of alternative visualization approaches such as those suggested above is stimulated by the geovisualization research agenda of the ICA Commission on Visualization and Virtual Environments (MacEachren and Kraak, 2001). Extending these visualization techniques into the visualization of metadata must be done with consideration that users need geographic metadata only occasionally and most of them are not necessarily experts in visual exploration. Interpretation of visualization of metadata has to be intuitive and interaction methods easily adoptable. Involving users in the design process from the beginning is vital in order to ensure users' acceptance of the methods (Beyer and Holtzblatt, 1998; Slocum et al., 2001; Fuhrmann and Pike, this volume (Chapter 31)). In this chapter, design concepts are drafted for a visualization environment for geographic metadata. We first consider the context of metadata visualization, and then focus on the multi-variate visualization methods and their application in a working prototype. A discussion about preliminary user feedback and research needs concludes the chapter.
32.2
Design Constraints of Metadata Visualization
The design requirements for the visualization methods and the exploratory interaction between users and the metadata derive from the three constraining factors: the tasks, the users and the characteristics of metadata. Whilst the constraining factors limit the possibilities of visualization they also help in targeting the design. The following constraints were drafted on the basis of evidence gathered from users through various metadata projects over the past years, such as the development of the Finnish metadata service (National Land Survey of Finland, 2003) and the Methods for Access to Data and Metadata in Europe project (MADAME, 2000). Communicating with users is necessary for refining the draft requirements.
32.2.1
The tasks of using metadata
Geographic data are needed whenever problems to be solved include location. When these data are not immediately available one will have to acquire the missing data from other sources. In this process metadata play a prominent role. Metadata can be used during three stages of the process. First, when searching for a dataset, second, when evaluating the fitness of a dataset for an intended use, and third, when accessing a dataset or transferring it to an information system (Lillywhite, 1991; Guptill, 1999). Visualization has the potential to be of particular use in the first two situations. Different metadata elements are needed for each task. When searching a dataset, its spatial location or extent and its thematic content are the most common search criteria. In the evaluation of a dataset for the fitness for an intended use the criteria depend upon the user and the use case. For example, a research analyst may emphasize the currency and reliability
614
P. Ahonen-Rainio & M.-J. Kraak
of the data whereas an application user may pay most attention to data density and spatial accuracy, and a data administrator may be most concerned by the fees and restrictions upon use. Users may benefit from a visual overview of metadata displaying the data resources available, before searching for the set of potential datasets. During the evaluation a user has to decide whether a dataset fits well enough for an intended use. The evaluation is a kind of a multi-criteria decision-making task, but the criteria vary from case to case. The criteria are linked to the metadata elements, but the users may not have any definite values or relative importance in mind at the beginning of the task. The users may change the criteria, their limiting values and their relative importance during the evaluation. If the described data are not satisfactory the user may make a new search for a wider, narrower or simply different series of datasets, or decide to stop the evaluation altogether. Otherwise, the user selects the most satisfactory dataset and may study its metadata more thoroughly. 32.2.2
Users of m e t a d a t a
Most of the users of metadata are users of GIS applications, who are looking for suitable data for a particular intended use. They may range from end users of well-specified applications to research scientists using a development environment or application toolbox. In addition, application and software developers, data administrators, and data producers use metadata. In each case, users are interested in the geographic data that the metadata describe. Metadata have value only as a means of providing information about the data proper. Therefore, users need to learn something about the geographic data in order to find satisfaction and motivation for the effort of using the metadata again. Users' knowledge about available geographic data is a result of a manifold learning process. In this process, metadata is only one of the sources of information about the data. Other sources providing formal and informal descriptions of data include experienced colleagues, direct contacts with data suppliers, relevant documents and advertising materials, and education. Experience of the process of visualization suggests that enabling users to interact with the information source instead of studying static information will enhance their abilities to synthesize, compare, filter and assess and to increase the likelihood that they will acquire the information they seek (Cartwright et al., 2001). Therefore, the visual environment for metadata must allow users to interact with and focus on those aspects that are most relevant to them. Adequate levels of knowledge in the application domain and GIScience as well as GIS skills are required from users of metadata (Ahonen-Rainio, 2002). However, we do not expect users to be competent with all the details of metadata as they are defined (e.g., in the ISO 19115 standard). 32.2.3
Characteristics of rnetadata
The ISO 19115 standard (ISO 19115, 2003) defines an extensive set of metadata elements for describing the characteristics of geographic data. The standard covers over 400 metadata elements altogether. However, some of the metadata elements are not
Towards Multi-variate Visualization of Metadata Describing Geographic Information 615
meaningful in isolation but only as parts of composites together with certain other elements. The standard recognizes varying needs for metadata by defining some elements as optional and by listing a core set of metadata elements that can be used as a minimum description of data. In addition, it defines conditional metadata elements for alternative descriptions meant for different types of data and gives rules for defining extensions to or a profile of the standard for a community with special needs. According to the ISO 19115 standard, structured text is the main form of metadata. The domain of a significant number of the metadata elements is free text, typically naming or describing an aspect of the data. Some of the textual metadata elements include metric information, for example, the fees element provides not only the amount of money but also the terms and the monetary units. A minority of metadata elements has a domain of metric values. In addition, several metadata elements represent classified data with either nominal or ordinal values. There are also domains composed of a mixture of both nominal and ordinal values. The standard also defines domains of Boolean values, but they appear only in composites of several metadata elements. Because of the various scales of measurement used to record metadata, some values have to be pre-processed for those visualization methods that deal with metric data. This process can be user-driven in an interactive environment. When evaluating datasets, information about data quality is among the most important metadata elements. The ISO 19115 standard follows a flexible approach and defines metadata elements for expressing each quality measure, its evaluation procedure and the resulting value or values. As a result the quality measures of various datasets may differ notably from each other and from the users' conception of quality requirements. Comparing the quality of various datasets when their quality measures differ can be facilitated by ontology-driven systems, such as that proposed by Fonseca et al. (2002), requiring that the ontologies are extended to cover quality aspects. Another problem with respect to comparison are the metadata elements that depend on the value of another metadata element. An example is the set of metadata elements for describing the spatial representation. When the spatial representation type is vector, description of the spatial object type, topology and scale are relevant. When the spatial representation type is grid, the name and resolution of its dimensions shall be described. A sample set of metadata was gathered for the visualization examples that follow. The sample set provides five metadata elements for nine road datasets extending to regions around Helsinki in Finland. The metadata elements are: maintenance frequency, spatial resolution (scale fraction), geometric object type, price and geometric object density. It is a small set for purpose of clarity, but when metadata sets are larger in size the power of visualization is likely to increase.
32.3
Multi-Variate Visualization Methods
Thus far we have seen the wide diversity of metadata elements. To achieve an overview of all of these data and to be able to compare different datasets and draw conclusions we can use multi-variate visualization techniques.
616
P. Ahonen-Rainio & M.-J. Kraak
Many different multi-variate visualization techniques have been developed over the past decade (Card et al., 1999). Of the categories of data visualization techniques proposed by Keim's typology (Keim, 2001; Keim et al., this volume (Chapter 2)) geometrically transformed displays and iconic displays are of interest with geographic metadata. There are methods in these categories that can be used with relatively small volumes of multi-variate data. The geometrically transformed displays include the scatter plot matrix, a commonly used method in statistics, and the parallel coordinates plot (Inselberg, 1985) that is a popular technique in exploratory data analysis and visualization. Star plots (Chambers et al., 1983) and Chernoff faces (Chernoff, 1973) are examples of iconic displays that visualize each data item as an icon and the multiple variables as features of the icons. Here, we consider the parallel coordinate plot, the scatter plot matrix, star plots and the Chernoff faces in the visualization of metadata. Examples of these visualizations are shown in Figure 32.1. A parallel coordinates plot organizes the axes of a multi-dimensional space in parallel and thus enables to assess a large number of variables concurrently. It is useful [daily]
[1:2000]
[complex]
[150e]
[5000pr]
N cTEE"":1 "IF" "'1.SCALE lr" '. L.
'tt. I ,
[annually]
[1:1.6M]
[point]
[28000e]
MAINTENANCE
SCALE
GEOMETRY
PRICE
[800000pr] OBJECT DENSITY
"
~
METRY
"'t :1 t''t :'t "
"
17 "
Figure 32.1. A sample set of metadata for nine geographic datasets has been visualized as a parallel coordinates plot (upper left), a scatter plot matrix (upper right), star plots (lower left) and Chernoff faces (lower right). The axes of the star plots run clockwise in the same order as the sequence shown in the parallel coordinates plot, starting from the upright axis. The Chernoff faces express only three variables: maintenance frequency by the curvature of mouth, scale fraction by the size of eyes and price by the density of eyebrows. Evaluation of the datasets becomes easier if the user first organizes the values of each variable on the basis of their fitness for an intended use.
Towards Multi-variate Visualization of Metadata Describing Geographic Information
617
for identifying relationships between groups of similar data items across a range of variables. Dykes (1997b) and MacEachren et al. (1999) give examples of the implementation of the parallel coordinates plot in a geovisualization context with several forms of user interaction. In a scatter plot matrix of n dimensions all pairs of variables are presented as scatter plots and these are then arranged in a matrix. Scatter plots are considered as a simple method of identifying global trends and local features in data; interpretation of the relation of the plots in the matrix requires interactive techniques such as brushing. In a star plot the axes for variables radiate in equal angles from a common origin. A line segment is drawn along each axis starting from the origin and the length of the line representing the value of the related variable. An outline connects the end points of the line segments, thus creating a star shape. Chernofffaces are multi-part glyphs in the shape of a human face and the individual parts, such as eyes, ears, mouth and nose represent the data variables by their shape, size and orientation (Chernoff, 1973). The idea of the use of the faces is related to the fact that humans easily recognize faces and can see small changes in them without problems. However, according to Morris et al. (1999) faces are not necessarily superior to other multi-variate techniques. Icons can not only be looked at on an individual basis but also in large groups to discover patterns especially if they have a natural location in relation to each other such as a geographic location that can be shown on a map. Examples of the application of Chernoff faces in the visualization of geographic data are the mapping of UK-census data by Dorling (1995). Schroeder this volume (Chapter 24) also provides details on the rationale behind these techniques. The parallel coordinates plot, the scatter plot matrix and the star plots each treat the whole set of variables equally because all the variables have the same graphic representation. However, the ordering of the axes influences the shape of the parallel coordinates plot and the star plots and so possibly their interpretation. Therefore, it is important that users can control the ordering during the exploration so that the variables of interest can be studied in adjacent axes. Values along a coordinate axis in a parallel coordinates plot, a scatter plot matrix and star plots are typically scaled in a linear fashion from the minimum to the maximum value of the related variable. These maxima and minima could refer to the whole set of values or just to that part of the set that is displayed. The first case provides a contextual view especially for users who are unfamiliar with the potential variation of characteristics of geographic data. Andrienko and Andrienko (2001) suggest scaling of the axes based on the medians and quartiles of the variable values or the minimum and maximum among all the values of all the variables for different perspectives during the exploration. However, then the variables must be comparable. Individual metadata elements are not comparable, but could be made comparable via a transformation process. Chernoff faces handle each variable differently. Because the features of the faces are of different importance in terms of our interpretations of the whole, the way in which variables are mapped to the features is critical. Therefore, as with the ordering of the axes in the other methods, the user should be allowed to set the priority for the metadata elements for the visualization. For example, in Figure 32.1 (lower right) we have selected
618
P. Ahonen-Rainio & M.-J. Kraak
the curvature of the mouth, the eye size and the density of the eyebrow as visual variables. These are among the perceptually salient face features (De Soete, 1986). Spence (2001) states that icons have an advantage over textual representation when there is a semantic relation between the icons and the task. There is no semantic relation between face features and the metadata elements linked to them, but the values of different metadata variables can be organized and scaled according to how well they satisfy the user (as is the case in the lower right example in Figure 32.1) and linked to face features that express satisfaction/dissatisfaction so that the expressions of the faces relate to the suitability of the datasets.
32.4
A Prototype of the Metadata Visualization Environment
The multi-variate visualization methods discussed above provide multiple views of the metadata and so support users in their efforts to acquire an insight into the geographic datasets. However, the way in which the variety of options can be interactively controlled needs careful consideration. It is important to strike a balance between versatility and stability that is wide enough to meet the interests of more advanced users, but does not confuse novice users. Here, a prototype of the visualization environment for metadata is drafted. The prototype is being developed and used for testing the visualization methods with users of metadata and refining the design. A prototype scenario (Nielsen, 1995) is given below. A user is searching geographic data for an application that sets certain requirements for the data. The user works with the metadata visualization environment that is connected to a geographic data clearinghouse. The user starts by using the query language of the clearinghouse in order to limit the exploration to those datasets that fit the theme in question and cover the geographic area of interest (Figure 32.2a). The user can either give the name of the area of interest or point the area on a map. The search results in nine datasets and the user should now explore the metadata in order to decide which one of the datasets fits best for the intended application. The user selects those metadata elements that are critical to the application, defines their priorities and organizes the values according to the use preferences (Figure 32.2b). She then opens views with multivariate visualizations of the selected metadata of the datasets that resulted from the search (Figure 32.2c). The user can now interactively brush and filter the metadata, reorganize the visualizations and, when appropriate, reselect the metadata elements and change the preferences. The user can also open and close the different views as and when required. In the view of star plots, each star represents a dataset. As a result of the organization of the metadata values, the length of each axis reflects how well the corresponding variable meets the requirements. Therefore, the user visually searches large and regular stars for further exploration and can mark those by brushing. In the parallel coordinates plot each line represents a dataset. The higher a line is located in the plot the better the dataset meets the user's preferences. If a line is running from high to low values it indicates that the dataset is not consistently satisfactory compared to the use preferences. When the user studies the first two axes of the parallel coordinates she finds more lines with high values than the three brushed ones. She can continue by brushing
Towards Multi-variate Visualization of Metadata Describing Geographic Information 619
Figure 32.2. A prototype is being developed for communicating the ideas of a metadata visualization environment. In this example, the user starts by searching in a geographic data clearinghouse for those datasets that fit the theme in question and cover the geographic area of interest (a). Then the user selects several metadata elements that are critical to the intended application (b) and opens views showing star plots and a parallel coordinates plot of metadata of the nine datasets that result from a previous search (c). The user is brushing the three datasets that seem to be most satisfactory and studies their geographic extents on a map (c). For further details of each of these datasets the user opens views providing a browse graphic and textual metadata (d).
those in order to recognize which are their lower values. She can also filter out those two datasets that have low values on the scale axis and moves the axis of object density adjacent to the scale axis in order to check whether their values correlate. The user opens a spatial view with a background map for orientation purposes in which the geographic
620
P. Ahonen-Rainio & M.-J. Kraak
extents of the datasets are shown (Figure 32.2c). If there is spatial variation in the dataset that can be expressed by metadata, the map view can show the spatial scope of each metadata subset. When the user explores and identifies the most potential datasets she opens views containing browse graphics and textual metadata (Figure 32.2d). The browse graphic is a metadata element that "provides an illustration of the dataset" (ISO 19115, 2003). It can be a sample of a digital map or a visualization of uncertainty of a dataset originally intended for computational purposes. Applicability of techniques developed for visualization of geographic data quality in the context of visualization of metadata should be considered (Wel et al., 1994; Goodchild et al., 1994; MacEachren, 1992; Hootsmans and Wel, 1993; McGranaghan, 1993; Fisher, 1994; Drecki, 2002). The views of the star plots and the Chernoff faces give an overview of the characteristics of the relevant datasets that are available. A user can study the meaning of the individual dimension of star plots as well as the features of the Chernoff faces through mouse-over legends. Effectiveness of these two methods will be tested and compared with users of metadata. The views of the parallel coordinates plot and the scatter plot matrix support comparison of the datasets. Edsall et al. (2001), in a usability assessment comparing the use of parallel coordinate plots and scatter plots in a case of geovisualization, came to a conclusion that a visualization environment is most effective for data exploration when a variety of tools are present. Significant differences between the effectiveness of the two visualization methods were not identified in their study that used a population of researchers. These two methods will be tested further to determine whether there is a difference for use in more specific tasks and with occasional users such as those in the case of metadata exploration.
32.5
Initial Assessment
It is essential that a user understands the objectives of the visual exploration and the meaning of what is visible in the patterns displayed. Novice users of geographic data have limited knowledge of the details of metadata and any user of geographic metadata may have limited experience with exploratory tools. Therefore, any visualization must be straightforward and uncomplicated in order to support rather than obstruct understanding. If the tools of metadata visualization lack users' acceptance they will be of little value. To address the issues above, a working prototype is being developed and introduced to users for empirical assessment and comment. Users' comments are vital, to determine firstly whether the techniques are comprehensible and secondly to determine the degree to which users may be able to benefit from metadata visualization. Subsequently, comments are needed in order to refine design requirements for a prototype visualization environment. As a first step in an iterative series of user studies and tool refinement, a concept testing was carried out. Concept testing is a communication process in an early stage of design between designers and potential users to ensure that users accept the principle design ideas (Ulrich and Eppinger, 2000; Kankainen, 2003). Simple prototypes can be used to illustrate the ideas to users in this stage. Erickson (1995) reminds us that rough
Towards Multi-variate Visualization of Metadata Describing Geographic Information
621
prototypes attract more comments on the idea than a final-looking prototype. In this testing, the design ideas of the visualization environment for metadata were communicated to users through a prototype that combined static displays and a use scenario. The displays showed various views of metadata with the four multi-variate visualization methods. The use scenario, similar in nature to the one provided above, demonstrated the process and the potential of interaction. Users were asked to think aloud when they studied the displays. In addition, a semi-structured interview was made to collect feedback. Twelve users of geographic data from the Finnish Defence Forces participated in the testing. The feedback from the users indicates that the parallel coordinates plot and star plots are most appreciated by users whereas the scatter plot matrix and Chernoff faces caused most problems (Ahonen-Rainio, 2003). Users judged the scatter plot matrix difficult to interpret and indicated that it does not support a holistic view of datasets. Chernoff faces were judged to arouse emotions that may disturb the interpretation of metadata; interpretation of extreme values from the faces is reliable, but intermediate values are easily confused. Users preferred star plots for gaining an initial impression of the datasets and the parallel coordinates plot for studying selected datasets in detail. Further studies are needed to resolve how many metadata variables users can deal with, how the ordering and scaling of values can be accessed and which interaction techniques with multi-variate visualization methods are the best according to usability and utility criteria. For example, the number of variables under exploration at a time must be limited because of the requirement of easiness of use. Spence (2001) proposed that a scatter plot matrix with more than about five variables would be unwieldy, and Ware (2000) recommends using in the region of three orientations for a rapid classification of star plots. The further experiments should consider the likelihood that solutions will be dependent upon different types of metadata users.
32.6
Discussion
We have proposed a visual approach for providing metadata to potential users of geographic datasets with an exploratory emphasis, with the aim of offering users an efficient way to select the dataset that meets their needs. Visualization of metadata using interactive multi-variate techniques can enable users to explore metadata in a structured manner rather than just sifting through static presentations. Exploration can allow comparison of datasets according to multiple variables and encourage users to seek insight into geographic data that could not be gained by the current textual, dataset-wise presentation of metadata. But effective approaches to handling some of the complexities and characteristics of metadata, such as multiple and missing values, requires further consideration before the visualization tools can be refined. The ISO 19115:2003 standard allows repeated values of metadata. These result in multiple values of an individual variable for a dataset and the values have to be shown in the visualization. If metadata are summarized for a whole dataset, which is a typical case, the internal variation of geographic datasets results in uncertainty of the metadata proper. If this uncertainty is known, for example a range of possible values can be derived
622
P. Ahonen-Rainio & M.-J. Kraak
from the metadata, it should also be represented or at least be accessible in the visualization. In practice, geographic metadata are incompletely available and thus the missing values provide a challenge for visualization. Depending on the purpose of visualization the missing values may be indicated as such or substituted by plausible values. At an overview, level substitution of metadata by predicted values may be acceptable. However, when evaluating datasets in detail it is important for users to understand clearly that the value is unknown. Transformation of metadata values to a scale that reflects user preferences can support the exploration. The transformation can be an ordinary normalization (e.g., linearly or logarithmically) from the smallest metadata values to the largest ones or vice versa. For example, a user may prefer the object density more the higher it is or prefer price more the lower it is. But the transformation can also be more complex. If the user prefers medium scale data the transformation should separate the smaller and larger scale fractions from the preferred scale range. In any case, the original values must be accessible in the visualization. Although we expect that the transformation to preference values support users in the exploration of metadata, at the same time, we have to ensure that the visualization environment does not become too complex for the users of metadata. Therefore, it is vital to understand users' conception of metadata and their tasks with metadata. We have considered a single task that relies upon metadata: that of finding a dataset fit for an intended purpose. Other types of metadata use, with different aims may offer a wider perspective on metadata. In the case presented here, however, the visualization techniques offer the opportunity to provide insight that is hard to achieve using currently available means. This kind of exploratory use can give new value to the increasing collections of metadata and might encourage data owners to supply metadata more frequently and consistently.
Acknowledgements This study relates to a research project on metadata of geographic datasets financed by the Topographic Service of the Finnish Defence Forces. The first author gratefully acknowledges their support.
References Ahonen-Rainio, P., (2002) "Contextual approach to metadata of geographic information", Finnish Journal of Surveying Sciences, 20(1-2), 64-79. Ahonen-Rainio, P., (2003) "Concept testing of some visualization methods for geographic metadata", Proceedings Ninth Scandinavian Research Conference on Geographical Information Sciences, Espoo, pp. 103-117. Albertoni, R., Bertone, A., Dem~ar, U., De Martino, M., and Hauska, H., (2003) "Knowledge extraction by visual data mining of metadata in site planning",
Towards Multi-variate Visualization of Metadata Describing Geographic Information 623 Proceedings Ninth Scandinavian Research Conference on Geographical Information Sciences, Espoo, pp. 155-166. Andrienko, G., and Andrienko, N., (2001) "Exploring spatial data with dominant attribute map and parallel coordinates", Computers, Environment and Urban Systems, 25, 5-15. Beyer, H., and Holtzblatt, K., (1998) Contextual Design - Defining Customer-Centered Systems. San Francisco: Morgan Kaufmann Publishers, p. 472. Bucher, B., (2003) "Translating user needs for geographic information into metadata queries", Proceedings Sixth AGILE Conference on Geographic Information Science, Lyon, pp. 567-576. Buja, A., McDonald, J. A., Michalak, J., and Stuetzle, W., (1991) "Interactive data visualization using focusing and linking", Proceedings Visualization '91, IEEE Conference on Visualization, San Diego, CA, pp. 156-163. Card, S. K., Mackinlay, J. D., and Shneiderman, B., (eds.), (1999) Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann Publishers. Cartwright, W., Crampton, J., Gartner, G., Miller, S., Mitchell, K., Siekierska, E., and Wood, J., (2001) "Geospatial information visualization user interface issues", Cartography and Geographic Information Science, 28(1), 45-60. Chambers, J. M., Cleveland, W. S., Kleiner, B., and Tukey, P. A., (1983) Graphical Methods for Data Analysis. Belmont (CA): Wadsworth Press. Chernoff, H., (1973) "The use of faces to represent points in k-dimensional space graphically", Journal of the American Statistical Association, 68(342), 361-368. Danish National Survey and Cadastre (2003) Geodata-info.dk - a public catalog for digital maps and spatial information in Denmark. Online: http://www.geodata-info.dk/(23/11/2003). De Soete, G., (1986) "A perceptual study of the Flury-Riedwyl faces for graphically displaying multivariate data", International Journal of Man-Machine Studies, 25(5), 549-555. Dorling, D., (1995) A New Social Atlas of Britain. Chicester, UK: Wiley. Drecki, I., (2002) "Visualisation of uncertainty in geographical data", In: Shi, W., Fisher, P. F., and Goodchild, M. F., (eds.), Spatial Data Quality. London: Taylor & Francis, pp. 140-159. Dykes, J. A., (1995) "Cartographic visualization for spatial analysis", Proceedings 17th International Cartographic Conference, Barcelona, pp. 1365-1370. Dykes, J. A., (1997) "Exploring spatial data representation with dynamic graphics", Computers and Geosciences, 23(4), 345-370, Online: http ://www.mimas. ac.uk/argus/ICA/J.Dykes/ Edsall, R. M., MacEachren, A. M., and Pickle, L. J., (2001) "Case study: design and assessment of an enhanced geographic information system for exploration of multivariate health statistics", In: Andrews, K., Roth, S., and Wong, P. C., (eds.), Proceedings of the IEEE Symposium on Information Visualization, pp. 159-162.
624
P. Ahonen-Rainio & M.-J. Kraak
Erickson, T., (1995) "Notes on design practice: stories and prototypes as catalysts for communication", In: Carroll, J. M., (ed.), Scenario-based design, Envisioning Work and Technology in System Development. New York: Wiley, pp. 37-58. Fisher, P., (1994) "Hearing the reliability in classified remotely sensed images", Cartography and Geographic Information Systems, 21(1), 31-36. Fonseca, F., Egenhofer, M., Agouris, P., and C~mara, G., (2002) "Using ontologies for integrated geographic information systems", Transactions in GIS, 6(3), 231-257. Goodchild, M., Buttenfield, B., and Wood, J., (1994) "Introduction to visualizing data validity", In: Hearnshaw, H. M., and Unwin, D. J., (eds.), Visualization in Geographical Information Systems, pp. 141-149. Green, D., and Bossomaier, T., (2002) Online GIS and Spatial Metadata. London: Taylor & Francis. Groot, D., and McLaughlin, J., (eds.), (2000) Geospatial Data Infrastructure - Concepts, Cases, and Good Practice. Oxford: Oxford University Press. Guptill, S. C., (1999) "Metadata and data catalogues", In: Longley, P., Goodchild, M. F., Maguire, D. J., and Rhind, D., (eds.), Geographical Information Systems, Volume 2, Management Issues and Applications. Cambridge: Wiley, pp. 677-692. Haslett, J., Wills, G. J., and Unwin, A. R., (1990) "SPIDER: An interactive statistical tool for the analysis of spatially distributed data", International Journal of Geographical Information Systems, 4(3), 285-296. Hootsmans, R. M., and Wel, F. J. M. v. d., (1993) "Detection and visualization of ambiguity and fuzziness in composite spatial datasets", Proceedings of the Fourth European Conference of Geographical Information Systems, Genoa, pp. 1035-1046. Inselberg, A., (1985) "The plane with parallel coordinates", The Visual Computer, 1, 69-91. International Standards Organisation (2003) ISO 19115(E) Geographic Information Metadata, International Organization for Standardization. Kankainen, A., (2003) "UCPCD - User-Centered Product Concept Design", Proceedings of the 2003 conference of Designing for User Experiences (DUZ 2003). New York, NY: ACM Press. Online: http://www.doi.acm.org/10.1145/997078.997087 Keim, D. A., (2001) "Visual exploration of large data sets", Communications of the A CM (CA CM), 44(8), 38-44. Kraak, M. J., and Ormeling, F. J., (2002) Cartography, Visualization of Geospatial Data. London: Prentice Hall. Lillywhite, J., (1991) "Identifying available spatial metadata: the problem", In: Medyckyj-Scott, D., Newman, I., Ruggles, C., and Walker, D., (eds.), Metadata in Geosciences. Leicester: Group D Publications, pp. 3-11. MADAME (2000) Comparative Evaluation of On-line Metadata Services and User Feedback. Online: http://www.shef.ac.uk/(scgisa/MADAMENew/Deliverables/d2.pdf (28/05/03). MacDougal, E. B., (1992) "Exploratory analysis, dynamic statistical visualization and geographical information systems", Cartography and Geographical Information Systems, 19(4), 237-246.
Towards Multi-variate Visualization of Metadata Describing Geographic Information 625 MacEachren, A. M., (1992) "Visualizing uncertain information", Cartographic Perspectives, (13), 10-19. MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Science, 28(1), 3-12. MacEachren, A. M., Wachowicz, M., Edsall, R., Haug, D., and Masters, R., (1999) "Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods", International Journal of Geographical Information Science, 13(4), 311-334. McGranaghan, M., (1993) "A cartographic view of spatial data quality", Cartographica, 30, 10-19. Morris, C. J., Ebert, D. S., and Rheingans, P., (1999) "An experimental analysis of the effectiveness of features in Chernoff faces", Applied Imagery Pattern Recognition '99: 3D Visualization for Data Exploration and Decision Making. National Land Survey of Finland (2003) Paikkatietohakemisto - a metadata service for geographic datasets in Finland. Online: http ://www.maanmittauslaitos.fi/paikkatiedot/default.asp ?id=449 (23/10/03 ). NCGI (2003) NCGI Metadata Service. Online: http://www.ncgi.nl/ncgi/(23/10/03). Nielsen, J., (1995) "Scenarios in Discount Usability Engineering", In: Carroll, J. M., (ed.), Scenario-Based Design. Envisioning Work and Technology in System Development. London, John Wiley and Sons, pp. 59-83. Schlieder, C., and V6gele, T., (2002) "Indexing and browsing digital maps with intelligent thumbnails", Proceedings of Joint International Symposium on Geospatial Theory, Processing and Applications, Ottawa. Shneiderman, B., (1997) Designing the User Interface - Strategies for Effective HumanComputer Interaction. Reading, MA: Addison-Wesley, p. 639. Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., and Hedley, N. R., (2001) "Cognitive and usability issues in geovisualization", Cartography and Geographic Information Science, 28(1), 61-75. Spence, R., (2001) Information Visualization. Harlow, London: Addison Wesley/ACM Press Books, 206 pp. Ulrich, K. T., and Eppinger, S. D., (2000) Product Design and Development. Boston: McGraw-Hill. Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, p. 384. Wel, F. J. M. v. d., Hootsmans, R. M., and Ormeling, F. J., (1994) "Visualization of Data Quality", In: MacEachren, A. and Taylor, D. R. F., (eds.), Visualization in Modern Cartography. Pergamon, Oxford, pp. 313-331.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Published by Elsevier Ltd. All rights reserved.
Chapter 33
Evaluating Self-organizing Maps for Geovisualization Etien L. Koua & Menno-Jan Kraak, International Institute for Geo-Information Science and Earth Observation (ITC), P.O. Box 6, 7500 AA Enschede, The Netherlands
Keywords: representation, usability, self-organizing map, information space, visualization
Abstract Information spaces are increasingly used to visualize complex data. With geospatial data specifically, there is a growing need for more flexible, accurate and usable information spaces to support exploratory analysis. Here, we present a prototype exploratory geovisualization environment based on a usability framework. The environment implements a self-organizing map neural network algorithm, and relates spatial analysis, datamining and knowledge discovery methods for the extraction of patterns. Some graphical representations are then used to portray extracted patterns in a visual form that contribute to improved understanding of the derived structures and the geographical processes. As part of the design and development process, a usability evaluation plan is proposed to assess ways in which users interpret these graphical representations and their appropriateness for exploratory visual analysis. The ultimate goal of the proposed evaluation strategy is to improve the visual design of the representations.
33.1
Introduction
Designing an effective visualization environment for analyzing large geospatial datasets has become one of the major concerns in the geovisualization community, as volumes of data become larger and data structures more complex. In these large and rich databases, uncovering and understanding patterns or processes presents a difficult challenge as they easily overwhelm mainstream geospatial analysis techniques that are oriented towards the extraction of information from small and homogeneous datasets (Gahegan et al., 2001; Miller and Han, 2001). A number of authors have proposed using Artificial Neural Networks as part of a strategy to improve geospatial analysis of large, complex datasets (Schaale and 627
628
E.L. Koua & M.-J. Kraak
Furrer, 1995; Openshaw and Turton, 1996; Skidmore et al., 1997; Gahegan and Takatsuka, 1999; Gahegan, 2000). Artificial Neural Networks have the ability to perform pattern recognition and classification. They are especially useful in situations where the data volumes are large and the relationships are unclear or even hidden. This is because of their ability to handle noisy data in difficult non-ideal contexts (Openshaw and Openshaw, 1997). Particular attention has been directed to using the self-organizing map (SOM) neural network model as a means of organizing complex information spaces (Chen, 1999; Girardin, 1995; Fabrikant and Buttenfield, 2001). The SOM is also generally acknowledged as being a useful tool for the extraction of patterns and the creation of abstractions where conventional methods may be limited because underlying relationships are not clear or classes of interest are not obvious. SOMs (and other artificial neural net methods) can be used to extract features in complex data. To interpret these (often abstract) features, appropriate visualization techniques are needed to represent extracted information in a way that can contribute to improved understanding of underlying structures and processes. The goal is to represent the data in a visual form in order to stimulate pattern recognition and hypothesis generation. The use of information spaces can play a role by offering visual representations of data that bring the properties of human perception to bear (Card et al., 1999). Spatial metaphors such as distances, regions and scale are used to facilitate the representation and understanding of information in such spaces (Fabrikant et al., 2002). An important step in the design of effective visualization tools will rely on understanding the way users interpret and build a mental model of these information spaces. SOMs have been successfully implemented and used for geographic data classification (Gahegan and Takatsuka, 1999; Gahegan, 2000). However, the relative effectiveness of integration of SOMs with visualization methods for exploration and knowledge discovery in complex geospatial datasets remains to be explored. In particular, we believe that the visual design of SOM graphical representations will significantly affect how successful they are for exploratory analysis purposes. This chapter considers the potential of using SOM in an integrated visual-computational environment, presents four alternative visual renderings of SOM that can be used to highlight different characteristics of the computational solution it produces, and proposes an evaluation strategy for assessing their relative effectiveness according to three common visualization tasks: (i) identifying clusters; (ii) relating distances (similarity); (iii) relating values. The chapter concludes with a discussion of some subsequent research requirements.
33.2
Context: Visualization and Knowledge Discovery in Large Geospatial Datasets
The SOM-based visualization environment presented here was developed to contribute to the analysis and visualization of large amounts of data, as an extension of the many geospatial analysis functions available in most GIS software. The objective of the tool is to help uncover structure and patterns that may be hidden in complex geospatial
Evaluating Self-organizing Maps for Geovisualization
629
datasets, and to provide graphical representations that can support understanding and knowledge construction. The framework includes spatial analysis, datamining and knowledge discovery methods, integrated into an interactive visualization system. Users can perform a number of exploratory tasks not only to understand the structure of the dataset as a whole but also to explore detailed information on individual or selected attributes of the dataset. In order to examine how users understand these representations, and to improve the overall effectiveness of the design, a usability assessment plan is proposed to evaluate the graphical representation forms accessible through the tool as well as visual variables used to depict data within each form of representation.
33.2.1
The self-organizing map
The SOM (Kohonen, 1995) is an artificial neural network used to map multidimensional data onto a space of lower dimensionality - usually a 2D representation space. The network consists of a number of neural processing elements (neurons or nodes) usually arranged on a rectangular or hexagonal grid, where each neuron is connected to the input. The goal is to group nodes close together in certain areas of the data value range. The resultant maps (SOMs) are organized in such a way that similar data are mapped onto the same node or to neighboring nodes in the map. Either a rectangular or hexagonal layout is typically used (Figure 33.1). This leads to a spatial clustering of similar input patterns in neighboring parts of the SOM and the clusters that appear on the map are themselves organized internally. This arrangement of the clusters in the map reflects the attribute relationships of the clusters in the input space. For example, the size of the clusters (the number of nodes allotted to each cluster) is reflective of the frequency distribution of the patterns in the input set. The SOM uses a distribution preserving property which has the ability to allocate more nodes to input patterns that appear more frequently during the training phase of the network configuration. It also applies a topology preserving property, which comes from the fact that similar data are mapped onto the same node, or to neighboring nodes in the map.
Figure 33.1. Selection of a node and adaptation of neighboring nodes to input data on (a) a hexagonal SOM grid; (b) a rectangular SOM grid. The black object indicates the node that was selected as the best match for the input pattern. Neighboring nodes adapt themselves according to the similarity with the input pattern. The degree of shading of neighboring nodes corresponds to the strength of the adaptation.
630
E.L. Koua & M.-J. Kraak
In other words, the topology of the dataset in its n-dimensional space is captured by the SOM and reflected in the ordering of its nodes. This is an important feature of the SOM that allows the data to be projected onto the lower dimension space while preserving, as much as possible, the order of the data in its original space. Another important feature of the SOM for knowledge discovery in complex datasets is the fact that it is basically an unsupervised learning network meaning that the training patterns have no category information that accompany them. Unlike a supervised method which learns to associate a set of inputs with a set of outputs using a training data set for which both input and output are known, SOM adopts a learning strategy where the similarity relationships between the data and the clusters are used to classify and categorize the data.
33.2.2
Computational analysis and visualization framework
The use of the SOM intends to contribute to exploratory data analysis techniques for complex geospatial data, and combines clustering and projection techniques for feature extraction, visualization and interpretation of large multi-dimensional datasets. For the user, the main goal is the acquisition of knowledge through discovery, for decisionmaking, problem solving and explanation. The first level of the computation provides a mechanism for extracting patterns from the data. Resultant maps (SOMs) are then visualized using graphical representations. We use different visualization techniques to enhance data exploration including brushing, multiple views and 3D views. Projection methods such as Sammon's mapping (Sammon, 1969) and Principal Component Analysis are also used to depict the output from the SOM. Spatial metaphors are used to guide user exploration and interpretation of the resulting non-geographic representation; this is an example of spatialization, an approach discussed more generally by Fabrikant (2001) and Fabrikant and Skupin, this volume (Chapter 35). These metaphors are combined with alternative 2D and 3D forms of representation and user interaction in the information spaces. The resulting information spaces suggest and take advantage of natural environment metaphor characteristics such as "near = similar, far = different" (MacEachren et al., 1999), which is epitomized by Tobler' s first law of Geography (Tobler, 1970). Various types of map representations are used, including volumes, surfaces, points and lines. This allows exploration of multiple kinds of relationships between items. A coordinate system allows the user to determine distance and direction, from which other spatial relationships (size, shape, density, arrangement, etc.) may be derived. Multiple levels of detail allow exploration at various scales, creating the potential for hierarchical grouping of items, regionalization and other types of generalizations.
33.2.3
Usability framework for the design of the visualization environment
The design of the visualization environment is based on a usability framework structured to develop a tool that is useful and appropriate for the user needs and tasks (w This framework not only includes the techniques, processes, methods and procedures for
Evaluating Self-organizing Maps for Geovisualization
631
designing usable products and systems but it also focuses on the user's goals, needs and tasks in the design process (Rubin, 1994). User characteristics, visualization tasks and operations are examined to improve user interaction and to support activities involved in the use of the visualization environment, and in related information spaces. Figure 33.2 shows the underlying design concept and usability framework. This framework is informed by current understanding of effective application of visual variables for cartographic and information design, developing theories of interface metaphors for geospatial information displays, and previous empirical studies of map and Information Visualization effectiveness. The framework guided initial design decisions presented here and will be used to structure subsequent user studies (the strategy for which is introduced in w
Figure 33.2. Usability framework for the design of the SOM-based visualization environment. Stage (1) describes the general geospatial data handling process; (2) represents the proposed computational analysis and visualization method based on the SOM algorithm for complex geospatial data; (3) is the design framework for the human-computer interface for the visualization of the SOMs and includes the representations to evaluate; (4) shows the usability measures used to test the outcome of interaction and use of the visualization tool.
E.L. Koua & M.-J. Kraak
632
33.3
The SOM Graphical Representations
The design of the visualization environment incorporates several graphical representations of SOM output. These include a distance matrix representation, 2D and 3D projections, 2D and 3D surfaces, and component planes visualization (in a multiple view). These representation forms are introduced briefly using a dataset described below.
33.3.1
Map stimuli
The dataset explored by the representations is a collection of socio-economic indicators related to municipalities in a region in the Netherlands. It consists of 29 variables including population and habitat distributions, urbanization indicators, income of inhabitants, family and land data, as well as industrial, commercial and non-commercial services data. The idea is to find multi-variate patterns and relationships among the municipalities. This dataset was selected for the study because of the fact that it is a known dataset in which we can test different hypotheses about both the geographic patterns and the representation/analysis methods investigated (for a discussion of technique evaluation through known data sets see Noy this volume (Chapter 12). The maps assist in the understanding of SOM representations. Unusual SOM patterns can be verified with reality. At the end, the use of SOM is aimed at far larger datasets than the one used in this experiment. Three attributes of the dataset (family size, income per inhabitant and average value of dwellings) as well as a reference map with the names of the municipalities are presented in Figure 33.3. The maps show, for example, that there are four municipalities where the average family size is four, compared to the rest of the region where the average family size is three. With the SOM, such relationships can be easily examined in a single visual representation using the component planes. Component planes show the values of the map elements for different attributes. They show how each input vector varies over the space of the SOM units. Unlike standard choropleth maps, the position of the map units (which is the same for all displays) is determined during the training of the network, according to the characteristics of the data samples. A cell or hexagon here can represent one or several political units (municipalities) according to the similarity in the data. Two variables that are correlated will be represented by similar displays. In the example described above, the SOM shows that there is a cluster of municipalities that have a family size of more than three (Figure 33.4a). It also shows the relationships between the municipalities for the different attributes. The two other attributes (income per inhabitant and average value of dwellings) are presented as component planes extracted from the SOM (Figure 33.4b and c) for exploratory analysis purposes. By relating component displays we can explore the dataset, interpret patterns as indications of structure and examine relationships that exist. For example, Figure 33.4b and c indicates that the highest dwelling values correspond to municipalities (StadDelden, Bathmen and Diepenheim), where the average income per inhabitant is the highest. The representations of the SOM make it possible to easily find correlations in a large volume of multi-variate data. New knowledge can be unearthed through this process of
Evaluating Self-organizing Maps for Geovisualization
633
Figure 33.3. Example of attributes of the test dataset: (a) average family size; (b) average income per inhabitant in the municipalities; (c) average value of dwellings; the names of municipalities are shown in (d).
Figure 33.4. SOM component planes depicting a univariate space for selected attributes of the dataset: (a) the average family size; (b) the income per inhabitant; (c) the average value of dwellings; the labels corresponding to the position of the map units (municipalities) are shown in (d).
634
E.L. Koua & M.-J. Kraak
exploration, followed by identification of associations between attributes using the various representations, and finally the formulation and ultimate testing of hypotheses.
33.3.2
Unified distance matrix representation
The unified distance matrix or U-matrix (Ultsch and Siemon, 1990) is a representation of the SOM that visualizes the distances between the network neurons or units. It contains the distances from each unit center to all of its neighbors. The neurons of the SOM network are represented here by hexagonal cells (Figure 33.5). The distance between the adjacent neurons is calculated and presented with different colorings. A dark coloring between the neurons corresponds to a large distance and thus represents a gap between the values in the input space. A light coloring between the neurons signifies that the vectors are close to each other in the input space. Light areas represent clusters and dark areas cluster separators. This representation can be used to visualize the structure of the input space and to get an impression of otherwise invisible structures in a multidimensional data space. The U-matrix representation (Figure 33.5) reveals the clustering structure of the dataset used in this experiment. Municipalities having similar characteristics are arranged close to each other and the distance between them represents the degree of similarity or dissimilarity. For example, the municipality of Enschede is well separated from the rest by the dark cells showing a long distance from the rest of the municipalities. This is expected, since Enschede is the largest, the most developed and urbanized municipality in the region. On the top left comer of the map, municipalities Genemuiden, Rijssen, Staphorst, Ijsselmuiden are clustered together. These are small localities that have common characteristics according to the data. This kind of similarity can be composed of a number of variables provided by the dataset. The U-matrix shows more hexagons than the component planes (discussed below) because it shows not only the values at map units but also the distances between map units. In contrast to other projection methods in general, the SOM does not try to preserve the distances directly, but rather the relations or local structure of the input data. While the U-matrix is a good method for visualizing clusters, it does not provide a very clear picture of the overall shape of the data space because the visualization is tied to the SOM grid. Alternative representations to the U-matrix can be used to visualize the shape of the SOM in the original input data space. Three are introduced and discussed below: 2D and 3D projections (using projection methods such as the Sammon's mapping and PCA), 2D and 3D surface plots, and component planes.
33.3.3
2D and 3D projections
The projection of the SOM offers a view of the clustering of the data with data items depicted as colored nodes (Figure 33.6). Similar data items (municipalities in this dataset) are grouped together with the same type or color of markers. Size, position and color of markers can be used to depict the relationships between the
Evaluating Self-organizing Maps for Geovisualization
635
Figure 33.5. The unified distance matrix showing clustering and distances between positions on the map. Municipalities having similar characteristics are arranged close to each other and the distance between them represents the degree of similarity or dissimilarity. Light areas represent clusters and dark areas cluster separators (a gap between the values in the input space).
data items. This gives an informative picture of the global shape and the overall smoothness of the SOM in 2D or 3D space. In 3D space, the weight of the data items according to the multi-variate attributes can be represented using the third dimension to show a hierarchical order or tree structure. Exploration can be enhanced by rotation, zooming and selection in the 3D representation and by interactive manipulation of features such as color, size and type of the markers. Connecting these markers with lines can reveal the shape of clusters and relationships among them.
636
E.L. Koua & M.-J. Kraak
Figure 33.6. Projection of the SOM results in (a) 2D space and (b) 3D space. Municipalities having similar characteristics according to the multi-variate attributes in the dataset are represented using points (markers) with color coding and connecting lines to depict relationships between them.
33.3.4
2D and 3D surface plots
The 2D surface plot of the distance matrix (Figure 33.7a) uses color value to indicate the average distance to neighboring map units. It is a spatialization (Fabrikant and Skupin, this volume (Chapter 35)) that uses a landscape metaphor to represent the
Evaluating Self-organizing Maps for Geovisualization
637
Figure 33.7. Surface plots: the density, shape and size or volumes of clusters are shown in 2D surface (a) and 3D surfaces (b) to depict a multi-variate space. Darker color indicates greater distance and light color small distance.
638
E.L. Koua & M.-J. Kraak
density, shape and size or volume of clusters. Unlike the projection in Figure 33.4 that shows only the position and clustering of map units, areas with uniform color are used in the surface plots to show the clustering structure and relationships among map units. In the 3D surface (Figure 33.7b), color value and height are used to represent the regionalization of map units according to the multi-dimensional attributes.
33.3.5
Component planes
Component planes represent a multi-variate visualization of the attributes of the dataset, allowing easy detection of relationships among the attributes as described above (Figure 33.8). Each component plane shows the values of one variable for all map units using color-coding that follows color scheme guidelines presented by Brewer (1994). This gives the possibility to visually examine every cell corresponding to each map unit or data item. By using the position and coloring, all relationships between different map units (municipalities in this dataset) can be easily explored in a single visual representation. For example, the average income per inhabitant is somewhat correlated with the number of inhabitants between the age of 45 and 64 years (INH_45-64Y) and the number of inhabitants older than 64 years (INH_65_p) in municipalities such as StadDelden, Bathmen, Diepenheim, Ootmarsum, Holten, Markelo (see Figure 33.4d for corresponding names of municipalities). These municipalities have the highest income of the region. These displays can be arranged in any order (alphabetical, geographical pattern, or any order that makes it easy to see relationships among them) in a way similar to the collection maps of Bertin (1981).
33.4
Usability Evaluation Plan
We now present a general strategy for assessing visual-computational environments for exploring multi-variate geospatial data. The strategy is developed for examining the effectiveness of the alternative representation forms such as the SOM-based visualization environment presented above and visual variable choices within those forms. A usercentered approach to developing integrated computational-viual analysis tools (whether based on SOM or other inductive learning methods) should include attention to the user' s understanding of the representation forms of multi-variate relationships in the data. Details are provided about specific components of a prototypical empirical study plan including: overview, methods and test instruments, selection of participants and experiment procedures.
33.4.1
Overview of the evaluation plan
The strategy developed to assess the visual-computational environments for exploring multi-variate geospatial data focuses on gathering information about how users interpret and understand the basic visualization features and representation forms, in order to improve their design. Specifically, we are interested to know whether users can actually comprehend the meaning of the proposed representations, how the different representation forms influence the effectiveness of the visualization tool in terms of analysis and exploration of data, and what type of representation is suitable for
I_...
Y
1-11
lHH-15-21Y
'
u,.
u"-'
13.6
47.2
'
C...,,
.
* INCOME
3som
HGHJNCOHE
MOP
592
Figure 33.8. SOM component planes depicting the different attributes of the dataset and the relationships among them for all the municipalities. Relationshipsbetween different municipalities in the dataset are explored in a single visual representation.
640
E.L. Koua & M.-J. Kraak
exploratory tasks. To this end, the general evaluation strategy is to conduct a controlled experiment through which different options for map-based visualization of the output of the SOM multi-variate analysis are compared.
33.4.2
Methods and test instruments
The proposed usability assessment plan focuses on examining different features implemented in the visual-computational analysis environment. The assessment plan includes several kinds of evidence, such as responses to specific questions focused on use of features and representations to perform specific tasks as well as follow up interviews to derive insight about strategies used to apply tools to tasks and reasons that specific tools performed well or poorly for specific tasks. Information about participants, including their experiences in the field of GIScience and their educational background can be important to relate users' responses to their previous knowledge in the field. Test rating scale and interviews were used to report the level of difficulty, the ease in use, and preferences (user subjective views) related to the use of specific features of the visualization environment.
33.4.3
Selection of test participants
Participants for such assessment should be selected to represent the target population of people who are the likely users. They should be domain specialists who have knowledge about the data and have both the motivation and qualifications to undertake appropriate interpretation and analysis of the data. In this case, the user group might include geographers, demographers, environmental scientists, epidemiologists, and others.
33.4.4
Experiment procedures
The proposed assessment plan focuses on an empirical, controlled experiment. An overview of experiment procedures including test measures, experiment variables and tasks is described below.
Test measures We propose a general approach useful for empirical assessment of exploratory visualcomputational tools. The proposed assessment plan includes three criteria: effectiveness, usefulness and user reactions. 9
9
9
Effectiveness focuses on tool usability and examines whether the users understand what the tool is for, how to interpret it, how to manipulate any parameters or controls available. Usefulness refers to the appropriateness of the tool's functionality and assesses whether the tool meets the needs and requirements of the users when carrying tasks. User reactions refer to user's subjective views and preferences about the representation features and metaphors used, the perceived effect of visual variables in the spatial arrangement including clusters, similarity, distance, relationships, and other map object characteristics.
Evaluating Self-organizing Maps for Geovisualization
641
Suggested experiment variables and subject's tasks A number of common visualization tasks associated with using visual-computational environment for data exploration can be used in the assessment. The idea is to allow users to visually examine the representations, respond to questions, and report their preferences and viewpoints about the representation forms and the effect of visual variables used. The objective is to measure and compare task performance and the user's level of understanding with the different representation forms and tasks. Some common tasks used for data exploration in visual-computational environments and related to the three test measures discussed above are identified to be included in the assessment. Tasks related to effectiveness can be measured by the success (and accuracy) in completing specific analysis tasks. Specific measures used to assess effectiveness include the correctness of response and time required to make a response (Sweeney, 1993; Fabrikant, 2001; Rubin, 1994). This can include various activities such as the identification of categories and characterization of relationships between data items (within categories and between different categories), comparison of values at different positions, comparison of distances between a set of points in each representation. Possible variables that may have an impact on effectiveness of tool for specific tasks include colors, proximity, shape and size of objects used to depict nodes or areas in resulting information spaces. Tasks related to usefulness can include comparison of the representation forms in relation to identified analysis tasks. This can include the identification of variables that are key in analysis and visualization tasks such as the judgment of similarity and dissimilarity. User reactions can be gathered through interviews, to provide information on flexibility, compatibility (between the way the tool looks and works and the user's convention and expectations), and the appropriateness of the tool features for specified tasks. This can also include a rating of the perceived effect of visual variables and different display formats such as 2D and 3D views used in the representation forms.
33.5
Conclusion and Discussion
The development of the tool presented in this chapter was based on an approach to combine visual and computational analysis into a visualization environment intended to contribute to the analysis of large volumes of geospatial data. This approach focuses on the effective application of computational algorithms to extract patterns and relationships in geospatial data, the visual representation of derived information, which involves effective use of visual variables used in such complex information spaces to facilitate knowledge construction. As a general strategy to assess ways in which users understand the representations, and to improve the overall effectiveness of the design of exploratory visual-computational environments in general, we have proposed an assessment plan.
642
E.L. Koua & M.-J. Kraak
The plan is based on a controlled experiment involving a representation of intended users and a number of selected tasks performed in visualization environments. This assessment plan will be applied in the specific case of the designed of the proposed visualization environment, to assess design concepts and aspects of the implementation of the computational visual analysis environment. This includes assessment of the appropriateness of representation metaphors applied as well as of visual variables used in the design of specific representations. The assessment can provide some insight into the effectiveness, the usefulness and user reactions (user's preferences and views) for the representations for exploratory visual analysis, interpretation and understanding of the structure in the dataset. We propose that such a study of the comprehension of the graphical representations accessible through the visualization environment, their ability to support specific tasks as well as the extent to which they support particular user goals, needs to be conducted as a controlled experiment. This can be done by attempting a number of basic visualization tasks. The next steps in the development of the tools discussed in this chapter will focus on the extension and improvement of the graphical representations presented, following the usability framework described in w (Figure 33.2). One part of this work will be to integrate the representation approaches into a multiple views approach to visualization. By combining user interactions with these forms of representation, the visualization environment will be extended and improved to focus on interactive manipulation of the representations to support the cognitive activities involved in the use of the visualization environment, and to provide querying and exploration of features in a user-friendly interface. Such advances are likely to have additional impacts upon the user's preferences and responses. These aspects will be the focus of subsequent usability testing, which will include user interaction. A goal will be to characterize the overall effectiveness of the representations and of the visualization environment when applied to complex geo-phenomena such as spatio-temporal data.
References Bertin, J., (1981) Graphics and Graphic Information Processing. Berlin: Walter de Gruyter, p. 273. Brewer, C. A., (1994) "Color use guidelines for mapping and visualization", In: MacEachren, A. M., and Taylor, D. R. F., (eds.), Visualization in Modern Cartography, Vol. 2. Oxford: Elsevier Science Ltd., pp. 123-148. Card, S. K., Mackinlay, J. D., and Shneiderman, B., (eds.), (1999) Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann Publishers. Chen, C., (1999) Information Visualization and Virtual Environments. London: Springer, p. 223. Fabrikant, S. I., (2001) "Evaluating the usability of the scale metaphor for querying semantic information spaces", In: Montello, D. R., (ed.), Spatial Information Theory: Foundations of Geographic Information Science. Berlin: Springer, pp. 156-171.
Evaluating Self-organizing Maps for Geovisualization
643
Fabrikant, S. I., and Buttenfield, B. P., (2001) "Formalizing semantic spaces for information access", Annals of the Association of American Geographers, 91(2), 263-290. Fabrikant, S. I., Ruocco, M., Middleton, R., Montello, D. R., and J6rgensen, C., (2002) "The first law of cognitive geography: distance and similarity in semantic space", Proceedings, GIScience 2002, Boulder, CO, pp. 31-33. Gahegan, M., (2000) "On the application of inductive machine learning tools to geographical analysis", Geographical Analysis, 32(2), 113-139. Gahegan, M., and Takatsuka, M., (1999) "Dataspaces as an organizational concept for the neural classification of geographic datasets", Fourth International Conference on GeoComputation, Fredericksburg, Virginia, USA, pp. 25-28, (July 1999). Gahegan, M., Harrower, M., Rhyne, T.-M., and Wachowicz, M., (2001) "The integration of geographic visualization with databases, data mining, knowledge construction and geocomputation", Cartography and Geographic Information Science, 28(1), 29-44. Girardin, L., (1995) "Mapping the virtual geography of the World Wide Web", Fifth International World Wide Web Conference, Paris, France, (6-10 May 1999). Kohonen, T., (1995) Self-Organizing Maps. Berlin: Springer-Verlag. MacEachren, A. M., Wachowicz, M., Edsall, R., Haug, D., and Masters, R., (1999) "Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods", International Journal of Geographical Information Science, 13(4), 311-334. Miller, H. J., and Han, J., (2001) Geographic Data Mining and Knowledge Discovery. London: Taylor and Francis. Openshaw, S., and Openshaw, C., (1997) Artificial Intelligence in Geography. Chichester: Wiley. Openshaw, S., and Turton, I., (1996) "A parallel Kohonen algorithm for the classification of large spatial datasets", Computers and Geosciences, 22(9), 1019-1026. Rubin, J., (1994) Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: Wiley, p. 330. Sammon, J. W. J., (1969) "A nonlinear mapping for data structure analysis", IEEE Transactions on Computers, C-18(5), 401-409. Schaale, M., and Furrer, R., (1995) "Land surface classification by neural networks", International Journal of Remote Sensing, 16(16), 3003-3031. Skidmore, A., Turner, B. J., Brinkhof, W., and Knowles, E., (1997) "Performance of a neural network: mapping forests using GIS and remote sensed data", Photogrammetric Engineering and Remote Sensing, 63(5), 501-514. Sweeney, M., Maguire, M., and Shackel, B., (1993) "Evaluating user-computer interaction: a framework", International Journal of Man-Machine Studies, 38(4), 689-711. Tobler, W., (1970) "A computer movie simulating urban growth in the Detroit region", Economic Geography, 46(2), 234-240. Ultsch, A., and Siemon, H., (1990) "Kohonen's self-organizing feature maps for exploratory data analysis", Proceedings International Neural Network Conference INNC '90P, Dordrecht, The Netherlands, pp. 305-308.
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 34
Evaluating Geographic Visualization Tools and Methods: An Approach and Experiment Based upon User Tasks Carolina Tob6n, Department of Geography and Centre for Advanced Spatial Analysis (CASA), University College London (UCL), 1-19 Torrington Place, London WC1E 7HB, UK
Keywords: geovisualization, task typology, usability, factorial experiment design, within-subjects ANOVA
Abstract This chapter discusses the contribution that fields such as experimental design and human-computer interaction can make to geovisualization. In particular, it shows how multiple techniques can be combined not only to evaluate a system' s effectiveness, but also to address the complex issue of task definition in geovisualization research. A methodology for defining a task typology is suggested and used to evaluate a geovisualization system. The results from the evaluation suggest that the approach taken can serve to refine existing task characterizations to reflect the cognitive visual operations that spatial data and geovisualizations can support.
34.1
Introduction
Advances in a number of fields such as visualization in scientific computing (ViSC), Information Visualization, exploratory data analysis (EDA), Cartography, image analysis, and geographic information systems (GIS) have provided theory and methods for geographic visualization or the "visual exploration, analysis, synthesis, and presentation of geospatial data" (MacEachren and Kraak, 2001, p. 1). Environments for visual data investigation have been developed to explore "what if" scenarios or questions that prompt the discovery of relations or patterns that are useful (Beshers and Feiner, 1993). Nevertheless, users other than their developers might perceive these tools as difficult to understand, to use, or to apply to their own work. These aspects have been shown to influence the adoption of systems by their potential users (Davis, 1989). This chapter considers the advantages and significance of incorporating usability evaluation in 645
646
C. Tobdn
the design and development of computer-based geographic visualization (geovisualization) systems or tools that are based on user requirements. Visual data exploration methods are said to be particularly useful when solving ill-defined problems where little is known about a dataset and the goals of its exploration (Keim, 2001). However, usability evaluation methods require the definition of goals and tasks users want to achieve and perform with a system. Some of the challenges for applying such techniques in the context of geovisualization are considered in w34.2. Their integration with experiment design is addressed in w in order to obtain information about system usability and refine the user tasks to be supported. The evaluations of two prototype geovisualization systems are discussed to illustrate how these methods can be combined in practice. The techniques include various usability evaluation methods for assessing the effectiveness of systems and eliciting potential users' perceptions on issues such as ease of use and usefulness of the geovisualization tools, as well as the design of experiments with which to test and measure the effect of factors thought to influence the data exploration process. The results point at the value of refining available task characterizations to reflect operations performed on spatial data, as well as the high degree of interactivity required from geovisualization methods.
34.2
Where do Usability and Geographic Visualization meet?
Human-Computer Interaction (HCI) is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use, as well as the physical, organizational and social environment in which they are embedded (Hewett et al., 2002). Central to the concept of HCI is usability, which deals with the description of a system's fitness for use and its ability to facilitate its potential users' tasks. Usability aims at enhancing user performance by helping to design, develop and deploy systems (Landauer, 1995) that meet user requirements and allow work to be carried out "safely, effectively and enjoyably" (Preece et al., 1994). Usability evaluation techniques comprise a good number of methods that are suitable for different types of systems and users and which are also appropriate for different stages in a system's design process, for example, see Fuhrmann et al., this volume (Chapter 28). These methods are aimed at obtaining either quantitative evidence about user or system performance, or qualitative information about user perception and opinion. They are also often combined in order to obtain richer insights into the question posed by the experiment. In practice, evaluating a system to meet user requirements in a particular application domain may demand the tailoring of available techniques, for example, (see Fuhrmann et al., this volume (Chapter 28)). Usability testing is one such technique, which involves having test participants complete tasks that potential users of the system would want to accomplish and observing and recording how they work and perform (Dumas and Redish, 1999). Performance measurements recorded during these tests may include the time it takes subjects to complete a task, as well as measures of success and error rates. The analysis of these performance measures - either statistical or otherwise - is frequently complemented with information obtained from post-test interviews or questionnaires. The main reason for this
Evaluating Geographic Visualization Tools and Methods
647
is better explained with an example. "Throughput" or "ease of use" is one metric used in HCI for system evaluation. It can be recorded by measuring the amount of time requested to perform tasks and the errors made (Preece et al., 1994). However, in the context of geovisualization, differences in these times might not be indicative of ease of use. Shorter times could be due to users gaining confidence and learning how to use the system, but they might very well be suggestive of users operating the software without consideration of the results they obtain. Longer times could signal difficulties with understanding the graphics or using the tool but they may also be due to users investigating some information in detail. Hence, performance measurements may depend on a number of inter-related factors such as the ease of use of the software, user' s experience, the complexity of the tasks and the functionality provided to support them. A combination of evaluation methods can help elucidate information about the various aspects of a system's usability, as well as the reasons for diverse user behaviour and experience.
34.2.1
Challenges for adopting usability evaluation: complex goals and task definition
Visualization is a cognitive activity (MacEachren, 1995; Spence, 2001) the main purpose of which is to gain understanding of a problem or dataset by representing the data in some graphical form that can reveal otherwise hidden information. Geovisualization extends this definition to include the rather broad goal of visually displaying and interactively exploring spatial data to support a process of hypothesis formation and knowledge construction (MacEachren and Kraak, 2001; Gahegan, 2001; Gahegan, this volume (Chapter 4)). Defining tasks for achieving this goal may not be a straightforward undertaking but it is a necessary condition for applying usability evaluation methods. Furthermore, geovisualization techniques can be useful and pertinent for a wide range of applications with an equally large variety of potential users and data types. At present, this implies that evaluations must be custom tailored to each system and the domain dependent tasks it is designed to support making the generalization of results about the geovisualization tool or concept under study difficult in most cases. Not all of these issues can be addressed by HCI or usability evaluation alone and many argue that an ontological approach may facilitate an agreement about shared conceptualizations that make the domain knowledge of geovisualization more transparent to the user community (Uschold and Gruninger, 1996; Slocum et al., 2001; Fabrikant and Buttenfield, 2001). Nevertheless, research in these areas may benefit from the empirical findings that are obtained via the evaluation of geovisualization tools and an improved understanding of user tasks. In addition, there is still the need to facilitate the design of systems that are fit for purpose and which support user requirements. The remainder of this chapter discusses the relevance of usability evaluation and experiment design techniques for the latter purpose, as well as a means to question whether our systems support all we claim they do.
34.3
Usability Evaluations: Coupling GIS to Visualization Tools
There are a number of systems available for the visual exploration of spatial data, which illustrate two approaches to tool design taken thus far. One has been the coupling between
648
C. Tobdn
GIS and EDA tools for extending GIS functionality. Anselin (1998) provides a thorough review of such approaches. Links between these systems have been attained by their loose and close coupling, which include solutions such as writing objects to intermediate files, calling commands between systems, or writing routines in a system's macro or scripting language to augment the environment's functionality. Depending on the type of coupling, disadvantages of this approach have included a limited size of the datasets that can be used; inefficient algorithms being implemented due to the differences in data structures between systems; or their slow execution which does not allow real-time dynamic interaction because of the form of the communication between systems. A second approach has been the development of stand-alone systems supporting the visual analysis of spatially referenced data such as cdv (Dykes, 1996), Descartes (Andrienko and Andrienko, 1999a-e), CommonGIS (Andrienko and Andrienko, 1999b) and GeoVISTA Studio (Takatsuka and Gahegan, 2002), to give a few examples. Some of these tools are either proof-of-concept systems that are not easily extensible or customizable, do not easily connect to existing databases, or tend to support the analysis of rather small datasets. Others are still work in progress, were not available at the time the experiments reported here were designed and conducted, or demanded a great deal of customisation and programming skills to be developed into applications. The architecture of the systems used for the two evaluations reported below follow the first design approach. Both tools integrate GIS technology to a visualization system. The first tool is commercially available and is intended to support spatial data users, while the second was purposely built for investigating user tasks. As Andrienko et al. (2002) point out, in geovisualization, the researcher often wishes to evaluate a concept or technique rather than a system, but the idea needs to be implemented before any evaluation is possible. Thus, the first evaluation presented here aimed at inspecting features or functionality that facilitated or impeded user tasks. The system proved to have a number of shortcomings discussed in w but its deficiencies highlighted some interesting findings about user needs. These results served as a basis for considering a methodology for defining tasks (w and developing a second prototype to support them (w The second evaluation was, therefore, concerned with analyzing the effect of the different tasks on user performance (w as well as with determining whether the second prototype better enabled visual data exploration (w
34.3.1
Profile and number of participants
The profile of subjects in an evaluation should be carefully considered to reflect the potential users who would be likely to use the system. Due to the nature of the environments, the people invited to take part in the studies were not expected to have any skill using visualization tools but rather basic knowledge of off-the-shelf GIS functionality. This user profile would control for participants who had to learn about GIS terminology and allow them to concentrate on the tasks and on familiarizing themselves with the rest of the visualizations. Hence, participants were carefully selected as it was not the purpose of the evaluations to test the usability flaws of the GIS used as part of the geovisualization environment or users' understanding of GIS operations.
Evaluating Geographic Visualization Tools and Methods
649
Table 34.1. Description of participants in the first evaluation. User
Occupation
Research interests or expertise
1
PhD Geomatic Engineering
Surface network modelling; digital elevation models
PhD Planning
Artificial intelligence in design and planning; models for spatial plan generation
3
PhD Planning
Decision support in architecture and urban planning; coordination and consensus in collaborative, multi-participatory environments
4
PhD Geography
Urban sprawl; agent-based models; cellular automata
5
PhD Geography
Urban classification and Morphology
6
PhD Geography and GIS Lecturer
Transport
7
MSc in GISc
Web-based GIS applications for decision support
8
GIS Professional
Logistics, vehicle fleet routing and scheduling
9
GIS Professional
Transport and database management
Nevertheless, it was of interest to recruit participants from a range of fields or application areas that could potentially use the tools evaluated in their own work. Therefore, participants in the two studies included professionals, masters and doctorate students with a variety of interests but who shared the characteristic of using geographical data regularly. Table 34.1 shows that in the first study six participants were PhD students with interests that ranged from artificial intelligence and decision support in urban planning, to urban sprawl and transport. Two GIS professionals with expertise in logistics and transport participated, as well as a GISc master's student interested in Web-based GIS applications for decision support. Table 34.2 shows a slightly different composition of participants for the second study (7 GIS professionals, 5 PhD and 8 master's students), also with a variety of interests that range from location-based services to issues of data accuracy and error propagation in GIS. 1 Apart from their profile, the number of participants is a crucial issue for any usability study and experiment design, as its selection is closely related to the purpose of the evaluation. The main purpose of the first study was to identify usability shortcomings in the system that could have an effect on the exploration of spatial data, as discussed below. In a study by Nielsen and Molich (1990), less than 50% of all major usability problems were detected with three participants, while 80% were detected by Virzi (1992) 1 Potential users in both studies were invited to take part by letter (sent via email). This was done in two separate batches (one for each evaluation) and each person was considered for one study only. Participants were not paid but rather explained the interest they could potentially have in the geovisualization system as a tool for data investigation. Only one recruitment attempt was necessary and the response rate was close to 50% in both cases.
650
C. Tob6n
Table 34.2. Description of participants in the second evaluation. User
Occupation
Research interests or expertise
1 2 3
PhD Geography PhD Geography PhD Geography
4
PhD Geography
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
PhD Geography MSc Remote Sensing MSc in GISc MSc in GISc MSc GI MSc GI MSc GI MSc GI MSc Conservation Cartographer GIS Professional GIS Professional GIS Professional GIS Professional GIS Professional GIS Professional
Location-based services; urban way-finding Multi-agent simulation; geodemographics Spatial data accuracy; errors and error propagation in GIS Retail Geography; geodemographic profiling and marketing Dynamic location strategies for emergency services Measuring biodiversity Projections in diagrams Retail Water distribution systems and GIS Transport and GIS Coastal management and GIS Web-based information systems and GIS Ecological patches Database management Utilities and telecommunications Gravity modelling Local government and planning Utilities Digital data production and processing Map production
with 4 and 5 participants and 90% with 10 participants. Furthermore, additional users were unlikely to elicit new information in Virzi's (1992) study. In the light of these findings, 9 participants were recruited and considered sufficient to provide this evidence in depth. The purpose of the second study was to further our understanding of the tasks that should be supported by geovisualization tools. In addition, the evaluation was designed to provide statistical evidence on the significance of the factors used in the task definition discussed in w For these reasons, the number of participants followed considerations on the desired power of the experiment. The standard error is regularly used for this purpose and is defined as follows: Standard Error - Standard Deviation x [2/(number of users per task)] 1/2
(34.1)
In Equation (34.1), the standard deviation refers to the uncontrolled variation of individual observations and is usually derived from previous experiments. The standard
Evaluating Geographic Visualization Tools and Methods
651
error refers to the random variation of an estimate from an experiment. Although the two evaluations discussed here examined slightly different pieces of software, they were both designed for the same purpose, and the tasks performed and the participants were similar. Hence, an estimation of the standard deviation from the first evaluation (0.48) was used to obtain an indication of the number of participants that would be necessary to achieve the desired power in the second evaluation. 2 The standard error for the first experiment was estimated at 0.48(2/9) 1/2 - 0 . 2 3 . We can derive a confidence interval from this calculation to suggest that the estimates from the experiment would be subject to a margin of error of no more than _+2 x 0.23 - _+0.46 with a probability or confidence level of 95%. In the second experiment, more participants would perform four times the amount of tasks than in the first. Furthermore, the experiment was designed so that each participant would complete two whole sets of similar questions on two different days as differences in performance with increased exposure to the system were of interest. Taken together, these adjustments would considerably reduce the standard deviation and the standard error to at least 0.48[2/(2 x20)] 1/2 - 0 . 1 1 . Therefore, estimates from the second experiment would be subject to a margin of error of no more than _+2 x 0.11 _+0.22 with a probability or confidence level of 95%.
34.3.2
Evaluations phases
Both evaluations were divided into three phases all of which were conducted by the researcher in a neutral venue. In an initial pre-test phase, participants were given an explanation of the environment's purpose, how to interpret each one of the graphic displays, demonstrated the functionality they would use and given a detailed description of the dataset. 3 Afterwards, participants were given a set of training or warm-up tasks to complete which were similar to those they would encounter in the test and involved using the same dataset as in the main evaluation. Users were then encouraged to ask any questions about the system, as they were advised that no help in solving the tasks would be provided during the evaluation phase. Most of their questions regarded the use and purpose of the graphical displays and the meaning of variables in the dataset. This pre-test stage took an average of 30 min. In the second or main testing phase, users completed a number of tasks on their own. Their on-screen interaction with the software and any comments they made while doing the tasks were captured. 4 These recordings allowed taking a variety of usability metrics such as time to complete tasks and error rates. This stage took users an average of 1 h in both evaluations. In a final post-test phase, the researcher conducted semi-structured interviews to obtain information about participants' perceptions of the environments, and 2 Details of how the standard deviation is estimated are not discussed here but worked examples can be found in Cox (1958) or Keppel (1982). 3 1991 Census data for Bristol (UK) was used in both experiments. The dataset consisted of 17 variables on population income, age, occupation, employment, home ownership, schooling and qualification. They were all provided to the enumeration district level, which is the smallest spatial unit for which census data is made publicly available in the UK. On average, the population of an enumeration district in urban areas is about 500. 4 ScreenCorder by Matchware was used for this purpose.
652
C. Tob6n
details about how they accomplished some of the tasks. For the second evaluation only, users were asked to complete a questionnaire on perceived ease of use and usefulness of the environment before the interview. This stage took an average of 30 min. Results from phases 2 and 3 of the evaluation are discussed in greater detail below.
34.4
The First Environment: The Importance of Dynamic Interaction in Geovisualization
The first usability evaluation was carried out on a commercially available tool: DecisionSite Map Interaction Services (Map IS). DecisionSite, a product of Spotfire Inc. (Boston, USA), is an interactive system for the visual and dynamic exploration of aspatial data designed for supporting decision making. Apart from providing various 2D and 3D graphic displays, including scatter plots, parallel coordinate plots, pie charts, line charts, bar charts, heat maps, and histograms, DecisionSite offers dynamic query filters used to perform SQL queries in near real-time (Ahlberg and Shneiderman, 1994). These graphic devices allow the selection of subsets of data based on the value of the attributes. ArcExplorer from ESRI (Redlands, USA) was coupled as a plug-in to DecisionSite (forming DecisionSite Map IS, a commercially available product) to provide the system with limited GIS functionality - simple user interface, basic tools, and data management and support users of spatial data. The usability study set out to test the suitability of DecisionSite Map IS for the purpose of exploring spatial data and to obtain information about the type of functionality that similar environments should provide. The environment, although available as a commercial product, is regarded here as a prototype since the coupling between the map and the rest of the visualizations in the version tested is rudimentary: the systems are not linked in a manner that allows a simultaneous coordination between views in DecisionSite and the map in ArcExplorer. 5 In other words, changes or selections in the map are not reflected concurrently in the rest of the views and vice versa. The lack of dynamic linking or coordination between the various graphics and map had an effect on both the users' process of data exploration and on the understanding of the environment itself. Two out of nine participants could not understand how to relate the information depicted by the two software components, did not see the utility of the environment for understanding spatial phenomena, and did poorly on the tasks (for similar results when users are presented with tools they do not understand see Harrower et al., 2000). Most other users took a great deal of time to recognize that the data visually encoded in all graphic displays were in fact different re-expressions of the same dataset. They also took time to learn the meaning of the various graphical depictions of data, how to decode the data represented by them, and when to use each one. 5 Instead, users have to press a "transfer" button to allow data selections made in the graphic displays of DecisionSite to be reflected in the map. This selection is shown in the map in the form of points drawn on top of the corresponding geographical features, which do not allow user manipulation. In addition, data selected from the map must also be "transferred" into DecisionSite as a new dataset, obliging the user to close the original data being explored. Alternatively, an image of the map can be put as a backdrop in a scatter plot in DecisionSite where the x and y axes are set to longitude and latitude, but again the map cannot be manipulated or interacted with.
Evaluating Geographic Visualization Tools and Methods
653
Nevertheless, the extra time spent working a way around the tool's limitations was justified for most participants who appreciated the combination of flexible and interactive displays with the mapping capabilities of a GIS data viewer for investigating patterns, trends and relations in the dataset. In their view, the potential benefits of geovisualization tools included the possibility of quickly combining the information provided by a number of graphic displays about a dataset and the speed for manipulating and investigating the dataset visually. However, they expressed the need for a design that not only supported a dynamic coordination between all the views, but also some standard GIS functionality that would allow them to perform spatial queries and possibly some spatial analysis. These two observations guided the development of a new prototype that supported them both, which is discussed in w
34.5
Refining Task Definition
A complex problem in the context of developing and evaluating geovisualization tools is that of task definition. Due to the exploratory nature of geovisualization, the specification of tasks and users can be difficult and there is often no comparable situation in the standard usability engineering practice to the kind of data investigation that dynamic geovisualization can enable (Slocum et al., 2001). Still, tasks must reflect potential users' work processes and allow for an exploration that would be conducive to the "gaining of insight and knowledge construction", which are acknowledged as the main goals of geovisualization methods and tool use (Gahegan, 2001). One factor considered to intervene in the visual exploration of data is the type of visual operation (VO) performed. Some work has been done outside the geovisualization field to characterize visual operators, or the cognitive visual actions necessary to accomplish a task's goal. Casner (1991) and Zhou and Feiner (1998, 1996) have worked on characterizing both visual tasks and data types for the purpose of automating the design of connected visual displays. Casner (1991) in particular is concerned with characterizing the task that a graphic is intended to support in order to automate the design of displays that are effective for their users and needs. Wehrend and Lewis (1990) explore a classification of visualization techniques that is independent of particular application domains but that would provide a conceptual framework for describing techniques and graphics relevant to particular problems. Keim (2002) further suggests a classification of visualization techniques that takes into account both the data type and the interaction and distortion techniques used in the visualization. A similar approach for geovisualization has been suggested by Crampton (2002). He characterizes interactivity types and argues they can be used and combined for designing exploratory digital environments. Edsall et al. (2001) suggest a task typology focused on the complexity of the task, the number of spatial units and the number of variables involved. Using a task analysis methodology, Knapp (1995) selected four visual operators from those considered by Wehrend and Lewis (1990) which she found to be the most used in work with visual designs. The same four cognitive visual actions, "identifying", "locating", "comparing" and "associating", are considered here. In addition to the visual operator, the task that the user attempts can have more components. Take for instance
654
C. Tob6n
the following example which considers area-based georeferenced data: "Do areas with a high proportion of children tend to be located towards the periphery of the study region?" Apart from locating such areas, this task can also be broken down into the number of spatial units explored and the number of variables or attributes that need to be interrogated to reach an answer, as was done by Edsall et al. (2001). Each of these components or factors - visual operator, number of spatial units and number of attributes - can have more than one level. As mentioned above, the visual operator is characterized here as having four levels (identify, locate, compare and associate). The number of spatial units and attributes could comprise any quantity or a continuum of levels. In order to limit the number of tasks that users would attempt in the evaluation and to keep the experiment design as simple as possible, two levels are defined for these latter factors: one (or two) and many. All combinations of the levels of these factors (4 x 2 x 2) define a typology of 16 tasks (see Table 34.3 for examples) used to evaluate the environment and users' behaviour.
34.6
Experiment Design
An experiment where we are interested in all or nearly all factor combinations is called a factorial experiment. This is a technique that allows estimating the effect of any of the factors in the outcome of a response variable (such as users' time performance), as well as their interactions. In other words, it enables us to measure the average effect of each factor as well as the differences of such effect at varying levels of the other factors. For instance, this experiment design can assist in the understanding of the effect of the VO on users' time to complete a task. However, the outcome of that performance measure may be different for different numbers of spatial units and attributes explored. Factors' main effects and interactions are typically estimated using analysis of variance (ANOVA) which is discussed in w Factorial experiments are well suited for understanding main effects and interactions for at least two reasons. On the one hand, all factors can be varied together allowing the researcher to adopt a rich and more revealing multi-dimensional view of users' response to the different tasks (Keppel, 1982). This is not to say that experiments dealing with only one variable are not appropriate in some occasions. However, if the effect of a factor on the outcome of a task varies with the levels of other factors, experimenting with one variable at a time will produce poor results because of the presence of interactions (Montgomery, 1997; Cox, 1958). On the other hand, the gain of using factorial designs in these situations increases with the number of factors considered without compromising the power of the experiment or its ability to detect differences between the various conditions or combinations offactors (Keppel, 1982). Factorial designs are also more efficient than experiments where one variable is considered at a time as fewer observations have to be recorded. These are desirable properties for at least two reasons. Participants tend to be affected by tiredness and lengthy tests and they tend to have restrictive time schedules that do not usually permit their taking part in multiple or long evaluations (for an example of how problematic these issues can be for an evaluator, see Andrienko et al., 2002). Therefore, an economic design is advantageous since a common limitation of usability evaluations is finding participants
Evaluating Geographic Visualization Tools and Methods
655
Table 34.3. Task typology. Visual operator
Number of spatial units
Number of attributes
Identify
1
Locate Compare
1 1
Associate Identify Locate
1 1 1
1 Many Many
Compare Associate Identify Locate Compare
1 1 Many Many Many
Many Many Many Many Many
Associate Identify
Many Many
Many 1
Locate Compare Associate
Many Many Many
1 1 1
Examples Can you identify the area with the lowest household income? Write down the ID. Are areas with a large proportion of children close to secondary schools?
Areas with a high proportion of working wives tend to have low proportions of children. Can you locate an area where this is not the case? Write down the ID.
Is the relation between aggregate household income and being a male resident with a qualification different in the north west than in the south east of the study area? Identify the areas with the lowest income score that are close to the river. Write down the IDs.
You are an estate agent with a client interested in an area where she could locate a Gourmet shop. Can you recommend three locations within the study area?
that fulfil a profile of potential system users who have the time to participate (Dumas and Redish, 1999). Apart from these practical considerations, this methodology allows for as many factors and levels of each factor to be defined. Although not without adding some
656
C. Tob6n
complexity to the analysis of the results especially as the number of levels increase, this implies that the experiment design discussed above can be adapted to incorporate any number of factors or levels which may be deemed of relevance for defining tasks to evaluate a geovisualization tool.
34.6.1
Evaluating the second prototype
The second prototype environment was developed to support the tasks defined above and to solve the shortcomings of the first system evaluated. The environment was built by coupling two software packages, a commercially available GIS (ArcGIS from ESRI) and software designed for visualization (DecisionSite from Spotfire Inc.) using Component Object Model (COM) tools. The COM standard allows the two applications to connect and communicate directly without using intermediate or temporary files for exchanging data or functions, and form an environment designed to support the dynamic and interactive exploration of spatial (vector) data-point, lines and polygons and their associated attribute values. Additional applications that rely upon COM include those reported by Williams and Kindel (1994) and Ungerer and Goodchild (2002). The environment includes a set of dynamically linked or coordinated graphical displays including the map, where dynamic querying and the simultaneous selection and highlighting of features in all views are supported. For the evaluations, only three types of views in DecisionSite were demonstrated and used by participants during the test sessions (scatter plots, parallel coordinate plots and histograms). Similarly, only a few GIS operations available in the GIS package were demonstrated. This was intended to simplify the evaluation and to avoid overloading users with functionality or techniques to learn or remember as they cannot be expected to master all components of a complex or large system in the short time period characteristic of an evaluation session. Nevertheless, participants were free to use other functionality and the more experienced GIS users did utilize some features of the GIS that were not demonstrated.
34.6.2
Analyzing the results: repeated measures ANOVA
A factorial experiment where all participants complete all tasks is known as withinsubjects or repeated measures design. This is the most common experimental design with which to study the changes in performance that result from successive experience with different conditions or tasks. 6 Repeated measures ANOVA is used to estimate factors' main effects and interactions when all members of the sample (all test participants) are measured under different conditions (all tasks) for the same characteristic (time to complete a task). In this case, all 20 users finished a set of 16 tasks in each of the two sessions they all attended. The time taken to complete each task was recorded separately for each individual and each session. A great deal of variation between participants is to be expected in this measurement not only due to their individual differences but also because each individual may respond differently to the tasks for reasons such as changes 6 Note that a requirement of the experiment design is that the tasks are administered in a random order which is different for each participant and each session. These issues are not discussed here further but for a detailed explanation see Montgomery (1997) or Cox (1958).
Evaluating Geographic Visualization Tools and Methods
657
in attention, motivation and learning (Keppel, 1982; recall, for instance, the large value for the standard error of the first evaluation). Repeated measures ANOVA is a powerful way of accounting for this heterogeneity because it takes into account the fact that there is more than one observation per subject, which can be used to supply information on the main effect of each factor. ANOVA tests the null hypothesis of no differences between population variances (or mean squares) of the performance variable measured for each task. In other words, it compares differences among task means (also called betweengroups differences) and differences among subjects receiving the same task within a task group (or within-group differences). This is reported by the F ratio which is approximately one when the null hypothesis is true and greater than one when it is false, with a significance given by the associated p value (Keppel, 1982). Main effects
Table 34.4 shows the results from the repeated measures ANOVA procedure which estimates all main or average effects and interactions. 7 For the VO factor, the null hypothesis tested is that the type of cognitive operation does not have an effect on user performance. Since the F ratio for the main effect of this factor is large IF(3, 57) = 28.59; p < 0.0001], we can conclude with confidence that in the population from which the sample was drawn, time performance does change with the visual operator. Similarly, the effect of the number of spatial units factor (NU) on time performance is highly significant as the F ratio suggests IF(l, 19) -- 55.95;p < 0.0001]. For the number of attributes factor (NA), the associated Fratio is large [F(1, 19) = 86.96;p < 0.0001] indicating an effect of the number of attributes explored in the time it took participants to complete a task. Finally, the session or day (D) when the evaluation was carried out [F(1, 19) -- 6.65;p < 0.0184] has also an effect on the response variable. This is in accordance with studies that show the importance of training for enhancing the effectiveness of visual interfaces for information retrieval (Andrienko et al., 2002; Sutcliffe et al., 2000), as well as to improve subject performance (Burin et al., 2000; Kyllonen et al., 1984). Interactions
Some significant and interesting interactions with the visual operator include those between NU x VO [F(3, 57) = 14.25;p < 0.0001] and NA x VO [F(3, 57) = 9.57;p < 0.0001]. This means that the effect of investigating more spatial units or variables is different for different cognitive visual actions. On the other hand, the results show that differences in time performance do not reliably depend on the number of spatial units explored in conjunction with the amount of variables investigated as the not significant interaction between NU x NA illustrates [F(3, 57) = 0.07; p < 0.7992]. 8 Note that one 3-way (NU x NA x VO) and the 4-way interactions (NU x NA x VO X D) are also significant - [F(3, 57) -- 12.49;p < 0.0001] for the 3-way and IF(3, 57) - 5.99; p < 0.0013] for the 4-way interaction. All other 3-way interactions that involved the session factor (D) are not significant. The 4-way 7 All estimations were generated using SAS version 5.8. 8 All 2-way interactions with the session or day (D) variable are significant (D x NU [F(1, 19) = 16.76;p < 0.0006]; D x NA [F(1, 19) -- 6.82;p < 0.0171]; D x VO [F(1, 19) = 19.92;p < 0.0001]).
658
C. Tob6n
Table 34.4. ANOVA results summary.
DF
Main NU Error NA Error VO Error D Error
Sum of squares
Mean square
F Ratio
P Value
55.95
0.000
86.96
0.000
28.59
0.000
6.65
0.018
0.07
0.799
14.25
0.000
9.57
0.000
16.76
0.001
6.82
0.17
19.92
0.000
0.000
Effects (NU) (NA) (VO) (Day)
1 19 1 19 3 57 1 19
16.46 5.59 24.61 5.38 25.19 16.74 4.24 12.10
16.46 0.29 24.61 0.28 8.40 0.29 4.24 0.64
1 19 3 57 3 57 1 19 1 19 3 57
0.01 3.57 15.07 20.09 7.84 15.57 4.48 5.08 2.09 5.82 14.39 13.72
0.01 0.19 5.02 0.35 2.61 0.27 4.48 0.27 2.09 0.31 4.79 0.24
3 57 1 19 3 57 3 57
8.65 13.16 0.12 4.65 0.82 16.78 0.11 14.32
2.88 0.23 0.12 0.25 0.27 0.29 0.04 0.25
12.49
3 57 620
4.55 14.44 295.64
1.52 0.25
2-Way Interactions NU x NA Error (NU x NA) NU x VO Error (NU x VO) NA x VO Error (NA x VO) D x NU Error (D x NU) D x NA Error (D x NA) D x VO Error (D x VO)
3-Way Interactions NU x NA x VO Error (NU x NA x VO) D x NU x NA Error (D x NU x NA) D x NU x VO Error (D x NU x VO) D x NA x VO Error (D x NA x VO)
0.48
0.50
0.92
0.44
0.14
0.94
5.99
0.001
4-Way Interactions NU x NA x VO x D Error (NU x NA x VO x D) Total
interaction is an interesting result because it indicates that the 3-way one occurred differently during the two sessions. This is explored in more detail below.
Graphic interpretation of main effects and interactions To interpret the 3-way significant interaction, it is necessary to fix one of the factors at each one of its levels and interpret the interactions of the other two. This can be done
659
Evaluating Geographic Visualization Tools and Methods
NUxNA (VO=ldentify) Session 2
NUxNA (VO=ldentify) Session 1 300
300
(1)
250
250 200
200 J
150 f
100
150
J
........._..-....=
100
f
J
50
50
0
0 One Attribute
Many Attributes
One Attribute
___Am One Spatial Unit --4--- ManySpatial Units
----&--
Many Attributes
One Spatial U n i t - - - . . - - - M a n y Spatial Units
Figure 34.1. Interaction between number of spatial units (NU) and number of attributes explored (NA) when the visual operator (VO) is set to its first level. (a) NU x NA interaction for session 1 when VO is fixed to its first level; (b) NU x NA interaction for session 2 when VO is fixed to "identify".
graphically as shown in Figures 34.1- 34.4, which plot the mean time response for tasks with different level combinations of NU and NA, for each level of VO. For instance, each point in Figure 34.1 shows the mean response of all users in tasks where the VO is identifying an attribute value or feature. In order to understand how the interactions differed in each session, it is necessary to further split the data into the two days. Therefore, the graphs on the left of all figures plot the means of response times for all participants and which were gathered during the first session and the graph on the right those of the second session. Each pair of points that represent tasks where NU has the same level are joined by a dotted line labelled "one unit" or "many units" explored. This is to facilitate the comparison between the factors. The y axis in all graphs indicates time in seconds. The x axis codes the two categories or levels of the NA factor, namely, how many attributes were explored. The graphs can be interpreted as follows: parallel lines
NUxNA (VO=Locate) Session 2
NUxNA (VO=Locate) Session 1 300
300~
250
250
200
200
150 100
....._ ....._ ......-. -lit
150 100
I f
50 0~n
50 0
(2)
One Attribute -..~i--One
Many Attributes
Spatial Unit - - - m - - -
Many Spatial Units
One Attribute ~9J , - -
Many Attributes
One Spatial Unit - - - = - - - Many Spatial Units
Figure 34.2. Interaction between number of spatial units (NU) and number of attributes explored (NA) when the visual operator (VO) is set to its second level. (a) NU x NA interaction for session 1 when VO is fixed to its second level; (b) NU x NA interaction for session 2 when VO is fixed to "locate".
660
C. Tob6n
NU xNA (VO=Compare) Session 2
NUxNA (VO=Compare) Session 1 300 250
300
(3) A"-..
250
200
200
150
150
100
100
50
50
(4) A ' ' " . . . .
---z'_..-,-,t
n--'""
0.-
0 One Attribute ---A--
Many Attributes
One Attribute
One Spatial Unit ---In--- Many Spatial Units
~--
Many Attributes
One Spatial Unit---..--- Many Spatial Units
Figure 34.3. Interaction between number of spatial units (NU) and number of attributes explored (NA) when the visual operator (VO) is set to its third level. (a) NU x NA interaction for session I when VO is fixed to its third level; (b) NU x NA interaction for session 2 when VO is fixed to "compare".
indicate no interaction between NU and NA at a particular level of VO. Positive or negative slopes indicate a main effect of NA of the same sign on the response variable. Horizontal lines indicate no average effect of NA. Coincident lines, on the contrary, are an indication of no main effects of NU. In Figure 34.1, most of the lines in the graphs show a positive slope which indicates the (significant) main effect of the NA factor in time performance. The lower levels in the mean times for the tasks completed during the second session indicate that participants got faster with practice. The fact that the lines are almost coincident in the fight-most graph shows that there was no main effect of NU during the second sessions (for VO set to "identify") and that the main effect of NA was also less prominent. More interestingly, note the point labelled (1) in Figure 34.1 (left) which is much higher, even in relative terms, than its counterpart in the second session. For that task, users had to
NUxNA (VO=Associate) Session 2
NUxNA (VO=Associate) Session 1 300
30~:)
250
254:) ,r 20,:)
200 j
150 J
I
9
f
100
15:)
'-
..... ~ 1 1
10:)
5:) :)..
50 0 One Attribute
Many Attributes
-9- One Spatial Unit ---"--- Many Spatial Units
One Attribute
Many Attributes
--~A-- One Spatial Unit ---..--- Many Spatial Units
Figure 34.4. Interaction between number of spatial units (NU) and number of attributes explored (NA) when the visual operator (VO) is set to its fourth level. (a) NU x NA interaction for session 1 when VO is fixed to its fourth level; (b) NU x NA interaction for session 2 when VO is fixed to "associate".
Evaluating Geographic Visualization Tools and Methods
661
identify a value or a feature relative to a geographic entity, while all other tasks only required identifying a value or feature (see Table 34.3 for examples). Before discussing this further, some other results are shown. Figure 34.2 shows the mean response of all users in tasks where the VO was to locate an attribute value or feature. Note that for the first day (Figure 34.2, left), the lines are parallel and both have positive slopes. This is an indication of the main effects of both NA and NU, but of no interactions between these factors (for VO set to "locate"). Once again, note the point in the fight graph labelled as (2). Participants were required for that task to locate a value or feature relative to a spatial entity, as opposed to simply locating the value or feature. Figure 34.3 shows the mean response of all users in tasks where the VO was to compare attribute values or spatial patterns. Hence, it shows the NU • NA interactions for sessions 1 (left) and 2 (fight) when the VO was at its third level ("compare"). The slopes of all lines in the graphs are an indication of main effects of NA on time performance. Once again, the points labelled as (3) and (4), correspond to tasks where users had to compare values or patterns relative to a spatial feature. Figure 34.4 shows the mean response of all users in tasks where the VO was to associate attribute values or spatial patterns. The point labelled as (5) has a slightly higher mean time than its "many units" counterpart in the same graph. This point corresponds to a task where users had to do an association between attribute values or patterns that occurred relative to a spatial feature. These results suggest that the number of spatial units and variables explored can have a significant effect on time performance. This came as no surprise because it seems intuitive that investigating more attributes or larger areas might take longer. However, it was thought that there would be significant interactions between NU and NA. In other words, that the effect of exploring more variables on user behaviour would differ for varying area sizes. The results do not support that idea. More interestingly for tool design considerations are the different effects of these factors for the different levels of the visual operator. Furthermore, the evaluation suggests a need to define visual cognitive operations of finer granularity that can describe spatial tasks and operations with geographical data more precisely. Andrienko et al., this volume (Chapter 5) explore some of the reasons for considering this issue. In addition, ongoing research is investigating the obtained difference in performance in tasks where a cognitive visual action is performed in relation to features in the Geography. The conjecture at present is that their difference is not simply due to a higher complexity of the tasks tested but rather due to the different cognitive processes involved. This idea is not discussed here further as it transcends the scope of the chapter but it points to the need to draw from research in other fields such as cognitive science in order to advance the state-of-the-art in geovisualization. It suggests that there may be fundamental differences between the cognitive processes required to participate in visualization of geographic spaces as opposed to information spaces, and the tools required to support such processes may also be different.
662
34.7
C. Tob6n
Perceived Ease of Use and Usefulness
Apart from examining the task typology suggested, it was crucial to know whether the coupling between the two software packages enabled the user sample to explore spatial data effectively and whether the tool was found useful and easy to use. These aspects were inspected using a questionnaire designed as a Likert scale, originally proposed by Davis (1989), which was tailored to reflect the system being evaluated. This type of questionnaires present a set of statements with which subjects are asked to express their agreement or disagreement according to a scale. Each degree of agreement is given a numerical value. The properties of the scale allow that a total can be calculated by adding all responses for each subject, with high values showing strong levels of agreement with a statement. In this case, a 7-point scale was used where a value of 7 was assigned to the highest level of agreement (strongly agree) and 1 to the most disagreement (strongly disagree). The middle of the scale was reserved for undecided views about a statement (do not know) and assigned a value of 4. The questionnaire administered showed that most users considered the environment to be very useful. They also perceived it as easy to use although less strongly. Users strongly believed that the tool increased the speed of their work, and helped them find information easily and quickly. They also perceived that the tool was easy to learn, control and master. In a post-test semi-structured interview, all participants suggested possible applications of the environment to their line of work, especially for comparing scenarios and distilling information from datasets. Although, some had not worked with or rarely used census data, the great majority of participants believed they had been able to understand various aspects of the data set provided. They also expressed that obtaining the same insights using a GIS only would have been much more time consuming and cumbersome, if possible at all. Users were especially interested in the flexibility of the visual displays, the speed of the environment, the possibility of quickly exploring scenarios and the easy selection of data in all graphical depictions. By the end of the two sessions, they highly valued being able to see the sensitivity of answers and observed spatial patterns to the inclusion or exclusion of criteria for solving a task. They also expressed plenty of ideas about possible extensions for particular application domains, as well as some perceived limitations of the environment for addressing some of the tasks.
34.8
Discussion and Conclusions
The main finding of this experiment is that there may be fundamental differences between the cognitive processes required to participate in the visualization of geographic spaces and those required for visualization of information spaces that are not spatial. It is argued that the reason for this is not merely a difference between the VOs considered but more fundamentally in the information processing that must be accomplished with geospatial data. This is an important new finding in the sense that it may imply that the processes that must be supported in geographic visualization are at least different and possibly more complex than those needed to investigate aspatial data.
Evaluating Geographic Visualization Tools and Methods
663
A question that remains is whether we can generalize the results from these experiments to new users or conditions. The results from the experiments were found to be robust to issues such as GIS expertise or frequency of computer use. In other words, these variables were not found to have any effects on the time it took participants to complete the tasks, or to have any interaction with the factors included in the experiment. Although generalizing the results in w to the population from which the sample participants were drawn cannot be achieved without some degree of uncertainty, the power of the experiment design plus the wide range of conditions investigated through the various combinations of factors and levels, provide some confidence in the appropriateness of extrapolating the findings about the environment tested beyond the bounds of the experiment. Nevertheless, further evaluations along these lines hopefully conducted by other research groups would provide results with which to compare those discussed here, enriching the debate about task definition and geovisualization tool design. The possibilities for cross-fertilization between the evaluation methods reported and the revived interest in cognitive research in geovisualization and GISc seem particularly pertinent at present. For instance, research in cognitive psychology on spatial cognition (Richardson et al., 1999; Montello and Freundschuh, 2004), the representation in memory of geographical knowledge and its effects on geographic judgements as well as in biases in geographical knowledge (Friedman et al., 2002; Friedman and Brown, 2000), have profound implications for building solid theoretical principles for geovisualizationmfor example, (see Ware and Plumlee this volume (Chapter 29))mas well as on the design of computer-based tools that aim to support hypothesis formation and the construction of knowledge about spatial data--for example, see Gahegan this volume (Chapter 4). It is suggested that evaluations similar to those presented here may contribute to such research by supplying information about user tasks and requirements that support their work processes. In addition, the techniques discussed are indeed an effective means to evaluate whether the tools we develop are usable, useful and fit for purpose, which are crucial considerations if geovisualization environments are to support a wider range of users. Those evaluated here had GIS users in mind and were tested for area-based georeferenced data. Nevertheless, the task definition and methods suggested for evaluating the systems can be easily adapted or extended to appraise a wide range of tools, data types and user requirements.
Acknowledgements The author wishes to thank the editors and the anonymous reviewers for their helpful comments which were very useful in the preparation of the final manuscript. Similarly, she wishes to thank those with whom interesting discussions about aspects of the work and this chapter have been held as they have greatly enriched her research.
References Ahlberg, C., and Shneiderman, B., (1994) "Visual information seeking: tight coupling of dynamic query filters with starfield displays", In: Adelson, B., Dumais, S., and
664
C. Tob6n
Olson, J. S., (eds.), CHI'S 94 Human Factors in Computing Systems, Boston, MA, 24-28 April. New York: ACM, pp. 313-317. Andrienko, G., and Andrienko, N., (1999a), Seasonal Migration of White Storks, 1998-1999. Online: http://www.ais.fhg.de/and/java/birds/(23/10/03). Andrienko, G., and Andrienko, N., (1999b) "GIS visualization support to the C4.5 classification algorithm of KDD", Proceedings of the 19th International Cartographic Conference, pp. 747-755. Andrienko, G., and Andrienko, N., (1999c) "Knowledge-based visualization to support spatial data mining", Proceedings of Intelligent Data Analysis. Berlin: Springer Verlag, pp. 149-160. Andrienko, G., and Andrienko, N., (1999d) "Making a GIS intelligent: CommonGIS project view", AGILE'99 Conference, Rome, April 15-17, pp. 19-24. Andrienko, G. L., and Andrienko, N. V., (1999e) "Interactive maps for visual data exploration", International Journal Geographic Information Science, 13(4), 355-374. Andrienko, N., Andrienko, G., Voss, H., Bernardo, F., Hipolito, J., and Kretchmer, U., (forthcoming) "Testing the usability of interactive maps in common GIS", Cartographic and Geographic Information Science, CaGIS (in press). Anselin, L., (1998) "Exploratory spatial data analysis in a geocomputational environment", In: Longley, P. A., Brooks, S. M., McDonell, R., and Macmillan, B., (eds.), Geocomputation: A Primer. Chichester: Wiley, pp. 77-95. Beshers, C., and Feiner, S., (1993) "Autovisual: rule-based design of interactive multivariate visualizations", IEEE Computer Graphics and Applications, 13(4), 41-49. Burin, D. I., Delgado, A. R., and Prieto, G., (2000) "Solution strategies and gender differences in spatial visualization tasks", Psicol6gica, 21,275-286. Casner, S. M., (1991) "A task-analytic approach to the automated design of graphic presentations", A CM Transactions on Graphics, 1O, 111-115. Cox, D. R., (1958)Planning of Experiments. Chichester: Wiley, p. 308. Crampton, J. W., (2002) "Interactivity types in geographic visualization", Cartography and Geographic Information Science, 29(2), 85-98. Davis, F. D., (1989) "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly, 13(3), 319-340, September. Dumas, J. S., and Redish, J. C., (1999) A Practical Guide to Usability Testing. Exeter: Intellect Books, p. 404. Dykes, J. A., (1996) "Dynamic maps for spatial science: a unified approach to cartographic visualization", In: Parker, D., (ed.),, Innovations in Geographical Information Systems 3. London: Francis and Taylor, pp. 171-181. Edsall, R. M., MacEachren, A. M., and Pickle, L. J., (2001) "Case study: design and assessment of an enhanced geographic information system for exploration of multivariate health statistics", In: Andrews, K., Roth, S., and Wong, P. C., (eds.), Proceedings of the IEEE Symposium on Information Visualization, San Diego, CA, pp. 159-162. Fabrikant, S. I., and Buttenfield, B. P., (2001) "Formalizing semantic spaces for information access", Annals Association of American Geographers, 91(2), 263-290.
Evaluating Geographic Visualization Tools and Methods
665
Friedman, A., and Brown, N. R., (2000) "Reasoning about geography", Journal of Experimental Psychology: General, 129(2), 193-219. Friedman, A., Kerkman, D. D., and Brown, N., (2002) "Spatial location judgments: a cross-national comparison of estimation bias in subjective North American geography", Psychonomic Bulletin and Review, 9, 615-623. Gahegan, M., (2001) "Visual exploration in geography: analysis with light", In: Miller, H. J., and Han, J., (eds.), Geographic Data Mining and Knowledge Discovery. London: Taylor and Francis, pp. 260-287. Harrower, M., MacEachren, A. M., and Griffin, A. L., (2000) "Developing a geographic visualization tool to support earth science learning", Cartography and Geographic Information Science, 27(4), 279-293. Hewett, T. T., Baecker, R. M., Card, S. K., Carey, J. G., Gasen, M. M., Mantei, G., Perlman, G. W., Strong, G. and Verplank, B., 2002 ASM SIGCHI Curricula for Human-Computer Interaction: 2 Definition and Overview of Human Computer Interaction. Online: http://www.acm.org/sigchi/cdg/cdg2.html#2_l (23/10/03). Keim, D. A., (2001) "Visual exploration of large data sets", Communications of the A CM (CACM), 44(8), 38-44. Keim, D. A., (2002) "Information visualization and visual data mining", IEEE Transactions on Visualization and Computer Graphics, 8(1), 1-8, (January-March). Keppel, G., (1982) Design and Analysis: A Researcher's Handbook. London: PrenticeHall, p. 669. Knapp, L., (1995) "A task analysis approach to the visualization of geographic data", In: Nyerges, T. L., Mark, D. M., Laurini, R., and Egenhofer, M. J., (eds.), Cognitive Aspects of Human-Computer Interaction for Geographic Information Systems. Dordrecht: Kluwer Academic, pp. 355-371. Kyllonen, P. C., Lohman, D. F., and Woltz, D. J., (1984) "Componential modeling of alternative strategies for performing spatial tasks", Journal of Educational Psychology, 76(6), 1325-1345. Landauer, T. K., (1995) The Trouble with Computers - Usefulness, Usability, and Productivity. Cambridge, MA: The MIT Press, p. 425. MacEachren, A. M., (1995) How Maps Work: Representation, Visualization, and Design. New York: The Guildford Press, p. 513. MacEachren, A. M., and Kraak, M. J., (2001) "Research challenges in geovisualization", Cartography and Geographic Information Science, Special Issue on Geovisualization, 28(1), 3-12. Montello, D. R., and Freundschuh, S. M., (2004) "Cognition of geographic information", In: McMaster, B., (ed.), Research Agenda for the UCGIS, (in press). Montgomery, D. C., (1997) Design and Analysis of Experiments. Chichester: Wiley, p. 704. Nielsen, J., and Molich, R., (1990) "Heuristic evaluation of user interfaces", In: Carrasco Chew, J., and Whiteside, J., (eds.), CHI's 90 Human Factors in Computing Systems, Seattle, Washington, April 1-5, ACM, pp. 249-256. Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T., (1994) HumanComputer Interaction. Harlow: Addison-Wesley, p. 775.
666
C. Tob6n
Richardson, A. E., Montello, D., and Hegarty, M., (1999) "Spatial knowledge acquisition from maps, and from navigation in real and virtual environments", Memory and Cognition, 27, 741-750. Slocum, T. A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D. R., Fuhrmann, S., and Hedley, N. R., (2001) "Cognitive and usability issues in geovisualization", Cartography and Geographic Information Science, 28(1), 61-75. Spence, R., (2001) Information Visualization. Harlow: Addison Wesley/ACM Press Books, 206 pp. Sutcliffe, A. G., Ennis, M., and Hu, J., (2000) "Evaluating the effectiveness of visual user interfaces for information retrieval", International Journal of Human-Computer Studies, 53, 741-763. Takatsuka, M., and Gahegan, M., (2002) "GeoVISTA Studio: a codeless visual programming environment for geoscientific data analysis and visualization", Computers and Geosciences, 28(10), 1131-1144. Ungerer, M. J., and Goodchild, M. F., (2002) "Integrating spatial data analysis and GIS: a new implementation using the component object model (COM)", International Journal of Geographic Information Science, 16(1), 41-53. Uschold, M., and Gruninger, M., (1996) "Ontologies: principles, methods and applications", Knowledge Engineering Review, 11(2), 93-115. Virzi, R., (1992) "Refining the test face of usability evaluation: how many subjects is enough?", Human Factors, 34, 457-468. Wehrend, S., and Lewis, C., (eds.), (1990) "A problem-oriented classification of visualization techniques", Proceedings of the First IEEE Conference on Visualization: Visualization 90, Los Alamitos, CA. New York: ACM Press, pp. 139-143. Williams, S. and Kindel, C., (1994) The Component Object Model: A Technical Overview. Online: http://msdn.microsoft.com/library/default.asp?url -- /library/ en-us/dncomg/html/msdn_comppr.asp (23/10/03). Zhou, M. X., and Feiner, S. K., (1996) "Data characterization for automatically visualizing heterogeneous information", IEEE Symposium on Information Visualization, San Francisco, CA, 28-29 October, pp. 13-20. Zhou, M. X., and Feiner, S. K., (1998) "Visual task characterization for automated visual discourse synthesis", A CM SIGCHI Conference on Human Factors in Computing Systems, Los Angeles, CA, 18-23 April, pp. 392-399.
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 35
Cognitively Plausible Information Visualization Sara Irina Fabrikant, Department of Geography, University of California Santa Barbara, Santa Barbara, CA 93106, USA Andr6 Skupin, Department of Geography, University of New Orleans, New Orleans, LA 70148, USA
Keywords: Information Visualization, spatialization, Cartography, spatial cognition, human-computer interaction
Abstract Information Visualization is concerned with the art and technology of designing and implementing highly interactive, computer supported tools for knowledge discovery in large non-spatial databases. Information Visualization displays, also known as information spaces or graphic spatializations, differ from ordinary data visualization and geovisualization in that they may be explored as if they represented spatial information. Information spaces are very often based on spatial metaphors such as location, distance, region, scale, etc., thus potentially affording spatial analysis techniques and geovisualization approaches for data exploration and knowledge discovery. Two major concerns in spatialization can be identified from a GIScience/ geovisualization perspective: the use of space as a data generalization strategy, and the use of spatial representations or maps to depict these data abstractions. A range of theoretical and technical research questions needs to be addressed to assure the construction of cognitively adequate spatializations. In the first part of this chapter we propose a framework for the construction of cognitively plausible semantic information spaces. This theoretical scaffold is based on geographic information theory and includes principles of ontological modeling such as semantic generalization (spatial primitives), geometric generalization (visual variables), association (source-target domain mapping through spatial metaphors), and aggregation (hierarchical organization). In the remainder of the chapter we discuss ways in which the framework may be applied towards the design of cognitively adequate spatializations.
35.1
Introduction
Timely access to relevant information has become a key element in a data-rich society. Graphic depiction of information is an interdisciplinary research endeavor involving 667
668
S.I. Fabrikant & A. Skupin
human-computer interaction (HCI), visual data mining, exploratory data analysis, and related fields in a search for mechanisms to navigate in and access information from vast databases. Such Information Visualizations often rely on the use of spatial metaphors for depiction. These representations are also known as spatializations, or information spaces. Spatialization is defined here as a data transformation method based on spatial metaphors, with the aim of generating a cognitively adequate graphic representation (e.g., a depiction that matches human's internal visualization capabilities) for data exploration and knowledge discovery in multi-dimensional databases. Spatialization not only provides the construction of visual descriptions and summaries of large data repositories but also creates opportunities for visual queries and sense-making approaches. Although information spaces are abundant and span a wide array of application areas mostly outside of Geography (Card et al., 1999), a structured approach based on solid theoretical foundations to formalize the underlying representational framework seems to be missing. Two key concerns should be addressed from a usability standpoint: the use of spatial metaphors as a data transformation strategy, and the effectiveness of spatial depictions for knowledge extraction. As argued in this chapter, an explicit and structured spatialization design strategy needs to be in place before usable information spaces can be constructed and tested for usability (Fabrikant and Buttenfield, 2001). Improving knowledge discovery in data-rich environments by visual means is also a key concern in the GIScience community (Buckley et al., 2000; Buttenfield et al., 2000). It is surprising, however, that most of the spatialization work is carried out outside of GIScience, with the exception of a handful of geographers (for example, Couclelis, 1998; Skupin, 2000, 2002a,b; Fabrikant, 2000a,b; Fabrikant and Buttenfield, 1997; Kuhn and Blumenthal, 1996; Tilton and Andrews, 1994). It seems obvious that GIScience (particularly through its cartographic roots) is well positioned to address the challenges of designing information spaces, but GIScientists should also transfer their geovisualization know-how to the InfoVis community. GIScience provides the perspectives of space and place, as well as the necessary visual, verbal, mathematical and cognitive approaches to construct cognitively adequate spatial representations (National Research Council, 1997). Cognitive adequacy extends the concept of cognitive plausibility, a term used by psychologists to assess the accuracy with which models are believed to represent human cognition (Edwards, 2001). We define cognitively plausible Information Visualization as a graphic display designed such that it matches human's internal visualization capabilities well. A cognitively adequate depiction is understood here as graphic display that not only supports humans' internal visualization capabilities optimally, but is able to augment people's mental visualization capabilities for complex reasoning and problem solving in abstract domains (Hegarty, 2002). The notion of cognitive plausibility aims at unifying aspects of usability and usefulness in Information Visualizations, as suggested by Fuhrmann et al., this volume (Chapter 28). In this chapter we first propose a spatialization framework based on GIScience/geovisualization, including semantic generalization and geometric generalization as components of a two-step transformation process. In the second part of the chapter
Cognitively Plausible Information Visualization
669
we show that the proposed framework can be linked to space conceptualizations and transformations by giving some examples in the context of text document visualization.
35.2
Spatialization Framework
The goal of this chapter is to devise a spatialization framework that is generic enough to be context independent, but specific enough to represent the domain appropriately. Due to its novelty, a major challenge for Information Visualization has been so far to identify relevant theoretical foundations to support rapid technical developments. We argue that without a solid theoretical foundation, measurement of success in Information Visualization (e.g., usability evaluation) may be hindered. Three main design challenges can be identified for generating cognitively adequate displays: 1. encoding database meaning into appropriate spatial representations for knowledge discovery (e g., database semantics represented with spatial metaphors); 2. employing adequate visuo-spatial structure to depict this meaning (e.g., space transformations, space types and symbology); 3. controlling the potentially experiential effects spatialized views have on information seekers when exploring semantic spaces to satisfy a particular information need (e g., navigation, visual browsing and knowledge construction). This chapter will focus on design challenges one and two, thus omitting the third issue from the discussion, as it has been addressed elsewhere (Fabrikant and Buttenfield, 2001). Based on design challenges (1) and (2) in above list, we see the construction of cognitively adequate spatializations as a two-step transformation process (Fabrikant and Buttenfield, 2001). First, a semantic generalization is applied to the database. At a theoretical level, the identification of appropriate spatial metaphors to adequately capture the database semantics is of primary concern during this phase. Semantic generalization includes cognitive, experiential and perceptual components. An ontological approach is proposed for this step; we examine how people conceptualize space, and we investigate how these concepts can be metaphorically mapped to preserve their characteristics as spatializations are constructed. The second phase of the spatialization process deals with depicting the semantics encapsulated in spatial metaphors with appropriate visual variables for visual information discovery and knowledge construction (e.g., geometric generalization). By means of cartographic generalization we propose in w how data attributes can be condensed to represent their essential relationships (semantic generalization), and how this meaning can be preserved in geometric characteristics of the depicted features (graphic generalization).
35.2.1
Cartographic generalization
Cartographic generalization is the process of reducing multi-dimensional real-world complexity for depiction in a typically 2D map. Generalization entails reduction in detail
670
S.I. Fabrikant & A. Skupin
as a function of depicting the world at a smaller scale. Two cartographic generalization types are distinguished - geometric generalization, sometimes referred to as graphic generalization, and semantic generalization, also known as object or conceptual generalization (Hake et al., 2002). Generalization is not just about information loss.
35.2.2
Semantic generalization in spatialization
The InfoVis community has been applying a wide array of spatial metaphors in a very diverse set of information space designs (Chen, 1999; Spence, 2001; Ware, 2000). Spatial metaphors are typically used in Information Visualization as semantic vehicles for the spatialization process. One may even argue that some InfoVis researchers are reinventing the cartographic wheel, considering that many information spaces attempt to depict large databases as maps. Information items in such visualizations are typically rendered as points in Euclidean space, and relationships between the data points are depicted with straight lines, or alternatively, with 2D or 3D surfaces (Skupin, 2002b; Fabrikant, 2001 a,b). Analogous to semantic generalization in Cartography, semantic generalization in Information Visualization is about identifying a phenomenon's essential characteristics from a large set of attributes describing it, and mapping those onto an abstract construct (metaphor) for subsequent depiction (graphic symbol). It is important to realize that a metaphor is only like the real thing, not the thing itself (Lakoff, 1987). This means that a metaphor may include only some, but not all characteristics, and may in fact have additional (magical) properties. Consider a digital folder in a computer filing system, for instance. The digital folder exhibits similar properties to a real manila folder in that "files" can be stored in it. However, the digital folder cannot be bent, and files never fall out if it gets too full. In addition, the digital folder exhibits "magical" powers in that it can hold many hierarchically stacked folders, and potentially store an infinite number of files (provided an infinite amount of digital storage space is available). Key questions in Information Visualization that remain include: which spatial metaphors should be utilized for particular data sets (and why), and which metaphors are particularly suited for specific knowledge discovery context? We propose an ontological approach for semantic generalization in Information Visualization. The ontological framework is based on a metaphorical mapping process from a physical source domain (e.g., geographic space) into a conceptual target domain (e.g., semantic document space), as shown in Figure 35.1. Before the spatial metaphors can be depicted, it is necessary to identify the source domain's essential characteristics and formalize the appropriate source-target domain mapping rules. For example, a geographic landscape may serve as a rich source domain for spatialization, as this metaphor includes many sub-metaphors that lend themselves to representing relationships of semantic entities in a data archive. The geographic source domain may provide metaphors, such as feature locations in space (information landmarks), distances between features (similarity between information entities), boundaries delineating regions (information density and information zones), and scale (level of detail) (Fabrikant, 2000b). The feature term used on the left-hand
Cognitively Plausible Information Visualization
671
Figure 35.1. Semantic generalization process.
side in Figure 35.1 suggests a phenomenon embedded within geographic context (i.e., context label). The time dimension is experienced as a sequence of events in the real world ("world has time" in Figure 35.1). However, humans use spatial metaphors verbally and graphically to represent and make sense of the sequential time concept ("space is time" in Figure 35.1). The phrase "We are approaching (or close to) the end of the test" to indicate the imminent end of a time period, or "our summer vacation has flown by" to suggest how quickly a period of time has passed, are two examples in common speech for a metaphorical time-space mapping. A face of a clock is a good graphic example for this. The partitioning of a circle into equal slices representing time units, and the clock hand moving along the "circular time line", are both graphic, spatial metaphors used to indicate the passage of time. The spatial metaphor of a "linear time line" is also very often used for representing departure and arrival times in train or bus schedules. In a spatialized view, time can be represented by spatial metaphors as well. For example, more current items can be placed in the foreground of a spatialization. As time goes by, older items would be pushed towards the back of the display. A metaphorical mapping (i.e., semantic generalization), as shown in Figure 35.1, may seem a straightforward process, if one assumes a simple one-to-one mapping between source and target domain. However, a geographer may take many different perspectives when conceptualizing the geographic domain, depending on the use or analysis context. We define these different contexts as spatial perspectives as shown in Table 35.1. For instance, one may conceptualize geographic space differently when navigating in it (navigable perspective), when analyzing patterns and spatial configurations (vista perspective), when formalizing it mathematically (formal perspective), when conceptualizing it mentally (experiential perspective), or when focusing on spatial processes over time (historic perspective). Regardless of the chosen spatial perspective, the appearance of the information space should change according to the semantic level of detail at which an information seeker wishes to explore the data space. For example, sometimes an information seeker may be interested in an individual
S.I. Fabrikant & A. Skupin
672
Table 35.1. Source domains are listed for each possible combination of geographic perspectives with semantic primitives.
Geographic perspectives Semantic primitives Navigable Vista
Formal
Experiential Historic
Locus
Landmark
Feature Occurrence Object
Point in time
Trajectory
Path
Route
Relation
Link
Period over time
Boundary
Edge
Border
Partition
Boundary
Change
Aggregate
District
Region
Set
Container
State
document, while at other times broader information themes or topics may satisfy information need. Consequently, the chosen spatial metaphors should contain a deep enough structure of coherent sub-metaphors to represent information spaces at various levels of database detail. This relates to the scale problem, a well-known phenomenon in Geography. The scale continuum is a fundamental characteristic of geographic analysis. We argue that even abstract information space designs should consider user's bodily experiences with the real world, as the power of the metaphor lies in the transfer of the familiar into the abstract for better understanding. Good metaphors not only combine semantic and geometric properties from a source domain, but also ideally contain cognitive, emotive and experiential aspects (Lakoff, 1987; Lakoff and Johnson, 1987). Similarly, geographic space is not only characterized by physical or geometric principles but also carries experiential meaning, reflected in human's knowledge structures (Lakoff, 1987; Lakoff and Johnson, 1987) and manifested in cognitive affordances (Gibson, 1979). The key question is whether it may be possible to identify fundamental representational primitives associated with a range of geographic source domains, to make metaphorical mappings useful for information exploration. Table 35.1 suggests a selection of semantic primitives that map onto a set of possible spatial metaphors. The metaphors are grouped by a selection of geographic perspectives. Lynchian (Lynch, 1960) feature types such as, landmarks, paths, edges and districts become important when navigating within a space (column 2: navigable space). Spatial configurations or patterns can be identified and described with geographic source domains when looking down onto a space from high above, or when flying over a space (column 3: vista space). A formal geographic perspective is useful when describing spatial relations mathematically, to be able to simulate spatial behavior or formulate spatial database queries (column 4: formal space). Lakoff-Johnsonian (Lakoff-Johnson, 1987) cognitive image-schemata become important when building a cognitive map of the space being explored. Finally, a historic perspective is important to integrate space and time concepts. The list of geographic perspectives and their respective source domains for metaphorical mapping is by no means exhaustive, but it covers a range of information exploration tasks a user may perform when using a spatialized display (Table 35.1). We have identified four semantic
Cognitively Plausible Information Visualization
673
primitives that are particularly important concepts applicable to a range of information types. They are locus, trajectory, boundary and aggregate as shown in Table 35.1. The left most column lists the semantic primitives that encapsulate essential attributes common to the spatial concepts listed as possible source domains (by row in Table 35.1), based on five geographic perspectives (by column in Table 35.1). Spatialized data can take the form of one (or more) of the four semantic primitives listed in Table 35.1. 9
9
9
9
L o c u s . An information item should have a meaningful location or place in an
information space. Based on a logical frame of reference, an item's relative location is determined by its semantic relationships with other information items in the data space. At its location of origin (locus), the information item is not only represented at its highest level of spatial detail but also at its highest possible database granularity. In some cases this may be a single document, in others one concept in a text, or maybe a pixel in an image. Based on the adopted geographic perspective (five examples are in Table 35.1) an information item may have the function of a landmark in a spatialization, when a user is navigating in the information space, for instance. At other times, when information seekers may want to get an overview of the database, the information item may simply be a structuring feature in the information space, such as a mountain or a depression (i.e., a surface discontinuity). T r a j e c t o r y . This is a linear entity type. We use the general meaning of the term trajectory /tO encompass concepts such as path, progression, or line of some type of development (Merriam-Webster Inc., 2003). In essence, trajectories are semantic relationships between information entities at different locations. For example, a semantic relationship may be a directed or a non-directed link, or a cross-reference between two information entities shown at two specific locations in a data space. The geographic analog would be a path or a route connecting information landmarks. Trajectories may also represent user activity in an information space, for example, search trails an information seeker may have left behind while navigating along a sequence of documents in a semantic space. B o u n d a r y . This constitutes a second linear entity type. Boundaries represent discontinuities (borders) in real spaces and in information spaces. They help partition an information space into zones of relative semantic homogeneity. Boundaries delineate semantic regions. A g g r e g a t e . This represents an areal entity type. Aggregates are the result of classification processes. First, quantitative or qualitative types are assigned to data items (i.e., taxonomy), and then the types are aggregated into groups (i.e., classification). In Geography, a regional system is a spatial classification system, where information entities cluster to form semantic aggregates in 2D (e.g., regions) or 3D (e.g., mountains). The aggregate primitive is not only understood here as a collection of items (e.g., many trees forming a forest) but also as a homogenous zone (with or without a discrete boundary) that can be distinguished from other zones (e.g., a mountain from a depression).
674
35.2.3
S.I. Fabrikant & A. Skupin
Geometric generalization in spatialization
In analogy to geometric generalization in Cartography, geometric generalization in spatialization entails a graphic transformation process where graphic marks are assigned to depict the data. The transformation process follows Saussure's (1993) notions of assigning graphic marks or signs (i.e., the signifier) to the semantic primitives described above (i.e., the signified). The transformation of large heterogeneous data sets into visually accessible information displays at various levels of detail for knowledge acquisition is a longstanding cartographic tradition (Bertin, 1967, 1998). Bertin's system of visual variables has also become known in the information design and Information Visualization communities (van der Waarde and Westendorp, 2001; Mackinlay, 1986). Not only are geovisualizers well positioned to address semantic generalization issues in spatialization, but Cartography also provides a solid generalization framework for identifying effective graphic design solutions, and resolving graphic density issues in Information Visualization. Once the semantic primitive locus, trajectory, boundary and aggregate are accepted as ontological building blocks for the semantic transformation process, they can be straightforwardly represented graphically, using visual variables (Bertin, 1967), including the extensions proposed by MacEachren (1995) and DiBiase et al. (1992). Figure 35.2 outlines how semantic primitives can be depicted as spatial metaphors in a semantic space using visual variables suggested by MacEachren (1995). A cartographer typically tries to match the dimensionality of the graphic symbols used for representation with the dimensionality of the represented feature. Depending on the
Figure 35.2. Semantic primitives matching geometric primitives and visual variables.
Cognitively Plausible Information Visualization
675
display scale, the semantic primitive locus may be represented by a point or an area, the linear primitives trajectory and boundary by a line, and the aggregate primitive by a point or an area symbol. Furthermore, a cartographer will select from a set of visual variables (Figure 35.2) the visual property of a symbol such that it conveys the data characteristics best.
35.3
Applying the Spatialization Framework
A designer can apply adequate geometric generalization principles, as suggested by the proposed theoretical framework, to effectively represent, and maximize the graphic information density in a spatialization (Figures 35.3 and 35.4). Figure 35.3 is an example of a typical Information Visualization devised by two computer scientists from the InfoVis community (Chen and Cart, 1999), based on the Pathfinder Network Scaling (PFNet) technique (Schvaneveldt, 1990). Figure 35.3 contains a static snapshot of a 3D VRML information space. Four variables are shown in this spatialization, depicting a database of conference papers from the ACM conference proceedings on Hypertext, over a period of nine consecutive years. The four variables are:
1. an author's location derived from co-citation relationships with other authors in the database (cyan spheres). Heavily cited authors are located in the center of the spatialization, illustrating their central role in the field. 2. a network of dominant citation links between authors, depicting who is mostly citing whom (orange pipes). 3. the amount of citations per author (height of vertical columns), within ... 4. a three-year sliding window along the overall nine-year period (color-coded stacked columns). The semantic primitive locus for the authors (e.g., act as information landmarks) and trajectory to depict the information flow between authors (e.g., cocitations as links between nodes) are well-chosen spatial metaphors, according to the proposed framework. However, we intend to show below how the spatialization in Figure 35.3 could be graphically improved following the geometric generalization process presented earlier. An information designer may critically argue that the information density is quite low in this spatialization, as the authors needed three dimensions to show four variables. In fact, the third dimension is mostly used for graphic effects (e.g., shaded pipes and spheres), thus lowering the data-ink ratio, which might be used as a measure of graphic effectiveness (Tufte, 1983). In this spatialization, the potential information increase afforded by adding an additional (third) dimension does not outweigh the disadvantages of cognitively, and technologically, more demanding 3D navigation (Westerman, 1998; Westerman and Cribbin, 2000; Ware and Plumlee, this volume (Chapter 29)). Using stacked columns, the designers follow Bertin's principle of encoding magnitude differences by means of the visual size variable. However, many of the 3D nodes are occluded by the stacked columns, even when exploring the space at
676
S.I. Fabrikant & A. Skupin
Figure 35.3. Co-Citation "landscape" (Chen and Carr, 1999). Chaomei Chen's permission to use this figure is gratefully acknowledged.
Figure 35.4. Section of the Reuters news map. An interactive application that matches geometric primitives with semantic primitives as level-of-detail (scale) is varied interactively.
Cognitively Plausible Information Visualization
677
different oblique viewing angles. Flat labels suggest the orthographic (e.g., top-down or birds-eye) perspective as the ideal exploration orientation, which, if chosen, would make the information encoded in the stacked columns obsolete. Figure 35.4 is based on the same spatialization technique (i.e., network scaling) as the previous figure, but we depict a 2D data space following the geometric generalization principles of the spatialization framework as proposed above. This 2D spatialization depicted in Figure 35.4 shows semantic relationships of Reuters news stories over a two-day period in 2000. The information space can be interactively explored in an off-the-shelf geographic information system (GIS). Earlier in this chapter it was argued that multiple geographic perspectives might be a source for spatial metaphors (Table 35.1). Similarly, multiple viewing perspectives (graphic solutions) should be provided to depict the chosen metaphors. For instance, users should be able to switch graphic (or geometric) perspectives by controlling the viewing angle (i.e., rotation in 2D, and azimuth in 3D), by selecting the location of the viewing footprint (i.e., panning), and being able to alter the graphic and semantic levels-of-detail depicted (i.e., zooming). Although the map in Figure 35.4 is in 2D, it can be dynamically rotated around its graphic center, to change the viewing perspective. The labels are always visible and readable, regardless of the chosen viewing perspective. One should be able to zoom in and out of the map to explore the information at various levels of graphic and semantic detail, adhering to the geographic scale principle discussed earlier. Contrary to this, when zooming in or out of the space depicted in Figure 35.3, the spheres and pipes will get bigger or smaller (i.e., graphic zoom). The symbology or geometry of the depicted features does not change according to the semantic level of detail. In cartographical scale changes, a feature may be shown with a point symbol on a small scale map (e.g., cities on an airline map), but the same feature may be shown with an areal symbol on a larger scale topographic map. Providing multiple viewing perspectives on the visualized data also requires that appropriate geometric primitives be matched with semantic primitives depending on the chosen level-of-detail (scale). At the highest level-of-detail, a document may be an individual point in space, which, when clicked on, shows the actual content of the document. At lower levels-of-detail, one may only want to see themes or topics in the information space, for example, represented by homogenous zones, separated by boundaries. In the screen shot from the example application shown in Figure 35.4, geometric primitives are linked to semantic primitives. For example, thematic regions are shown by default at the lowest level of graphic detail (smaller scale map). Unwanted detail is filtered out by aggregating individual items to homogenous zones. The individual documents only become visible when zooming into higher levels of detail (larger scale map), that is, to depict increasing information density while the screen real-estate is kept constant. Five variables are shown in the 2D map, in an attempt to increase the data-ink ratio. Users can dynamically switch variables on or off, to reduce visual complexity if desired. The network map represents: (1) the location of news stories based on semantic relationships to other documents in the news archive (point symbols). Following people's expectation that similar
S.I. Fabrikant & A. Skupin
678
(2) (3) (4) (5)
things form clusters in real space, documents similar in content tend to cluster in the 2D display space; a network of dominant semantic links between documents (line symbols); the magnitude of dominant semantic links between documents (color value of line symbols); the magnitude of document connectivity (graduated point symbols); dominant themes in the database (area symbols).
Unlike Figure 35.3, labels (first-level keywords) are fully visible at all times, and legible at any chosen viewing angle. Labels can also be switched off, if necessary (only a selection is shown in Figure 35.4). Depending on the semantic level of detail, additional labels can be shown (e.g., second- and third-level keywords).
35.3.1
Space types
A certain affinity of geographic and non-geographic Information Visualization becomes apparent as one investigates the procedures by which a high-dimensional input data set is first projected onto a low-dimensional space, then transformed within it, and ultimately visualized. Accordingly, a view of spatialization informed by Cartography and GIScience may contribute to making sense of the myriad of proposed techniques and systems for non-geographic Information Visualization. The ultimate goal of such a viewpoint is to derive methods that implement geographic metaphors in a more complete, and systematic manner than current approaches. For example, once traditional cartographic maps of different scale are understood as expressing geographic phenomena that actually operate at different spatial scales (e.g., global vs. regional vs. local), zoom operations in map-like Information Visualization can be detached from the level-of-detail (i.e., performance-driven) approach common in computer graphics. From the framework presented in Figure 35.2 it follows that the use of semantic primitives is dependent on the depiction scale. Individual entities represented as point features in the information space are useful at the highest level of detail, when navigating in the space, or looking down onto the space, for instance (Table 35.1, top rowmLocus). When viewing the information space at coarser level of detail (e.g., overview), point features may aggregate to regions, and can be represented as homogenous zones (Table 35.1, bottom row--Aggregate). Different 2D layout techniques such as multi-dimensional scaling (MDS), PFNet and self-organizing maps (SOM) may correspond to different conceptualizations of the fundamental make-up of an information space. In GIScience, geographic space is typically conceptualized in one of two ways (Longley et al., 2001): either as an empty space populated by discrete objects, such as point, linear, and areal entities (object/entity view); or as a continuous field, with an infinite number of locations, whose properties can be described by an infinite number of variables (field view). Depending on the phenomenon represented digitally, either a field view or an object view may be more appropriate. For example, human-made features such as houses or bridges are typically represented as
Cognitively Plausible Information Visualization
679
discrete objects, while many natural phenomena such as, elevation or humidity, are commonly conceptualized as continuous fields. The conceptual distinction made between objects and fields tends to translate into a choice between vector and raster data structures in GIS implementations (Longley et al., 2001), and is relevant through all the stages of a GIS project. Looking at the various approaches to dimensionality reduction utilized commonly in Information Visualization, one finds a similar object/field distinction (Skupin, 2002b). Use of a particular projection technique requires the a priori existence of either an object or field conceptualization of the high-dimensional space that is being mapped. This is reflected in the data models used for representing spatialized geometry, and in the ways in which certain depiction solutions can or cannot be used. The framework presented in Table 35.1 and Figure 35.2 is flexible enough to be utilized with either of the two space views. The "locus" primitive in Table 35.1 can be depicted with "pixels" (field view) or with point symbols (object view), for instance. The same is true for the linear and areal semantic primitives. They can be depicted with a linear series or group of pixels, as well as with line or areal symbols. The same applies to the other semantic primitives listed in Table 35.1. The remainder of w highlights some examples.
35.3.2
Object/entity view
Many techniques, like MDS (Kruskal and Wish, 1978) or pathfinder network scaling (Schvaneveldt, 1990), start out with a conceptualization of high-dimensional information space as consisting of distinct, discrete objects. This conceptualization is at play when these methods proceed with a pair-wise computation of object similarities, and finally produce discrete coordinate geometry for individual observations (MDS, PFNet), or explicit network topology between observations (PFNet). Data models and formats used in vector GIS are at that point applicable. To those cartographers well versed in the use of desktop GIS, it is from here a very small step to create engaging, and visually compelling visualizations. Arguably, this is another reason for the current level of success enjoyed by cartographers engaged in nongeographic Information Visualization.
35.3.3
Field view
There are also techniques, like SOM (see Koua and Kraak, this volume (Chapter 33)), that employ a field-like conceptualization of abstract information (Kohonen, 1995). Instead of focusing on individual observations, these are interpreted as representative samples of an information continuum. It is through a tessellation of that highdimensional continuum that visualization efforts are enabled to later implement semantic aggregates, as is necessary for such tasks as the labeling of document clusters (Skupin, 2002a). Raster data models are the most common approach to representing fields in GIS. SOMs do in fact also utilize a raster model, usually with either square or hexagon
680
S.I. Fabrikant & A. Skupin
elements. The transparent, interchangeable use of regular point lattices and pixel grids for training and outputting a SOM, again bears close similarity to how such data are handled in GIS. This raster-like nature also provides a natural control over granularity. When SOMs are used for classification, changing granularity simply leads to finer or coarser classification. When used for visualization, the change of SOM granularity amounts to a control over the scale of the representation. The differences between a 10,000-node SOM and a 50,000-node SOM can thus be considered similar to the differences between satellite images of different spatial resolutions. Armed with this realization, one can thus begin to draw on advances in scale-dependent geographic analysis (Quattrochi and Goodchild, 1997; Wood, this volume (Chapter 15)), including fractal analysis, towards new forms of information space investigations. Another implication of the object vs. field conceptualization of an information space relates to how observations are handled that are not part of the original data set. Field conceptualization allows the mapping of new observations onto an existing spatialization. In a SOM, the high-dimensional continuum is chunked into a limited number of pieces. Each of these will eventually occupy a portion of low-dimensional real estate, as well. Mapping a new observation is simply a matter of finding the highdimensional chunk into which it falls and then finding the low-dimensional location of that chunk. All this is not easily possible with object-based methods, like MDS or spring models (see Rodgers, this volume (Chapter 7)) since the high-dimensional space between original observations remains an ill-defined void.
35.3.4
Space transformations
In geovisualization, including GIS, data required for spatial analysis and display are rarely available in a directly usable form. Thus it becomes important to know the semantic and geometric characteristics of the data used, and how spatial data from different sources (and in different formats) can be fused such that they work jointly for multi-variate analyses. Examples of geometric data transformations may include map projections, affine transformations to register two data sets, edge-matching, etc. Similarly, in non-geographic Information Visualization, transformation procedures become necessary when first projecting a high-dimensional input data set into a lowerdimensional space, then further transforming results of this initial projection towards actual depiction. Once a low-dimensional, geometric configuration is established through a particular projection technique, further transformations on the basis of that geometry may be required in order to achieve a particular depiction. For example, a point configuration created by MDS could be transformed into a surface through interpolation in order to create a terrain-like visualization. The number of possible visualization methods and parameters traditionally used by cartographers is large, and so is consequently the number of transformation methods to achieve these visualizations. Some even see Cartography's potential for transformation of spatial data as its most distinguishing characteristic (Tobler, 1979a,b).
Cognitively Plausible Information Visualization
681
Today, the design of Information Visualizations still tends to be closely linked to the characteristics of the geometric primitives (points, lines, areas) that are generated through a particular projection technique (Skupin, 2002b). For example, if a method like MDS assigns 2D coordinate pairs to a spatialized entity, then those observations tend to be visualized by point symbols. Further geometric transformation is the exception rather than the rule. Arguably, the fascination with such efforts as Themescapes (Wise et al., 1995; Wise, 1999) is in large part due to their attempt to go beyond the bounds of initial geometric configurations toward map-like representation via space transformation (i.e., surface interpolation in the case of Themescapes). Knowledge of Cartography's potential for transformation and of the particular transformations it employs can inform non-geographic Information Visualization in a number of significant ways: It can help to derive from a given low-dimensional configuration a large number of alternative visual representations (Figures 35.5 and 35.6). Having a set of alternatives, one is then in a better position to choose a design which fulfills one or more of the following conditions: 9 9 9
9
fitting certain known characteristics of the mapped high-dimensional domain (e.g., gradual vs. abrupt change); corresponding to the conceptualization of the high-dimensional domain underlying a particular projection technique; corresponding to our knowledge about users' cognitive abilities, preferences, or domain ontology (e.g., dependent on a geographic perspective, shown in Table 35.1); conveying a message pursuant to a certain agenda (in the best cartographic tradition).
These will often be conflicting goals, and are quite similar to design decisions cartographers need to make for geographic datasets. By means of geometric and semantic transformations one can derive spatialized views with different spatialization primitives in order to achieve a depiction that matches users' needs and cognitive capabilities best. The depiction of discrete objects in Figure 35.5, or the field-based depiction in Figure 35.6 are two alternative realizations derived from the same data set, but highlighting different characteristics of the data, dependent on the depiction purpose. The proposed framework guided our design decisions. This included identifying the appropriate semantic primitives, and then matching the relevant geometric transformation technique used for depiction, as discussed in more detail below. The same database of Reuters news stories represented in Figure 35.4 was used to derive different spatialization types in 2D and 3D, as depicted in Figures 35.5 and 35.6. Figure 35.5a illustrates the configuration of a subset of Reuters news stories as discrete semantic loci (depicted as points) in an empty, 2D information space. According to the proposed framework, these points act as landmarks in the otherwise empty semantic space (Table 35.1). Point locations are the result of a 2D spring embedder algorithm (Kamada and Kawai, 1989). In the second panel (Figure 35.5b), dominant semantic links connect document locations to a semantic network in 2D. The semantic links are an instance of the semantic primitive "trajectory". These links show semantic paths or routes
682
S.I. Fabrikant & A. Skupin
6
\ -
J
uc
-a
2.
dtstepped density surface (from vmnui polygons) eontinuoua demiiy surfam (from points) -
G
3 Figure 35.6. Four 3D visualizations derived from a single 2D configuration. (Skupin and Fabrikant, 2003; reprinted with permission from Cartography and Geographic Information Science, 30(2) p. 108).
52 W
684
S.I. Fabrikant & A. Skupin
through the information space, connecting the landmarks. The third panel, Figure 35.5c, shows thematic regions emerging from the database. These regions, examples of the "aggregate" primitive in Table 35.1, are derived from a point-to-area transformation, based on a Voronoi tessellation. In a first transformation step each node is represented with its own zone of influence (e.g., grey polygon boundaries around every point). Secondly, Voronoi polygon boundaries are merged based on cluster membership derived from a hierarchical clustering solution (Masser and Brown, 1975). Two emerging clusters, "world affairs" and "middle east", are highlighted in Figure 35.5c. One can transform semantic primitives from one space model (e.g., object/entity view) into another (e.g., field view) and match the geometric primitive accordingly. Utilizing the same discrete, 2D spring embedder configuration from Figure 35.5a, 3D continuous and discrete surface types can be derived, as examples in Figure 35.6 show below. A switch from an object to a field view may be useful (but not necessary) when looking at the space at coarser level of detail (scale change), and to give viewers a general sense of where entities are densest, or how they cluster ("aggregate" primitive). The four different depictions in Figure 35.6 were generated by means of interpolation, where new data is created to fill the void between the discrete data observations in 2D, and are then depicted in 3D. Which of these representations is more appropriate, and for which particular types of user tasks? Does one of the design solutions convey a detected pattern more effectively than another one? Should we use a particularly compelling method even if it may convey a false sense of the volume and richness of the source data? Is the stepped density surface the most honest depiction, since the underlying 2D geometry was based on an object conceptualization of the highdimensional space? Do "valleys" carry as much meaning as "ridges" do? These are questions that any of the proposed terrain-like Information Visualization techniques, like Themescapes (Wise et al., 1995) or VxInsight (Davidson et al., 1998) will have to able to confront. Answering these questions is not possible, if the application of any arbitrary interpolation technique is seen as sufficient. The ability to consider these questions depends on having available both a solid theoretical framework and, arguably, a rich set of cartographic transformation tools. Those tools should be applied in a systematic, justifiable manner, and informed by the theoretical scaffold. A great deal of research in Cartography and geographic information science has focused on understanding the implications of different methods for generating surfaces from information sampled at points in a field (Lam, 1983) and for generating a surface from values representing discrete objects, such as enumeration units (Tobler, 1979a,b). Scale-related transformations (e.g., to enable zoom operations) are another example for the intersection of GIScience and Information Visualization interests and expertise. The implementation of a scale metaphor through nested cluster hierarchies has recently received attention by geographers and non-geographers alike (Guo et al., 2002; Seo and Shneiderman, 2002; Skupin, 2000, 2002a). Figures 35.7 and 35.8 show an example in which a hierarchical clustering tree (Figure 35.7) drives the creation of scaledependent, map-like, visualizations (Figure 35.8). A base map consisting of Association
Cognitively Plausible Information Visualization
685
Figure 35.7. Hierarchical clustering tree for a finely grained SOM (4800 neurons). Also shown are three horizontal cuts corresponding to a l O-cluster solution (blue), a lO0-cluster solution (red), and an 800-cluster solution (green) (from Skupin, 2002a; @2002 IEEE).
of American Geographers (AAG) conference abstracts is here generalized by first clustering a fine-resolution SOM, then finding the closest SOM neuron to each abstract, and finally merging neighboring abstracts based on shared cluster membership at defined zoom levels (Skupin, 2002a). The computational extraction of label terms (based on the distribution of author-chosen keywords across the full text of all abstracts) is driven by the desire to express characteristics of a cluster while distinguishing it from other clusters that exist at the same scale level. For example, the cluster labeled "precipitation" "nino" - "climate" will contain abstracts dealing with the climate aspect of physical Geography, while the cluster labeled "spatial" - "information" - "data" refers to geographic research focusing on GIS and Cartography. These multi-scale representations are best explored as part of a rich interactive interface. Extension of the (static) "optimal" 2D map design paradigm, typically associated with traditional Cartography, towards interactive exploration has been a main research focus of the geovisualization community in the last decade. Exploratory spatial analysis tools, such as Descartes or GeoVISTA Studio (Andrienko and Andrienko, 1998; Gahegan et al., 2002a,b), featuring dynamically linked multi-dimensional cartographic and statistical data depictions, allow the interactive exploration of different views of the
Figure 35.8. Implementing the scale metaphor via nested semantic aggregates: (a) a complete map of conference abstracts shown in a lO-cluster solution; (b) map portion shown for a lO0-cluster solution; (c) map portion shown for an 800-cluster solution (from Skupin, 2002a; 9 IEEE).
S.I. Fabrikant & A. Skupin
686
same geographic data set. These methods can also be utilized for spatialized displays. The framework presented here supports this in three ways: 1. semantic primitives can be matched to many possible geographic perspectives (five task-dependent perspectives are shown in Table 35.1); 2. adequate geometric primitives can be matched to the semantic primitives (Figure 35.2); 3. long-standing cartographic transformation methods can be used when transformations between semantic and geometric primitives are necessary
35.4
Conclusions and Outlook
This chapter outlines a theoretical framework for the construction of cognitively plausible semantic information spaces. A cognitively plausible Information Visualization is designed such that it matches human' s internal visualization capabilities. The proposed framework focuses on the use of geographic space as a data generalization strategy (ontology), and the use of spatial representations or maps to depict these data abstractions. The building blocks of this spatialization framework are informed by geographic information theory and include principles of ontological modeling, such as semantic generalization (spatial primitives), geometric generalization (visual variables), association (source-target domain mapping through spatial metaphors), and aggregation (hierarchical organization). A sound spatialization framework enables information designers not only to construct conceptually robust and usable information spaces but also allow information seekers to more efficiently extract knowledge buried in large digital data archives. Such a framework can be substantially supported through two major strands of work, namely: (i) research into the cognitive and ontological foundations and implications of how people interact with non-spatial data on the basis of familiar spatial metaphors; (ii) work on the computational techniques that can produce meaningful spatialized geometries, visualizations, and methods of analysis. Ongoing work at the University of California Santa Barbara (UCSB) and the University at Buffalo, is concerned with investigating empirically fundamental cognitive and ontological issues in spatialization, as highlighted in this chapter. A research project at the University of New Orleans is underway for building a system for map-like browsing and analysis of conference abstracts. The goal of the project is to develop a proof-of-concept spatialization system, adhering to the cognitively plausible framework discussed in this chapter. All the semantic primitives proposed in this chapter (including the scale metaphor) have been empirically evaluated in a first series of controlled experiments. Initial empirical evidence looks very promising, and are discussed elsewhere (Fabrikant, 2000a, 2001a; Fabrikant et al., 2002). Outcomes of previous experiments have led to revisions of the initial framework. These empirical results also suggest that cartographic design guidelines are applicable and useful for designing non-geographic information spaces (Fabrikant, 2000a). A new series of human subject testing is currently underway at UCSB to replicate and extend initial
Cognitively Plausible Information Visualization
687
findings. We believe that spatialization designs will greatly benefit from additional empirical evidence gained from fundamental cognitive evaluations. A sound theoretical scaffold is an important starting point for designing adequate experiments to test the validity of an underlying conceptual model used for depiction. This is what this chapter set out to do. Test outcomes will be used in turn to refine the theory, and derive appropriate spatialization design guidelines. Recognition of the existence of different space conceptualizations and of the possible range of geometry-based and semantically driven transformations can help shape future Information Visualization efforts. Specifically, such a view should help to move beyond the current engineering-inspired paradigm, in which specific visualization systems are evaluated for usability within the bounds of ad hoc choices made by system designers. An incorporation of GIScience-inspired ideas regarding space and its transformation in accordance with the notion of cognitive plausibility may lead to a more systematic understanding of issues of usability and usefulness emphasized elsewhere in this volume.
Acknowledgements We would like to thank the book editors Jason Dykes, Alan MacEachren and Menno-Jan Kraak for the stimulating ICA Commission meeting in London (UK), in 2002, and for providing this wonderful opportunity for scholarly debate and interdisciplinary exchange. We are grateful for the detailed feedback provided by the book editors, and the anonymous reviewers, which helped improving the presentation of our ideas. Finally, we gratefully acknowledge funding support by the National Imagery and Mapping Agency (NMA-201-00-1-2005: Sara Fabrikant), and by the Louisiana Board of Regents Support Fund, grant # LEQSF(2002-05)-RD-A-34:Andr6 Skupin).
References Andrienko, G. L., and Andrienko, N. V., (1998) "Dynamic categorization for visual study of spatial information", Programming and Computer Software, 24(3), 108-115. Bertin, J., (1967) Sdmiologie Graphique: Les Diagrammes - les Rdseaux - les Cartes. Paris" Mouton. Bertin, J., (1998) Sdmiologie Graphique: Les Diagrammes - les Rdseaux - les Cartes. Paris" l~ditions de L'l~cole Pratique des Haute l~tudes. Buckley, A. M., Gahegan, M., and Clarke, K., (2000) Emerging themes in GIScience research" geographic visualization. Online: http://www.ucgis.org/priorities/ research/research_white/2000%20Papers/emerging/Geographicvisualization-edit. pdf (23/10/03). Buttenfield, B. P., Gahegan, M., Miller, H., and Yuan, M., (2000) Emerging themes in GIScience research: geospatial data mining and knowledge discovery. Online: http:// www.ucgis.org/priorities/research/research_white/2000%20Papers/emerging/gkd. pdf (23/10/03).
688
S.I. Fabrikant & A. Skupin
Card, S. K., Mackinlay, J. D., and Shneiderman, B., (eds.), (1999) Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann Publishers. Chen, C., (1999) Information Visualization and Virtual Environments. Springer: London, 223 pp. Chen, C., and Carr, L., (1999) "Visualizing the evolution of a subject domain: a case study", Proceedings of the Ninth IEEE Visualization '99 Conference, San Francisco, CA, pp. 449-452, (October 24-29, 1999). Couclelis, H., (1998) "Worlds of information: the geographic metaphor in the visualization of complex information", Cartography and Geographic Information Systems, 25(4), 209-220. Davidson, G. S., Hendrickson, B., Johnson, D. K., Meyers, C. E., and Wylie, B. N., (1998) "Knowledge mining with VxInsight: discovery through interaction", Journal of Intelligent Information Systems, 11 (3), 259-285. DiBiase, D., MacEachren, A. M., Krygier, J. B., and Reeves, C., (1992) "Animation and the role of map design in scientific visualization", Cartography and Geographic Information Systems, 19(4), 201-214, see also 265- 266. Edwards, G., (2001) "A virtual test bed in support of cognitively-aware geomatics technologies", In: Montello, D. R., (ed.), Spatial Information Theory. Foundations of Geographic Information Science, Lecture Notes in Computer Science 2205. Berlin: Springer, pp. 149-155. Fabrikant, S. I., (2000a) Spatial Metaphors for Browsing Large Data Archives, University of Colorado-Boulder, Department of Geography. Fabrikant, S. I., (2000b) "Spatialized browsing in large data archives", Transactions in GIS, 4(1), 65-78. Fabrikant, S. I., (2001a) "Evaluating the usability of the scale metaphor for querying semantic information spaces", In: Montello, D. R., (ed.), Spatial Information Theory: Foundations of Geographic Information Science. Berlin: Springer, pp. 156-171. Fabrikant, S. I., (200 l b) "Visualizing region and scale in semantic spaces", Proceedings of the 20th International Cartographic Conference, ICC 2001, Beijing, China, pp. 2522-2529, (August 6-10, 2001). Fabrikant, S. I., and Buttenfield, B. P., (1997) "Envisioning user access to a large data archive", Proceedings GIS/LIS '97, Cincinnati, OH, pp. 686-692, (October 28-30, 1997). Fabrikant, S. I., and Buttenfield, B. P., (2001) "Formalizing semantic spaces for information access", Annals of the Association of American Geographers, 91(2), 263-290. Fabrikant, S. I., Ruocco, M., Middleton, R., Montello, D. R., and J6rgensen, C., (2002) "The first law of cognitive geography: distance and similarity in semantic space", Proceedings of GIScience 2002, Boulder, CO, pp. 31-33. Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002a) "GeoVISTA Studio: a geocomputational workbench", Computers, Environment and Urban Systems, 26, 267-292.
Cognitively Plausible Information Visualization
689
Gahegan, M., Takatsuka, M., Wheeler, M., and Hardisty, F., (2002b) "Introducing GeoVISTA Studio: an integrated suite of visualization and computational methods for exploration and knowledge construction in geography", Computers, Environment and Urban Systems, 26(4), 267-292. Gibson, J. J., (1979) The Ecological Approach to Visual Perception. Boston, MA: Houghton Miflin. Guo, D., Peuquet, D., and Gahegan, M., (2002) "Opening the black box: interactive hierarchical clustering for multivariate spatial patterns", Proceedings of the Tenth A CM International Symposium on Advances in Geographic Information Systems, McLean, VA, pp. 131-136, (November 8-9, 2002). Hake, G., Grtinreich, D., and Meng, L., (2002) Kartographie - Visualisierung raumzeitlicher Informationen (German). Berlin: de Gruyter. Hegarty, M., (2002) "Mental visualizations and external visualizations", In: Proceedings 24th Annual Conference of the Cognitive Science Society (Cog Sci 2002), August 7-10, George Mason University, FairFax, VA, p. 40. Kamada, T., and Kawai, S., (1989) "An algorithm for drawing general undirected graphs", Information Processing Letters, 31 (1), 7-15. Kohonen, T., (1995) Self-Organizing Maps. Berlin: Springer. Kruskal, J. B., and Wish, M., (1978) Multidimensional Scaling. Sage University Papers. Series on Quantitative Applications in the Social Sciences, Series Number 07-011, Vol.11. Newbury Park: Sage Publications. Kuhn, W., and Blumenthal, B., (1996) Spatialization: Spatial Metaphors for User Interfaces. Vienna: Technical University of Vienna. Lakoff, G., (1987) Women, Fire, And Dangerous Things: What Categories Reveal About The Mind. Chicago, IL: University of Chicago Press. Lakoff, G., and Johnson, M., (1987) Metaphors We Live By. Chicago, IL: University of Chicago Press. Lam, N. S.-N., (1983) "Spatial interpolation methods: a review", The American Cartographer, 10, 129-149. Longley, P. A., Goodchild, M. F., Maguire, D. J., and Rhind, D. W., (2001) Geographical Information Systems and Science. New York: Wiley. Lynch, K., (1960) The Image of the City. Cambridge, MA: MIT Press. MacEachren, A. M., (1995) How Maps Work: Representation, Visualization, and Design. New York: Guildford Press. Mackinlay, J. D., (1986) "Automating the design of graphical presentations of relational information", ACM Transactions on Graphics, 5(2), 110-141. Masser, I., and Brown, P. J. B., (1975) "Hierarchical aggregation procedures for interaction data", Environment and Planning A, 7, 509-523. Merriam-Webster Inc. (2003) Merriam-Webster online language center. Online: http://www.m-w.com (23/10/03). National Research Council (1997) Rediscovering Geography: New Relevance For Science and Society. Washington, DC: National Academy Press. Quattrochi, D. A., and Goodchild, M. F., (1997) Scale in Remote Sensing and GIS. New York: Lewis Publishers.
690
S.I. Fabrikant & A. Skupin
Saussure, F., (1993) Course in General Linguistics. London, UK: Dockworth. Schvaneveldt, R. W., (ed.), (1990) Pathfinder Associative Networks: Studies in Knowledge Organization. Norwood, NJ: Ablex. Seo, J., and Shneiderman, B., (2002) "Interactively exploring hierarchical clustering results", IEEE Computer Graphics and Applications, 35(7), 80-86. Skupin, A., (2000) "From metaphor to method: cartographic perspectives on information visualization", In: Roth, S. F., and Keim, D. A., (eds.), IEEE Symposium on Information Visualization (InfoVis 2000), Salt Lake City, UT, pp. 91-97, (October 9-10, 2000). Skupin, A., (2002a) "A cartographic approach to visualizing conference abstracts", IEEE Computer Graphics and Applications, 22(1), 50-58. Skupin, A., (2002b) "On geometry and transformation in map-like information visualization", In: B6rner, K., and Chen, C., (eds.), Visual Interfaces to Digital Libraries (Lecture Notes in Computer Science 2539). Berlin: Springer, pp. 161-170. Skupin, A., and Fabrikant, S. I., (2003) "Spatialization methods: a cartographic research agenda for non-geographic information visualization", Cartography and Geographic Information Science, 30(2), 95-119. Spence, R., (2001) Information Visualization. Harlow: Addison Wesley/ACM Press Books, 206 pp. Tilton, D. W., and Andrews, S. K., (1994) "Space, place and interface", Cartographica, 30(4), 61-72. Tobler, W. R., (1979a) "Smooth pycnophylactic interpolation for geographical regions", Journal of the American Statistical Association, 74(367), 519-530. Tobler, W. R., (1979b) "A transformational view of cartography", The American Cartographer, 6, 101-106. Tufte, E. R., (1983) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, 197 pp. van der Waarde, K., and Westendorp, P., (2001) "Theme: Jacques Bertin's theories", Information Design Journal, 10(1). Ware, C., (2000) Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann Publishers, 384 pp. Westerman, S. J., (1998) "A comparison of the cognitive demands of navigating two- vs three-dimensional spatial database layouts", Ergonomics, 41,207-216. Westerman, S. J., and Cribbin, T., (2000) "Mapping semantic information in virtual space: dimensions, variance, and individual differences", International Journal of Human-Computer Studies, 53(5), 765-788. Wise, T. A., (1999) "The ecological approach to text visualization", Journal of the American Society of Information Science, 53(13), 1224-1233. Wise, T. A., Thomas, J. J., Pennock, K. A., Lantrip, D. B., Pottier, M., Schur, A., and Crow, V., (1995) "Visualizing the non-visual: spatial analysis and interaction with information from text documents", In: Proceedings of the IEEE Information Visualization (InfoVis, '95). Los Alamitos, CA: IEEE Computer Press, pp. 51-58, (October 30-31, 1995).
Conclusion Advancing Geovisualization
36. Advancing Geovisualization Jason Dykes, Alan M. MacEachren & Menno-Jan Kraak ........................................................................ 693
This Page Intentionally Left Blank
Exploring Geovisualization J. Dykes, A.M. MacEachren, M.-J. Kraak (Editors) 9 2005 Elsevier Ltd. All rights reserved.
Chapter 36
Advancing Geovisualization Jason Dykes, Department of Information Science, City University, London EC1V
0HB, UK Alan M. MacEachren, GeoVISTA Center, Department of Geography, Penn State University, 303 Walker, University Park, PA 16802, USA Menno-Jan Kraak, Department of GeoInformation Processing, ITC, International Institute of Geoinformation Science and Earth Observation, P.O. Box 6, NL-7500 AA Enschede, The Netherlands
Keywords: geovisualization, science, GIScience, Cartography, users, maps, distributed cognition
Abstract This short concluding chapter identifies a number of themes, issues and tensions that permeate through the contributions presented in this collected volume. These include: a focus on the use of geovisualization tools to support a wide range of users in tasks involving the acquisition, processing, and sharing of information; the need to support collaborative work in geovisualization; the various relationships between Information Visualization and geovisualization and ways in which synergies can be developed; the many issues surrounding the advantages and disadvantages of using 2D and 3D representations; the increasing demand for supporting geovisualization through mobile devices and the specific differences and constraints associated with static and mobile tools; the various drivers for developing geovisualization applications and the relationships between them. An argument for grounding efforts to support geovisualization in a deep understanding of the broader scientific process is drawn upon. It is suggested that doing so enables us to link the issues of instrument design, graphical representation, connecting distributed resources and the effective use of technological developments that structure the contributions to this book. Such a perspective provides a framework upon which the work reported here, emanating from a series of related disciplines, can draw. Doing so can help us develop the functionality and tools required to deploy and operationalize geovisualization through effective usable solutions that better employ the knowledge and visual functionality we have separately developed. The concept of "distributed cognition" is drawn from cognitive science as a means of informing such efforts to develop "the map" as a tool for presenting, using, interpreting and understanding information about geographic phenomena. 693
694
36.1
J. Dykes, A.M. MacEachren & M.-J. Kraak
Advancing Geovisualization--Introduction
In the introduction to this book we define geovisualization as a loosely bounded domain that addresses the visual exploration, analysis, synthesis and presentation of data that contains geographic information by integrating approaches from disciplines within GIScience, including Cartography, with those from scientific visualization, image analysis, Information Visualization and exploratory data analysis (Dykes et al., this volume (Chapter 1)). The chapters that have followed offer individually and collectively presented examples of ways in which these and other disciplines can provide theory, methods and tools for a field that can be considered a new branch of Cartography that results in an increasingly interdisciplinary role for the map. As we have seen, this is not the "map" as many readers will know it. While retaining the traditional roles of information repository and presentation device, the modern map should also be seen as a flexible, usable and carefully designed interface to geospatial data. Maps for geovisualization draw upon sophisticated and elegant computational tools to offer interaction with the data behind the representation. They are instruments that encourage exploration of the nature of the geospatial data at hand. As such they are used to stimulate (visual) thinking about geospatial patterns, relationships, and trends and are increasingly employed throughout the GIScientific process. As many contributions in this book have demonstrated, maps for geovisualization often consist of multiple transitory linked views, each displaying a specific alternative representation of any number of phenomena. When designed from a user-centered perspective these instruments for insight can support distributed information access and act as an active mediator among human collaborators in group work with geospatial information. Creating maps that support map use of this form requires geovisualization to be wide in scope, as our exploration has demonstrated. The consequences of this situation are detailed by Research Challenges in Geovisualization (MacEachren and Kraak, 2001b) in which four pertinent themes for research activity are addressed: representation of geospatial information, integration of visual with computational methods of knowledge construction, interface design for geovisualization environments and cognitive/usability aspects of geovisualization. Particular additional challenges cut across these related themes dealing with: leveraging the advances in display and interface technology for effective cartographic representation, developing and extending geovisualization methods and tools to support knowledge construction, facilitating multi-user geovisualization, and understanding geovisualization users and meeting their needs. These challenges were introduced in w1.3 (see also Dykes et al., this volume (Chapter 1)). The chapters presented here draw from each of these themes and begin to address a number of the cross-cutting challenges. This collection of work demonstrates and promotes interdisciplinary communication, one of the key objectives of this project. It also allows us to explore the nature of geovisualization from a number of disciplinary perspectives and identify interdependencies between geovisualization, its practice, issues and challenges, and related academic fields. Cross-disciplinary perspectives are documented here and have been arrived at through the production of the co-authored introductory chapters to each of the sections of the book. We hope that the issues raised
Advancing Geovisualization
695
and perspectives outlined will prompt cross-disciplinary research collaborations to meet the research challenges.
36.2
Advancing Geovisualization--Issues
The chapters presented here also reveal some interesting issues and tensions that have arisen through the process of exploring geovisualization as carried out at our workshop and through subsequent efforts to complete and structure this book. Some of these are documented here with reflective comments offered in order to support our attempts to advance geovisualization through multi-disciplinary effort.
36.2.1
Professional/public geovisualization map use
One issue identified by our exploration of geovisualization relates to the range and type of intended users of any particular geovisualization environment or tool. An early emphasis on the use of interactive maps by individual experts to reveal unknowns (MacEachren and Kraak, 1997) has since been augmented by the development and extensive use of interactive maps in the public domain. This evolution results from the widespread adoption of techniques initially used to further scientific advancement to meet the information needs of a far wider range of map users. Such maps may still be used by individuals to prompt thought (although see w but the design process must consider and cater for a far more extensive and heterogeneous user group than the "maps for experts" typical of early visualization efforts. Fairbairn, this volume (Chapter 26), Brodlie et al., this volume (Chapter 21) and Treinish, this volume (Chapter 20) are amongst those who provide examples of geovisualization applications designed for mass consumption that focus on exploratory tools intended for a wide range of public users. Yet Theus, this volume (Chapter 6) makes a case for a continuing emphasis on tools designed for expert users when arguing that whilst the advantages of an easy-to-use multi-functional tool may satisfy most users, specialists are likely to require the highest possible specification to support their more sophisticated and precise requirements. The tension is illustrated by Treinish, this volume (Chapter 20), who provides Web-based solutions designed to meet the diverse needs of a wide range of specialist and non-specialist users - yet feedback is reported suggesting that specialist users required more functionality than was originally offered as some products were too genetic for particular decision-making tasks. The three approaches to "helping users get started" presented by Plaisant, this volume (Chapter 3) may provide solutions here - the concept of "multi-layered design" offering particular opportunities for addressing the tension between "expert" and "novice" users. Indeed, a number of developing approaches to software personalization may be applicable to addressing this tension caused by the diverse skills and requirements to be found in the range of geovisualization users (including adaptive systems and end-user programming, for example).
36.2.2
Individual/collaborative geovisualization
The sharing of resources and development and use of environments that permit Computer Supported Cooperative Work (CSCW) are related key themes that emerge from a number
696
J. Dykes, A.M. MacEachren & M.-J. Kraak
of the chapters presented here (Andrienko et al., this volume (Chapter 5); Brodlie et al., this volume (Chapter 21); MacEachren, this volume (Chapter 22); Brodlie, this volume (Chapter 23); Schroeder, this volume (Chapter 24); Fuhrmann and Pike, this volume (Chapter 31)). These contributions demonstrate that in addition to moving from the realm of experts well into the public domain geovisualization has progressed from a focus on the individual (user, tool, dataset) to support collaborative and distributed geovisualization. With distributed geovisualization, the inputs can come from many sources, geovisualization resources can be built from distributed components, and multiple individuals can collaborate with and through the resulting geovisualization tools. Several chapters in this volume identify common areas in which we can work together, particularly those in Section D on "connecting people, data and resources", which indicate that we are progressing in facilitating multi-user geovisualization. Further advances can be achieved by addressing the issues that arise in association with new technologies and forms of map use and employing these approaches effectively in conjunction with other available methods such as the kind of "radical collocation" (Brodlie, this volume (Chapter 23)) used to explore the status of geovisualization in this project--a traditional collocated meeting and workshop (see w1.5).
36.2.3
Information Visualization/geovisualization
The relationships between geovisualization and Information Visualization are highlighted throughout the chapters of this book by examples of the continuing use of methods from one discipline in applications relating primarily to the other. Rodgers, this volume (Chapter 7), Roberts, this volume (Chapter 8) and Koua and Kraak, this volume (Chapter 33) provide examples of techniques from Information Visualization that have application in geovisualization. Fabrikant and Skupin, this volume (Chapter 35) show how views and methods developed in geovisualization might be successfully applied to Information Visualization landscapes and Keim et al., this volume (Chapter 2) offer new algorithms for transforming spatial data to support visualization. These examples demonstrate clear opportunities to address challenges associated with effective representation and ensuring that our geovisualization tools are usable through crossfertilization between geovisualization and Information Visualization. Nevertheless, the collective experience of those participating in this project suggests that the scope of such interaction has been relatively limited so far. Despite the communication that has taken place in developing Exploring Geovisualization, the limited set of common references cited by authors from each domain is a clear demonstration of a lack of regular sharing across disciplines. Our workshop and this resulting book (which involved a crossdisciplinary peer review procedure) provide a stimulus toward more focused and systematic sharing of perspectives. We hope that the process continues. The situation may not be as simple as this, however. Despite the opportunities for sharing methods and techniques across disciplines, Tob6n, this volume (Chapter 34) presents some evidence that geovisualization may be fundamentally different from Information Visualization. Her experimental results suggest that comparison between geographic features is more difficult than comparison across data spaces. Whilst in this
Advancing Geovisualization
697
case the findings may be dependent upon the relatively loose linking between the particular software products used, the existence of such deep-seated distinctions between the types of task, data and representation involved in the two fields deserves concerted and additional attention, and may shape future relations between Information Visualization and geovisualization.
36.2.4
2D/3D geovisualization
The use of 3D views can clearly produce attractive depictions of geospatial information, the advantages of 3D are well argued and the growing practicality of 3D geovisualization is supported by a number of developments reported in Section C and elsewhere (see Wood et al., this volume (Chapter 14); Wood, this volume (Chapter 15); D611ner, this volume (Chapter 16); Kirschenbauer, this volume (Chapter 18); Treinish, this volume (Chapter 20); Coors et al., this volume (Chapter 27); Fabrikant and Skupin, this volume (Chapter 35)). Collaboration between those developing computational approaches to 3D and those needing to use 3D to model and represent their data should be mutually beneficial. But the message from Ware and Plumlee, this volume (Chapter 29) is that from the perspective of the user, 3D is "different", as physiological considerations mean that the third dimension along the line of sight has a more limited capacity for conveying information than the two dimensions of space that are orthogonal to it. As a result, Ware and Plumlee, this volume (Chapter 29) argue that the third dimension should be treated as non-uniform and "used sparingly". Many open questions remain concerning how and when to take best advantage of what 3D rendering technologies have to offer. Kirschenbauer, this volume (Chapter 18) provides some experimental evidence from which we may begin to develop answers and D611ner, this volume (Chapter 16) demonstrates that the opportunities for using 3D are increasing. Constructing and sharing the technical, theoretical and empirical knowledge derived from the kinds of disciplines represented in this book will help us develop our approach to 3D applications and uses of other novel, experiential and multi-sensory representation techniques.
36.2.5
Static/mobile geovisualization
The range of work and perspectives presented here also indicates that technological advances continue to drive and require geovisualization research (w explores a tension relating to this issue). Efforts focused around portable and wearable PDAs and other communications devices that draw upon real-time personal global positioning are discussed in a number of contributions (for example, Andrienko et al., this volume (Chapter 5); Mountain, this volume (Chapter 9); Fairbairn, this volume (Chapter 26); Coors et al., this volume (Chapter 27)). Such exciting developments are likely to continue to attract significant attention in the near future. This location-based personalization reemphasizes one of the original defining themes of visualization as a form of map use the map devoted to particular needs and designed based upon these requirements (see w Advances in communications technologies mean that users of such maps can now be loosely connected through phone and wireless networks providing universal
698
J. Dykes, A.M. MacEachren & M.-J. Kraak
access to customizable, map-based interfaces to geospatial information that may be location and context sensitive. And as we have seen, geovisualization is increasingly supporting and being used to promote group work by drawing upon these technologies. The combination of mobile communications devices and the technology being developed to support CSCW discussed in w could result in powerful information access and decision-making aids for use in a full range of activities that might include coordinated decision-making for crisis management, real-world data collection in support of field sciences, or on-the-fly datamining through which mobile (information) consumers can be identified and targeted. Should the advances in the power and capabilities of the personal computer be repeated in the case of mobile devices, new research issues will develop from new ways of working with and using maps that draw upon high performance distributed collaborative computing. The likelihood of these developments occurring is supported by significant research efforts in this area (Jobst, 2004; Gartner, 2004), progress associated with global spatial data infrastructures (The Global Spatial Data Infrastructure Secretariat, 2004) and the development of computing and standards for geoprocessing interoperability (Open GIS Consortium Inc., 2003), since devices can be easily connected with the resources available via this infrastructure. Developing GridGIS efforts (Schroeder, this volume (Chapter 24)) may add significant technical solutions and processing power to further strengthen such possibilities (National Institute for Environmental eScience, 2003).
36.2.6
Human/computer
A final issue relates to whether geovisualization developments are motivated by known requirements and the need for solutions or developing possibilities and the desire to explore and apply them. This can be considered a tension between approaches that are initiated by user needs and technological opportunities, respectively and draws attention to geovisualization research initiated by "demand" as opposed to "supply". The series of drivers for innovation cited by the authors of Section B (Andrienko et al., this volume (Chapter 5)) characterize this distinction. These include technology, data and interoperability as motivations for creating new instruments in addition to the users and tasks that dominate the user-centered perspective offered in Section E (Fuhrmann et al., this volume (Chapter 28)). These different motivations for tool development are both valid and have important roles to play. Very specific research efforts that "chase ideas" driven by technical possibilities and data opportunities remain a viable and valuable activity. Andrienko et al., this volume (Chapter 5) argue strongly for a place for the rhetorical approach of "show and tell" (see Gahegan, this volume (Chapter 4)) that typifies supply driven demonstrator development. Equally, usability testing and cognitive studies that focus on very specific tasks, tools and users can lead to important insights regarding user requirements and the nature of likely demand. Indeed, as circumstances change, so new geovisualization approaches are both possible and required, with motivations derived from both human needs and technological potential.
Advancing Geovisualization
699
The key requirement of each of these activities is that the results are fed in to the wider realm of efforts to advance geovisualization, and that the broader picture is considered by researchers with each of these emphases. Tensions occur when prototypes that are effective for testing and presenting ideas, and that may be deployable in specific situations (perhaps by their developers), are far from easy to use in an applied and systematic manner by a wider range of users. It is not always clear what such prototypes are best used for or who they are best used by. As Fuhrmann et al., this volume (Chapter 28) remind us, there is indeed a tendency for these custom geovisualization tools to appear "largely technology driven" when the tools are utilized more widely than by their developers, resulting in considerable usability concerns. Whilst elegant software design and increasing levels of interoperability mean that the boundaries between "developer" and "user" are narrowing in a field in which tasks are often difficult to define, predict or constrain (Gahegan, this volume (Chapter 4); Dykes, this volume (Chapter 13)), the usercentered approach and techniques that it encompasses offer an important opportunity for addressing this tension by "bridging the gap between developers and users" (Fuhrmann et al., this volume (Chapter 28)). The need for cooperation and collaboration to advance geovisualization emerges once more in order that the expertise of both groups, driven by different motivational factors, is drawn upon to build these bridges and address these tensions. Testing our tools "in the wild" (Hutchins, 1996) is one approach that may encourage and support such cooperation and contribute towards this objective (see also w The approach may offer a useful means of drawing technology driven work in to the wider research efforts towards user-centered design and could complement (or even initiate) subsequent laboratory tests of a more formal nature. Developing a task-driven approach to software development (Andrienko et al., thisvolume (Chapter 10); Treinish, this volume (Chapter 20)) is another way of relating the possibly conflicting stimuli of human and computer, need and opportunity or demand and supply. When linked with the wider body of research, such groundwork, moves us towards the development of theoretical methods that support verification and testing and from which we can apply deductive and model-based reasoning to our knowledge of geovisualization tools and techniques. Once this knowledge has been developed, the incorporation and use of techniques that are shown to be effective into operational software within a broader framework is equally important research activity.
36.3
Advancing Geovisualization---Priorities and Futures
These and other trends and tensions can be addressed and advanced through continuing research that draws upon cross-disciplinary collaboration, making this an exciting time for geovisualization. Indeed, the continuing advances in technology and knowledge that are energizing geovisualization applications, expectations and practice leads Gahegan, this volume (Chapter 4) to argue that we are entering a new phase of visualization. Gahegan's thesis is that the maturation of the relevant disciplines and technologies and the technical and organizational advances in the sharing of resources
700
J. Dykes, A.M. MacEachren & M.-J. Kraak
will allow us to move away from a focus on technique design towards clear interoperability (Gahegan, this volume (Chapter 4)). Achieving this interoperability is a particularly ambitious objective. Doing so requires the geovisualization community to progress from research that produces useful examples and demonstrators to one where we build and use organizational infrastructures to develop standards, share knowledge and generate deployable tools and techniques that are fully operational and ready for effective application in GIScience (w Advancing and supporting geovisualization in the way that Gahegan describes involves more coordination and agreement than existing attempts towards interoperability (for example, through sharing data and techniques; see Andrienko et al., this volume (Chapter 5)). We will need to "move beyond tools" and direct attention towards using geovisualization to support science (Gahegan, this volume (Chapter 4)) by grounding our efforts to build geovisualization environments within a deep understanding of the broader scientific process. Much of the work presented in Exploring Geovisualization can be used to support this approach and it is worth considering using Gahegan's model as a basis for our continuing research efforts. Gahegan's perspective offers a focus on the use and evaluation of our tools and can be used to establish clear links between the four topics that structure the discussions and ultimately the contributions to this book, namely issues of instrument design, dimensionality of representation, collaborative work and the effective use of the techniques at our disposal. Gahegan's three goals of defining a logical framework, adopting an open approach to tool development and developing an infrastructure to support this activity delineate an opportunity for collaboration amongst the geovisualization community to coordinate the development of the functionality and tools that are required to fully deploy and operationalize geovisualization. The work and approaches reported here can be usefully applied to achieving these objectives, which Andrienko et al., this volume (Chapter 5) describe as "the next major challenge - one that must be faced collectively for progress to be made". The documented dependencies between geovisualization and the other domains represented in this book suggest that doing so will require coordinated contributions from a number of related academic disciplines. More generally, Gahegan's approach also reminds us that geovisualization is focused on supporting human thinking. We are moving beyond an early emphasis on simply making dynamic visualization tools work. This is evident in Sections D and E, with the focus on distributed and collaborative systems and human-centered design to support task achievement. A growing emphasis on geovisualization usability is addressing the need for a "human-centered approach" in geovisualization. As Fabrikant and Skupin, this volume (Chapter 35) note, we should "... move beyond the current engineering-inspired paradigm, in which specific visualization systems are evaluated for usability within the bounds of ad-hoc choices made by system designers." The key challenge here is to develop a comprehensive understanding of how humans, in a variety of application domains, use dynamic visual displays to think and to perform a multitude of tasks and apply this understanding to our coordinated attempts to develop deployable tools to support thinking. Once again Hutchins (1996) provides a perspective from cognitive science that is worth considering as a basis for tackling this challenge.
Advancing Geovisualization
701
Distributed cognition treats cognition not as a process confined uniquely to the mind of individuals but as a distributed process that can involve both multiple individuals collaboratively sharing the development of ideas and understanding as well as external artifacts onto which aspects of cognition can be offloaded. A potentially useful analogy exists between this concept and the way in which geovisualization researchers are sharing knowledge and working on separate issues to advance the field, again demonstrating the potential benefits to geovisualization of drawing from a wide range of scientific endeavor. Hutchins (1996) states that "there is scarcely a more important concept than the division of labor. In terms of the energy budget of a human group and the efficiency with which a group exploits its physical environment, social organizational factors often produce group properties that differ considerably from the properties of individuals". Gahegan's approach this volume (Chapter 4) requires us to develop systems that will use our collective knowledge and labor effectively to advance geovisualization. The Research Agenda (MacEachren and Kraak, 2001 a), the exploration of geovisualization reported in this book and the desire to seek wider knowledge and employ wider efforts are part of this process. They may be useful in helping us to structure our efforts in order to make effective use of the resources available by dividing labor in ways that will achieve the wider collective aims of the geovisualization community. The calls from the instrument creators for more interoperability, from those with a focus on "connecting resources" for collaborative work to be used and supported more fully and from those with a focus on "design and evaluation issues" for more usable tools and techniques that enable us to assess usability are all transactions in this process. Exploring Geovisualization is a step in scoping this framework and developing these systems and this book could be considered an external artifact that supports this process. Hutchins (1996) argues that human cognition is "situated in a complex sociocultural world." This perspective can be usefully applied to support geovisualization "in the wild," as our collective approach to geovisualization develops. Ultimately, this knowledge of geovisualization is also situated and so developed and used in a complex sociocultural world. The teams that Hutchins describes as "cognitive and computational systems" are precisely those who we are trying to include, rally and leverage through this project to make most effective use of our collective research. The approaches offered by Hutchins and Gahegan provide considerable opportunities for informing and structuring the kinds of research activity reported in this book and coordinating the continued efforts required in order to address the tensions, issues and research priorities identified here.
36.4
Advancing Geovisualization--Conclusion
The work reported in the chapters of this book introduces the diverse efforts being directed towards advancing geovisualization as a discipline and practice. Through the processes of presenting, discussing, completing, reviewing and publishing this work the authors represented here are both participating in and promoting the kinds of interdisciplinary communication and cross-disciplinary work called for in Research Challenges in Geovisualization (MacEachren and Kraak, 2001a). The chapters, revised following inter-disciplinary discussion, and the work reported through the co-authored
702
J. Dykes, A.M. MacEachren & M.-J. Kraak
introductions are a step in this direction. Many of the established geovisualization challenges (MacEachren and Kraak, 2001b; Dykes et al., this volume (Chapter 1)) are being addressed, and new issues are being identified through collaborative work, discussion and the sharing of multi-disciplinary perspectives as we advance the research agenda. Within the scope of this exploration of geovisualization we offer some views on the various tensions, issues and trends to which this collection of contributions draws attention and identify some priorities that may establish a bright geovisualization future. Yet the work presented here neither defines the full scope of influence of geovisualization nor addresses the full range of research challenges. Nor are all motivations for geovisualization documented. Our exploration is more precise in some areas than others due to the research foci and experience of those attending the workshop upon which this book is based and those whose written contributions have been accepted for publication. The front cover of the book illustrates that this is the case and is described in the Preface. In this volume, we explore geovisualization predominantly from the perspectives of Cartography, Computer Science and Information Visualization. A variety of additional disciplines continue to offer considerable potential to geovisualization including, but by no means limited to, information science and the cognitive sciences. Continued research effort and collaboration with experts in these fields remain key objectives for advancing geovisualization in directions that remain unexplored in the context of this project. Despite the clear and mutually beneficial influences between geovisualization and cognate disciplines, the nature of geovisualization will always be strongly rooted in the GISciences. This is apparent if we consider the future of the map, which, in its escalating variety of exciting and sometimes even astonishing forms, is likely to remain the primary tool with which we present, use, interpret and understand geospatial data. With effective, elegant and sophisticated maps, that draw upon developments in our knowledge, improvements in our models and innovations in our techniques we will aim to support and advance our individual and collective knowledge of locations, distributions and interactions in space and time.
References Gartner, G., (2004) Location Based Services & TeleCartography (Geowissenschaftliche Mitteilungen, Heft 66). Vienna: Institute of Cartography and Geo-Media Techniques and International Cartographic Association. Hutchins, E., (1996) Cognition in the Wild. Cambridge, MA: MIT Press, p. 408. Jobst, M., (2004) "LBS & TeleCartography", In: Jobst, M., (ed.), Second Symposium Vienna 2004, Online: http://linservl.ikr.tuwien.ac.at/symposium2004/htm/program. html (23/10/03). MacEachren, A. M., and Kraak, M. J., (1997) "Exploratory cartographic visualization: advancing the agenda", Computers and Geosciences, 23(4), 335-344.
Advancing Geovisualization
703
MacEachren, A. M., and Kraak, M.-J. (Eds.), (2001a) Research challenges in geovisualization, cartography and geographic information science (Special Issue on Geovisualization), 28(1), American Congress on Mapping and Surveying, 80 pp. MacEachren, A. M., and Kraak, M. J., (200 lb) "Research challenges in geovisualization, cartography and geographic information science", Special Issue on Geovisualization, 28(1), 3-12. National Institute for Environmental eScience (2003) Environmental eScience: GridGIS, Online: http://niees.ac.uk/events/GridGIS/(23/10/03). Open GIS Consortium Inc. (2003) Open GIS Consortium, Inc. (OGC). Online: http://www.opengis.org (23/10/03). The Global Spatial Data Infrastructure Secretariat (2004) The Global Spatial Data Infrastructure. Online: http://www.gsdi.org (23/10/03).
This Page Intentionally Left Blank
Index "visualization effort" 13, 265, 276, 278, 554, 679, 687, 695 3D 3, 10, 13, 14, 25-8, 30, 34-9, 55, 57, 59, 71, 73-5, 105, 109, 112, 117, 153, 155, 163-4, 166-7, 170-1, 174, 181, 184, 185, 191,210, 212, 240, 243, 250-1, 281,283, 295 - 307, 309, 313-16, 318-19, 322, 325-8,330, 333,335,337, 339-42, 345-7, 349, 352-3, 357-8, 363-78, 380-4, 389-90, 392-4, 396, 400, 403-6, 408,415-16, 420, 432, 434, 439, 448, 454, 466, 473, 483,485, 488, 529-30, 534-8, 541-8, 567-8, 570-2, 574-5, 578, 630, 632, 634-6, 641,652, 670, 673, 675, 677, 681,684, 693, 697 3D computer graphics 13, 55, 281,283, 307, 325-7, 340, 342, 345 3D maps 71,306, 337-8, 341,364, 529, 535, 537-8, 541-6 3D modelling 389-90, 392-4, 396 3D rendering 315-16, 318-19, 325-7, 335, 337, 339, 341-2, 697 abductive reasoning 223, 227-8, 233 accuracy 83, 182, 246, 255-8, 260, 327, 345-6, 357-8, 365, 376-80, 382, 438, 561,581,584, 586, 614, 641,649-50, 668 activities 4, 83-8, 92-5, 130, 233, 235, 281, 394, 425, 427, 445,455-6, 464, 491, 498, 502, 516, 558, 592, 595, 599-601, 604, 606, 631,641-2, 698-9 agents 13, 117, 196, 223, 226, 228-33, 235, 239-40, 253-54, 258, 432, 434, 438, 477, 482, 489, 522 animated maps 105, 201,203, 582
animation 5, 14, 63, 107-8, 163, 174, 191, 202-3, 206-8, 211-13, 215, 240, 269, 272, 274-6, 339, 346, 394, 398, 404-7, 409-10, 412-20, 437, 471,473, 518, 520, 535, 577, 580, 582-6 application sharing 463,467, 470-2 autostereoscopic display 13,363,367-9, 377, 382 bilinear/trilinear interpolation 345 blind users 53, 70-3, 75 cartographic representation 111, 210, 224, 295, 313-14, 322, 333,496, 517, 694 cartography 3-8, 10-11, 15, 27, 127, 130, 136-7, 148, 152,201,205,215-16,257, 266, 269, 274, 278, 295, 300, 304-6, 518, 523, 553-4, 563, 578, 611,645, 667, 670, 674, 678, 680-1,683-5, 693-4, 702 challenges 3, 5-12, 14-15, 18-19, 45, 53, 55, 60, 83, 94-5, 104, 106-7, 127-8, 138, 147, 159, 163, 166, 170, 174-5, 306, 425, 427, 436-8, 440, 445-9, 454, 457, 463, 489-91,515-16, 525-6, 538, 559, 607, 646-7, 668-9, 694-6, 701-2 cognitive modeling 567, 572 cognitive science 10, 15, 104, 562-3, 577, 580, 661,693, 700, 702 cognitive style 363, 375-6, 381-3 collaboration 3, 6, 9-11, 14-15, 75, 96, 104, 106, 115, 120-1,183, 280, 285-7, 309, 342, 360, 431-3,436, 438, 445-9, 451, 454-5, 457, 463-8, 470-3, 477, 506, 508-9, 591-2, 596-602, 604-7, 695, 697, 699, 700, 702 705
706 collaborative environment 223-6, 230, 233, 235,240, 433,448, 464, 473,495,498, 602 collaborative geovisualization 8, 285-6, 439, 445, 455, 463-4, 472, 497-500, 509, 591-3, 595-8, 605-7, 695 collaborative visualization 14, 425, 432-3, 446-7, 449-50, 457, 463-4, 466, 471 - 3, 596, 598 collapsed real-time 181, 191 comprehension of visual depictions 243 computer graphics 5, 7, 11, 44, 59, 127, 283, 295, 299, 304, 307, 309, 326, 335, 337, 340, 394, 397, 678 computer supported cooperative work 392, 425, 445, 463-4, 592, 695 contour 213,256, 345-8,351-2, 355-6, 365, 374, 382, 406, 409-10, 416, 471,506 coordination 9, 94, 115, 119, 159, 166-72, 174-5, 230, 285, 398, 447, 456-7, 649, 652-3,700 correctness 91,345, 348-53, 355, 357, 641 cost of navigation 567-8, 574 coupling 159, 165, 170-1, 174-5,647-8, 652, 656, 661
data 3-14, 23-45, 53-75, 83-95, 103-22, 127-39, 143-55, 159-75, 181-97, 201-19, 223-40, 243-60, 265-87, 295-309, 313-21,325-40, 345-60, 364-83,389-92, 396, 403-19, 425-40, 447-57, 463-73, 477-91,495-509, 513-26, 530-42, 555-63,567,577-87, 597-604, 611-22, 627-36, 639-42, 645-63, 667-86, 694-702 data-centred geovisualization 495 datamining 5, 10, 14, 23-45, 54, 104-5, 116, 138-9, 144, 223-39, 244, 256-9, 429-30, 437-8,448, 477-8, 483, 488-91,499, 509, 516, 627, 629, 698 Delphi method 591-604 depth perception 363, 365-6, 563 diagram layout 143 digital elevation model 313-14 dimension reduction 25, 109, 243, 248-9, 260, 483,486, 489 distributed cognition 432, 456-7, 693, 701
Index distributed computing 425-7, 430, 432, 437, 478 environmental crisis management 425-6 experience 6, 13-14, 18, 68, 73, 84, 87, 90, 93, 108- 21, 128, 182, 189, 201 - 2, 228, 238, 250, 265-87, 299-300, 303, 306-7, 363-83, 391,396, 433, 463, 466-7, 470-1,506, 509, 542-3, 545, 558, 562, 602, 604, 612, 614, 620, 640, 647, 656, 671-2, 696, 702 expertise 3, 4, 9-12, 14, 85, 87, 103-4, 113, 115, 117, 121-2, 139, 226, 232, 245, 247, 285, 287, 384, 425-6, 431-3,439, 470, 557, 570, 577, 591,601,604, 649, 662, 684, 699 exploratory data analysis (EDA) 3-4, 11-12, 116, 127-39, 193,645, 648 exploratory visualization 87, 159-75, 304, 426 factorial experiment design 645 flexibility 24, 94, 113-14, 118-19, 134, 218, 228, 230, 265-7, 276, 278-9, 281, 284-7, 337, 405, 448, 470, 472, 504-5, 508-9, 526, 581-2, 662 gaming 295,299, 304, 307,309, 315,384, 518 geographic knowledge discovery 15, 611 - 13, 616, 620, 622 geographic metadata 15, 611 - 22 geographic relevance 181 geographic visualization 137, 143, 155, 201, 339, 513,645-6, 622 geovisualization 3-18, 23-45, 57, 59-60, 62, 83-96, 103-21, 127, 137, 139, 143, 145, 147, 155, 159-75, 183, 196, 201-3, 218, 223-39, 245,265-87, 295-309, 322, 325-42, 345-60, 363-84, 389-400, 425-40, 445-57, 463-73, 477-91,495-509, 513-26, 538, 553-63, 567, 577-87, 591-697, 611-13,617, 620, 627, 645-63, 667-8, 680, 685, 693-702 GIScience 3-7, 9, 11-12, 15, 83-95, 116, 160, 197, 448,456-7, 563,614, 640, 667-8, 678, 684, 687, 693-4, 700, 702 graph drawing 12, 28, 117, 143-55
Index Grid 14, 27, 131,235,325,337, 345-53,403, 425-37, 448, 472, 477-91,505-6, 508, 510, 615, 629, 634, 680 Grid computing 14, 425, 430-1,437, 477, 491,510 group work 6-7, 437, 439-40, 445,457, 463, 510, 694, 698
help 15, 18, 23, 27, 53-73, 83, 86-7, 93, 104, 111-12, 117, 133, 137, 144, 163-5, 175, 187, 189, 211-12, 216-17, 225-7, 229, 233,235,245,254-5,257,259, 266, 278, 300, 357,403,426,456-7,473,529,534, 546-7,554, 559, 562-3,570-1,577-8, 606, 613,628, 647, 651,673,681,687, 693,697 high interaction user interfaces 127 human-computer interaction 12, 53, 63, 75, 104, 116, 201,429, 445-6, 483, 553-4, 593, 645-6, 667-8 ideation 3, 10, 12, 60, 94, 103-5, 107, 109-11,114, 121-2, 265-87, 491,563, 578 immersive 5, 298, 303-4, 363-4, 371,374, 389, 391,396, 439, 448 inference 14, 83-95, 182-224, 226, 235,355, 429, 579 information maps 243, 259 information space 16, 150, 257, 508-9, 570, 627-8, 630-1,641,661-2, 667-8, 670-3, 675, 677-80 information visualization 3-15, 23-45, 53-75, 95, 104, 110, 112, 115-16, 127, 136, 143, 151,169, 186, 201,248, 366, 383, 445, 451,477, 483,485, 488, 554, 563,579, 612, 631,645,667-70, 674-5, 678-81,684, 686-7, 693-4, 696-7, 702 instruments 3, 10, 12, 25, 60, 94- 5, 103-14, 117, 120-1,129, 139, 218, 265-8, 276, 278, 280-1,284-7, 427, 431,491,556, 563, 638, 640, 694, 698 intellectual design 265-7, 271,274, 276, 279, 280, 284
707 interaction 4, 6-7, 12-13, 15, 26-7, 38-9, 44, 53, 57, 60, 63, 75, 87, 104-18, 120, 127, 130-1, 137-9, 166-7, 173, 184, 201,203, 224-5, 229-30, 233, 237-8, 240, 243-4, 246-7, 265-9, 271, 276-81,283-7, 295, 302-3, 307, 318, 339, 355-6, 371,383-4, 390, 392, 394-5, 398, 403,405-6, 409, 418, 428-9, 432, 437-8,445-7, 451-2, 456, 483,489-91,499, 509, 516, 522, 537, 553-4, 557-8, 561-2, 572, 577-82, 585-6, 591-3,595-7,600, 602, 604-6, 613, 617, 621,630-1,642, 645-6, 648, 651-4, 656-9, 661,663, 667-8, 694, 696, 702 interactive 4-5, 13, 18, 26-7, 34, 39, 41-2, 44, 53-4, 57, 63, 65, 68, 75, 87, 95, 104, 107-9, 112-13, 116, 127-39, 153, 160-75, 181-97, 201,203, 206, 210, 212, 216, 218, 224, 240, 245, 250, 254, 259, 265-86, 295-9, 315, 326-42, 345-60, 510, 520, 538, 556, 559, 561, 563,569, 577-82, 587,593-4, 611, 613, 615, 617-18, 621,629, 635, 642, 646, 652-3, 656, 667, 677, 685, 695 interactive data displays 201 interfaces 3- 5, 9, 13-15, 18, 53- 75, 86, 104, 114, 119-20, 127, 136, 173, 232, 246, 257-9, 266-9, 276-7, 280-1,287, 325, 327, 337, 339, 341,355, 391,403, 437, 439, 447, 468, 470, 480, 482, 489, 495, 506, 516, 521,524, 548, 554, 563, 567-74, 577-9, 585-6, 591-2, 595, 604, 657, 698 interoperability 12, 94-5, 103-4, 113, 117-19, 121,139, 232, 268, 279, 285, 287, 427, 429, 436, 497-8, 508, 698-701 isosurface 164, 345-60, 408, 415-16 journey planning 513, 521-2, 524, 526 knowledge discovery 13, 104, 116, 138, 223-4, 240, 259, 428-9, 438-9, 490, 495-6, 499-500, 502, 516, 522, 554-5, 627-30, 667-70
708
Index
large data sets 7, 13, 23-4, 39, 44, 109, 127-39, 189, 225, 346, 448, 457, 489-90 level-of-detail 327-8, 389, 394, 677-8 linking 5, 27-44, 60, 85-7, 118, 130-1, 159-75, 181,214, 218, 225, 271,326, 394, 426, 433-4, 472, 483,518-19, 652, 697 maps 3, 4, 6-7, 15-16, 18, 26-45, 54-75, 83, 105-13, 131-6, 143-4, 161,201, 216, 223-4, 238, 243-59, 267-76, 300, 305-6, 313, 325-41,345, 355, 363-80, 398, 408, 410, 415-16, 428, 452-7, 486-90, 502, 513-21,529-47,563, 568-75, 577-82, 612, 627-38, 652, 667, 670, 678, 686, 693-702 Marching Cubes 345, 349 meteorology 403, 432 mipmapping 59, 313, 319, 322 mobile devices 6, 106, 181-2, 325,432-3, 437, 440, 518,523-26, 529-31,537-8, 544, 548, 693,698 mobile telecommunication 181-2, 523 mobile trajectory 181, 183, 185, 187, 189, 191 modeling 16, 34, 84, 86, 88, 90, 92, 95, 327, 403, 405, 413, 446, 448, 554, 563,567, 572, 667, 686 multi-agent systems 223-4, 228, 258 multidimensional analysis 495 multi-disciplinary 3, 8-9, 11, 14, 557, 593, 695, 702 multimodal presentation 536, 546 multiple linked views 12, 114, 117, 159, 166, 611-12 multiple views 105, 114, 145, 159, 161-2, 165-6, 169-70, 173-4, 437, 517, 520, 563, 567, 574, 618, 630, 642 multi-scale 25, 313, 315, 317, 685 multi-variate visualization 15, 43, 611-13, 615-16, 618, 621,638 ontologies 118-20, 425,428-9, 435-6, 452, 454, 478, 481-2, 490, 495, 498-500, 502-4, 509, 615 positional information 529-31,545-8 public transport 14, 151,513-17, 520-6
reactive agent architecture 223 real-time 3, 13-14, 59, 74, 108, 181,185, 191,269, 283,299, 303-4, 307, 313, 322, 325-7,333,335,337,339-42, 389, 392, 395,398-9, 405,430, 438-9, 489-90, 498, 509, 514-15, 520-1, 523-5,597, 599, 602, 605,648,652, 697 real-time rendering 299, 325, 333, 335 reasoning 12, 83, 91, 93, 95, 129, 202, 223, 225-8, 233, 235, 258-9, 429, 436, 452, 454, 483, 496, 499, 508, 668, 699 representation 5-7, 15, 28, 41, 44, 53-4, 57, 75, 87, 106-9, 113, 115-17, 128, 137, 139, 145, 147, 152, 164-7, 173, 181, 184-5, 201-3, 235, 243-5,247, 249, 253-4, 256, 260, 268-9, 271,274, 277-8, 295-303, 306, 309, 313-15, 318-19, 322, 326, 328,330, 333-4, 337, 339, 345-8, 352, 355-8, 360, 364-5, 371-2, 382, 384, 389-90, 392-3, 395-400, 403, 405-6, 418, 429, 434, 436-8, 445, 448, 451-4, 456, 473,488, 496, 499, 501,513-23,525-6, 530, 532, 535,538-9, 541,554, 562, 568,578-83, 587, 615, 617-18, 627-30, 632, 634-5, 638-42, 663, 668, 674, 680-1,693-4, 696-7, 700 research agenda 3, 5-6, 9-10, 342, 393, 425-6, 435-6, 438-40, 445, 451, 463-4, 495, 530, 515-17, 525-6, 591, 606, 613, 701-2 robustness 230, 345, 357 route instructions 14, 529-37, 541,547-8 schemata 13,298-9, 364, 577, 579, 582, 672 science 3- 5, 7-1 O, 12, 14-15, 23, 44, 75, 83- 96, 104- 20, 160, 181, 197,202, 265, 295, 304-5, 313, 325, 345, 360, 384, 389-90, 396, 403, 421,426-40, 446-57, 464-5, 496, 529, 553-4, 556, 562-3, 571,575, 577, 580, 587, 591-2, 607, 611,627,661,683-4, 693-702 scripting 265, 277, 280-1,283, 287-8, 648 selection and linked highlighting 127, 130 self-organizing map 627-9, 678 semantic knowledge 429, 495-6, 499, 503-4, 509
Index Semantic Web 14, 425, 429, 435, 477-8, 481-2, 489-91,499 semantics 14, 25, 83, 118, 145, 372, 495, 500, 611,669 signature exploration 13, 243-8, 250, 255, 257, 259-60 simulation 13, 45, 53, 103, 247, 252-3, 304, 336, 371,390, 395, 403-4, 428, 430-4, 437-9, 448, 464, 468,470, 497, 524, 571,594 software 5, 12-13, 26, 28, 30, 57, 59, 67-8, 95-6, 103-7, 109, 113-15, 117, 120-1, 127-8, 130, 134-6, 145, 167, 185, 187, 191,201-2, 210, 212, 216, 218, 229, 243, 258, 265,267-9, 271-2, 274, 276, 278-81,283, 285-8, 305-6, 309, 325-342, 360, 371,389, 393-4, 396, 416, 427,429-31,434, 447-8,464, 466, 470, 472-3, 477, 479, 482, 504, 508-9, 514, 516, 553-8, 561-2, 592-4, 596-602, 606-7, 614, 628, 647, 651-2, 656, 661 space perception 567 spatial behaviour 181-2 spatial cognition 8, 299, 663, 667 spatial history explorer 181, 183 spatialization 16, 18, 109, 424, 485, 579, 630, 636, 667-71,673-5, 677-8, 680-1, 686-7 spatio-temporal data 5, 14, 60, 111, 182, 195, 201-7, 216-18, 234-5, 240, 337, 508, 581-2, 642 spatio-temporal query 181, 185, 190, 196 spotlights 181, 183 statistical thinking 116, 128 task 6, 8, 12, 13, 15, 18, 30, 32, 44, 53, 55, 57, 59, 60, 62, 67, 68, 71, 74, 83-5, 90, 91, 93-4, 103-14, 116-18, 121-2, 130, 143, 164-6, 171,173-5, 201-7, 213, 215, 218, 223-4, 228-30, 234, 237-8, 244, 257, 266-9, 276, 278-80, 283-4, 286, 368, 370-1,373, 378, 380-3, 391, 393, 406, 418, 420, 427-8, 438, 440, 451,454-6, 465, 478-80, 483,490, 498-500, 508-10, 513-16, 520, 522-24, 526, 530, 534-5,537,542, 544,
709 546, 553-62, 567-8, 571-2, 574-5, 578, 581-2, 584-5, 591-5, 595-602, 604, 606, 612-14, 618, 620, 624, 628-31,640-2, 645-48, 650-54, 656-63, 672, 679, 684, 686 task typology 645, 653, 661 techniques 257, 300, 306-7, 309, 313-14, 327-8, 330, 333, 337, 339-41,403, 406, 408-9, 418, 428, 434, 437, 680, 684 technology 6-8, 12-13, 23, 25, 44, 45, 53, 62, 70, 72, 103-5, 107-8, 112, 115, 120-122, 153, 173, 182, 184, 229, 233, 258, 266, 280, 285, 295-6, 299, 304-7, 309, 322, 326, 340-1,345-6, 360, 363, 371-2, 391,400, 432, 437-40, 445-47, 455, 472, 490, 496, 498, 513, 523, 553, 556, 597-8, 602, 604, 611,648, 667, 694, 698, 699 terrain 257,300, 306-7,309,~ 313-14, 327-8, 330, 333,337,339-41,403,406,408-9, 418,428, 434, 473,680, 684 texturing 313, 325-6, 328, 330, 333,339-42 time geography 181, 184, 191, 195 tool design 13, 15, 104, 110-11, 114, 202, 206, 245,553-55,557,558,577-8,591, 593,596, 598, 600, 606-7, 647, 661,663 tools 4-8, 12-15, 30, 39, 44-5, 60, 68-9, 83-7, 91, 93-5, 103-6, 108, 110-21, 129-31, 135-37, 139, 159, 161, 165, 167, 175, 201-3, 206-7, 210, 213, 216-19, 223-4, 238, 240, 243, 245-6, 249, 255-9, 265-8, 271,274, 276, 278-80, 283-7, 299, 304, 306, 393-4, 418, 425,428-30, 437-39, 445,447-8, 451,454, 456-7, 463-4, 470, 479, 489, 490, 496-7, 499, 506, 509, 515, 520, 529, 553-6, 559-60, 563, 577-8, 580-3, 587, 591-602, 604-7, 620-1, 628, 640, 642, 645-650, 652-3, 656, 661,663, 667, 684-5, 693-6, 698-701 transport information systems 14, 433, 513-16, 520, 522, 524-5 transport maps 513 true 3D 13,298-9, 306, 363-78, 385-88, 485
710
universal usability 6, 53, 55, 60, 62, 69, 75 usability 5-6, 8, 12-13, 15, 53, 55, 60, 62-3, 65, 69, 75, 119, 173, 230, 240, 306, 340-1,400, 437, 439, 516, 524-6, 553-6, 558-61,563, 577, 580, 584, 591-9, 602, 604-7, 611-12, 620-1, 627, 629-31,638, 640-2, 645-9, 651-4, 668-9, 687, 694, 698-701 usefulness 24, 33, 165, 259, 363, 372, 553, 555, 561,593, 631,640-2, 646, 652, 661,668, 687 user analysis 363, 557 user interfaces 53-4, 62, 127, 268, 329, 339, 341,391,470, 495, 548, 554, 574, 591-2, 595 user-centered design 15, 553, 555-9, 591, 593, 596, 699 users 4, 6, 8, 12, 14-15, 26, 34, 39, 53-5, 57, 59, 60, 62-3, 65, 67-75, 95-6, 103-4, 106, 112-14, 117, 121-2, 143, 150, 152-3, 181-2, 185, 189, 196, 202-3, 206, 208, 217, 224-6, 228-9, 233-7, 240, 243-7, 250, 255-60, 266, 272, 276, 278, 281,283-7, 299, 303, 306-7, 309, 315, 337, 363, 373-77, 380-4, 392, 395, 398, 400, 404, 420, 427-8, 430, 439, 447-8, 451,454-5, 457, 464, 466, 468, 470-2, 477, 479-81,488, 495-7, 499, 500, 504-6, 508-9, 514-15, 518, 521-26, 530-3, 536-7, 542-5, 553-61,572, 575, 577-81,584-7, 591-8, 601-2, 604, 611 - 15, 617-18, 620-2, 629, 638, 640-2, 645-56, 659-63, 677, 681, 693- 5, 697- 9 verisimilar 389, 396-8, 400 video conferencing 432-3, 439, 448, 464, 473, 591, 596- 9, 602, 604 virtual environments 3-5, 9, 10, 13-14, 73, 122, 288, 295, 304, 337, 365, 384, 389-91,391-6, 398, 400, 425-7, 445, 448, 495, 613
Index
virtual reality 171,240, 295, 302, 325, 327, 330, 448, 571,646, 648 visual data exploration 23-5, 30, 44, 201, 646-8 visual datamining 5, 14, 23-5, 33-4, 54, 437, 477-8, 483, 488-91 visual support 83, 91 visualization 3-16, 18, 23-8, 30-6, 38-9, 41-2, 44-5, 53-5, 57, 59-60, 62-3, 65, 68-9, 74-5, 83-7, 91, 93-6, 103-10, 112-22, 127, 130-1, 136-9, 143-5, 147, 151,155, 159-67, 169-72, 174-5, 181,183, 185, 190, 193, 195-6, 201-3, 207, 210, 212-13, 215-18, 223-26, 229, 233-4, 236, 238-40, 243-48, 250-51,254, 256-9, 265-9, 271-2, 274, 276-81,283-8, 295-6, 298-300, 302-7, 309, 313-15,318-19, 322, 325-28, 330, 333, 335, 337, 339-42, 345-6, 355, 358, 360, 363-9, 371-4, 383-4, 389-94, 398, 400, 403-6, 409-10, 413, 416, 418, 420-1, 425-8, 430-4, 436-40, 445-9, 451, 454-7, 463-7, 471-3, 477-8, 482-3, 485-9, 491,495-500, 502-6, 508-9, 513-18, 520-6, 529, 535-8, 545, 546-8, 553-63, 567, 577, 579-81, 591-2, 595-6, 598, 601,604, 611-18, 620-2, 627-32, 634, 638, 640-2, 645-8, 652-3, 656, 661-2, 667-70, 674-5, 678-81,684, 686-7 visualization pipeline 13,295-6, 298-9, 303, 496, 508, 538 within-subjects ANOVA 645 workshop 3-16, 122, 139, 445-6, 448, 695-6, 702 World Wide Web 28, 244, 368, 403, 435-6, 495-6, 504, 509, 524 zooming 26, 41, 59, 63, 65, 68, 131-3, 165, 208, 266, 269, 278, 314, 346, 483, 508-9, 567-8, 572-5, 635, 677.
This Page Intentionally Left Blank
Can you judge a book by its cover ... ? Sophisticated interactive maps are increasingly used to explore information - guiding us through data landscapes to provide inforn~tion and prompt insight and understanding. Geovisualization is an emerging domain tl~t draws upon disciplines such as computer science, human-computer interaction design, cognitive sciences, graphical statistics, data visualization, information visualization, geographic information science and cartography to discuss, develop and evaluate interactive cartography. This review and exploration of the current and future status of geovisualization has been produced by key researchers and practitioners from around the world in various cognate fields of study. The thirty-six chapters present summaries of work undertaken, case studies focused on new methods and their application, system descriptions, tests of their implementation, plans for collaboration and reflections on experiences of using and developing geovisualization techniques. Sections of the book focus on software approaches to creating geovisualization tools; using 3d in geovisualization; distributed geovisualization; design and evaluation issues that make geovisualization useful and usable. Each is preceded by a joint perspective, co-authored by contributors to the section.A context:setting introduction features key authors in information visualization, human-computer interaction design and GI science. In total, over 50 pages of colour are provided in the book along with more than 250 colour images in the associated digital appendices (provided on CD). The increasing importance and use of spatial information and the map metaphor establishes geovisualization as an essential element of 21st century information use, a genuine opportunity for 21st century cartography and a requirement for modern map users. The chapters are mapped on the cover of this book using spatialization techniques that are described and explored within.Whilst you may not be able to judge the book from its cover, these maps provide some insight into the scope of the book and the relationships between the chapters.They demonstrate some of the techniques and solutions that are developing in this exciting field and documented inside.
ISBN
ELSEVIER
elsevierocom
~)
0 08
044531
080 4 5
4
