Handbook of Data Visualization
Chun-houh Chen Wolfgang Härdle Antony Unwin Editors
Handbook of Data Visualization With Figures and Tables
123
Editors Dr. Chun-houh Chen Institute of Statistical Science Academia Sinica Academia Road, Section Taipei Taiwan
[email protected]
Professor Wolfgang Härdle CASE – Center for Applied Statistics and Economics School of Business and Economics Humboldt-Universität zu Berlin Spandauer Straße Berlin Germany
[email protected]
Professor Antony Unwin Mathematics Institute University of Augsburg Augsburg Germany
[email protected]
ISBN ----
e-ISBN ----
DOI ./---- Library of Congress Control Number: © Springer-Verlag Berlin Heidelberg his work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September , , in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. he use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting and Production: LE-TEX Jelonek, Schmidt & Vöckler GbR, Leipzig, Germany Cover: deblik, Berlin, Germany Printed on acid-free paper springer.com
Table of Contents I. Data Visualization I.1 Introduction Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle . . . . . . . . . . . . . . . . . . . . . . . . . . 3
II. Principles II.1 A Brief History of Data Visualization Michael Friendly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 II.2 Good Graphics? Antony Unwin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 II.3 Static Graphics Paul Murrell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79 II.4 Data Visualization Through Their Graph Representations George Michailidis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 II.5 Graph-theoretic Graphics Leland Wilkinson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 II.6 High-dimensional Data Visualization Martin heus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 II.7 Multivariate Data Glyphs: Principles and Practice Matthew O. Ward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 II.8 Linked Views for Visual Exploration Adalbert Wilhelm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 II.9 Linked Data Views Graham Wills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 II.10 Visualizing Trees and Forests Simon Urbanek . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
VI
Table of Contents
III. Methodologies III.1 Interactive Linked Micromap Plots for the Display of Geographically Referenced Statistical Data Jürgen Symanzik, Daniel B. Carr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 III.2 Grand Tours, Projection Pursuit Guided Tours, and Manual Controls Dianne Cook, Andreas Buja, Eun-Kyung Lee, Hadley Wickham . . . . . . . . . . . . . . . . 295 III.3 Multidimensional Scaling Michael A.A. Cox, Trevor F. Cox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 III.4 Huge Multidimensional Data Visualization: Back to the Virtue of Principal Coordinates and Dendrograms in the New Computer Age Francesco Palumbo, Domenico Vistocco, Alain Morineau . . . . . . . . . . . . . . . . . . . . . . 349 III.5 Multivariate Visualization by Density Estimation Michael C. Minnotte, Stephan R. Sain, David W. Scott . . . . . . . . . . . . . . . . . . . . . . . . 389 III.6 Structured Sets of Graphs Richard M. Heiberger, Burt Holland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 III.7 Regression by Parts: Fitting Visually Interpretable Models with GUIDE Wei-Yin Loh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 III.8 Structural Adaptive Smoothing by Propagation–Separation Methods Jörg Polzehl, Vladimir Spokoiny . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 III.9 Smoothing Techniques for Visualisation Adrian W. Bowman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 III.10 Data Visualization via Kernel Machines Yuan-chin Ivan Chang, Yuh-Jye Lee, Hsing-Kuo Pao, Mei-Hsien Lee, Su-Yun Huang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539 III.11 Visualizing Cluster Analysis and Finite Mixture Models Friedrich Leisch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 III.12 Visualizing Contingency Tables David Meyer, Achim Zeileis, Kurt Hornik . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589 III.13 Mosaic Plots and Their Variants Heike Hofmann . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 III.14 Parallel Coordinates: Visualization, Exploration and Classiication of High-Dimensional Data Alfred Inselberg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 III.15 Matrix Visualization Han-Ming Wu, ShengLi Tzeng, Chun-Houh Chen . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681 III.16 Visualization in Bayesian Data Analysis Jouni Kerman, Andrew Gelman, Tian Zheng, Yuejing Ding . . . . . . . . . . . . . . . . . . . . 709 III.17 Programming Statistical Data Visualization in the Java Language Junji Nakano, Yoshikazu Yamamoto, Keisuke Honda . . . . . . . . . . . . . . . . . . . . . . . . . 725 III.18 Web-Based Statistical Graphics using XML Technologies Yoshiro Yamamoto, Masaya Iizuka, Tomokazu Fujino . . . . . . . . . . . . . . . . . . . . . . . . 757
Table of Contents VII
IV. Selected Applications IV.1 Visualization for Genetic Network Reconstruction Grace S. Shieh, Chin-Yuan Guo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793 IV.2 Reconstruction, Visualization and Analysis of Medical Images Henry Horng-Shing Lu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813 IV.3 Exploratory Graphics of a Financial Dataset Antony Unwin, Martin heus, Wolfgang K. Härdle . . . . . . . . . . . . . . . . . . . . . . . . . . . 831 IV.4 Graphical Data Representation in Bankruptcy Analysis Wolfgang K. Härdle, Rouslan A. Moro, Dorothea Schäfer . . . . . . . . . . . . . . . . . . . . . . 853 IV.5 Visualizing Functional Data with an Application to eBay’s Online Auctions Wolfgang Jank, Galit Shmueli, Catherine Plaisant, Ben Shneiderman . . . . . . . . . . . 873 IV.6 Visualization Tools for Insurance Risk Processes Krzysztof Burnecki, Rafał Weron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899
List of Contributors Adrian W. Bowman University of Glasgow Department of Statistics UK
[email protected]
Chun-houh Chen Academia Sinica Institute of Statistical Science Taiwan
[email protected]
Andreas Buja University of Pennsylvania Statistics Department USA
[email protected]
Dianne Cook Iowa State University Department of Statistics USA
[email protected]
Krzysztof Burnecki Wroclaw University of Technology Institute of Mathematics and Computer Science Poland
[email protected]
Michael A. A. Cox University of Newcastle Upon Tyne Division of Psychology School of Biology and Psychology UK
[email protected]
Daniel B. Carr George Mason University Center for Computational Statistics USA
[email protected]
Trevor F. Cox Data Sciences Unit Unilever R&D Port Sunlight UK
[email protected]
Yuan-chin Ivan Chang Academia Sinica Institute of Statistical Science Taiwan
[email protected]
Yuejing Ding Columbia University Department of Statistics USA
[email protected]
X
List of Contributors
Michael Friendly York University Psychology Department Canada
[email protected] Tomokazu Fujino Fukuoka Women’s University Department of Environmental Science Japan
[email protected] Andrew Gelman Columbia University Department of Statistics USA
[email protected]
Burt Holland Temple University Department of Statistics USA
[email protected] Keisuke Honda Graduate University for Advanced Studies Japan
[email protected] Kurt Hornik Wirtschatsuniversität Wien Department of Statistics and Mathematics Austria
[email protected]
Chin-Yuan Guo Academia Sinica Institute of Statistical Science Taiwan
Su-Yun Huang Academia Sinica Institute of Statistical Science Taiwan
[email protected]
Wolfgang K. Härdle Humboldt-Universität zu Berlin CASE – Center for Applied Statistics and Economics Germany
[email protected]
Masaya Iizuka Okayama University Graduate School of Natural Science and Technology Japan
[email protected]
Richard M. Heiberger Temple University Department of Statistics USA
[email protected] Heike Hofmann Iowa State University Department of Statistics USA
[email protected]
Alfred Inselberg Tel Aviv University School of Mathematical Sciences Israel
[email protected] Wolfgang Jank University of Maryland Department of Decision and Information Technologies USA
[email protected]
List of Contributors XI
Jouni Kerman Novartis Pharma AG USA
[email protected] Eun-Kyung Lee Seoul National University Department of Statistics Korea
[email protected]
David Meyer Wirtschatsuniversität Wien Department of Information Systems and Operations Austria
[email protected] George Michailidis University of Michigan Department of Statistics USA
[email protected]
Yuh-Jye Lee National Taiwan University of Science and Technology Department of Computer Science and Information Engineering Taiwan
[email protected]
Michael C. Minnotte Utah State University Department of Mathematics and Statistics USA
[email protected]
Mei-Hsien Lee National Taiwan University Institute of Epidemiology Taiwan
Alain Morineau La Revue MODULAD France
[email protected]
Friedrich Leisch Ludwig-Maximilians-Universität Institut für Statistik Germany
[email protected]
Rouslan A. Moro Humboldt-Universität zu Berlin Institut für Statistik und Ökonometrie Germany
[email protected]
Wei-Yin Loh University of Wisconsin-Madison Department of Statistics USA
[email protected] Henry Horng-Shing Lu National Chiao Tung University Institute of Statistics Taiwan
[email protected]
Paul Murrell University of Auckland Department of Statistics New Zealand
[email protected] Junji Nakano he Institute of Statistical Mathematics and the Graduate University for Advanced Studies Japan
[email protected]
XII
List of Contributors
Francesco Palumbo University of Macerata Dipartimento di Istituzioni Economiche e Finanziarie Italy
[email protected] Hsing-Kuo Pao National Taiwan University of Science and Technology Department of Computer Science and Information Engineering Taiwan
[email protected] Catherine Plaisant University of Maryland Department of Computer Science USA
[email protected] Jörg Polzehl Weierstrass Institute for Applied Analysis and Stochastics Germany
[email protected] Stephan R. Sain University of Colorado at Denver Department of Mathematics USA
[email protected]
Grace Shwu-Rong Shieh Academia Sinica Institute of Statistical Science Taiwan
[email protected] Galit Shmueli University of Maryland Department of Decision and Information Technologies USA
[email protected] Ben Shneiderman University of Maryland Department of Computer Science USA
[email protected] Vladimir Spokoiny Weierstrass Institute for Applied Analysis and Stochastics Germany
[email protected] Jürgen Symanzik Utah State University Department of Mathematics and Statistics USA
[email protected]
Dorothea Schäfer Wirtschatsforschung (DIW) Berlin German Institute for Economic Research Germany
[email protected]
Martin Theus University of Augsburg Department of Computational Statistics and Data Analysis Germany
[email protected]
David W. Scott Rice University Division Statistics USA
[email protected]
ShengLi Tzeng Academia Sinica Institute of Statistical Science Taiwan hh@stat.sinica.edu.tw
List of Contributors XIII
Antony Unwin Mathematics Institute University of Augsburg Germany
[email protected] Simon Urbanek AT&T Labs – Research USA
[email protected] Domenico Vistocco University of Cassino Dipartimento di Economia e Territorio Italy
[email protected] Matthew O. Ward Worcester Polytechnic Institute Computer Science Department USA
[email protected] Rafał Weron Wrocław University of Technology Institute of Mathematics and Computer Science Poland
[email protected] Hadley Wickham Iowa State University Department of Statistics USA
[email protected] Adalbert Wilhelm International University Bremen Germany
[email protected]
Leland Wilkinson SYSTAT Sotware Inc. Chicago USA
[email protected] Graham Wills SPSS Inc. Chicago USA
[email protected] Han-Ming Wu Academia Sinica Institute of Statistical Science Taiwan
[email protected] Yoshikazu Yamamoto Tokushima Bunri University Department of Engineering Japan
[email protected] Yoshiro Yamamoto Tokai University Department of Mathematics Japan
[email protected] Achim Zeileis Wirtschatsuniversität Wien Department of Statistics and Mathematics Austria
[email protected] Tian Zheng Columbia University Department of Statistics USA
[email protected]
Part I Data Visualization
Introduction
I.1
Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle
1.1
1.2
1.3
Computational Statistics and Data Visualization .. . . . . . . . . . . . . . . . .. . . . . . . . .
4
Data Visualization and Theory . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Presentation and Exploratory Graphics . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Graphics and Computing . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
4 4 5
The Chapters . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
6
Summary and Overview; Part II .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Summary and Overview; Part III .. .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Summary and Overview; Part IV . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . The Authors .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
7 9 10 11
Outlook .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
12
4
1.1
Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle
Computational Statistics and Data Visualization his book is the third volume of the Handbook of Computational Statistics and covers the field of data visualization. In line with the companion volumes, it contains a collection of chapters by experts in the field to present readers with an up-to-date and comprehensive overview of the state of the art. Data visualization is an active area of application and research, and this is a good time to gather together a summary of current knowledge. Graphic displays are oten very effective at communicating information. hey are also very oten not effective at communicating information. Two important reasons for this state of affairs are that graphics can be produced with a few clicks of the mouse without any thought and the design of graphics is not taken seriously in many scientific textbooks. Some people seem to think that preparing good graphics is just a matter of common sense (in which case their common sense cannot be in good shape), while others believe that preparing graphics is a low-level task, not appropriate for scientific attention. his volume of the Handbook of Computational Statistics takes graphics for data visualization seriously.
1.1.1
Data Visualization and Theory Graphics provide an excellent approach for exploring data and are essential for presenting results. Although graphics have been used extensively in statistics for a long time, there is not a substantive body of theory about the topic. Quite a lot of attention has been paid to graphics for presentation, particularly since the superb books of Edward Tute. However, this knowledge is expressed in principles to be followed and not in formal theories. Bertin’s work from the s is oten cited but has not been developed further. his is a curious state of affairs. Graphics are used a great deal in many different fields, and one might expect more progress to have been made along theoretical lines. Sometimes in science the theoretical literature for a subject is considerable while there is little applied literature to be found. he literature on data visualization is very much the opposite. Examples abound in almost every issue of every scientific journal concerned with quantitative analysis. here are occasionally articles published in a more theoretical vein about specific graphical forms, but little else. Although there is a respected statistics journal called the Journal of Computational and Graphical Statistics, most of the papers submitted there are in computational statistics. Perhaps this is because it is easier to publish a study of a technical computational problem than it is to publish work on improving a graphic display.
1.1.2
Presentation and Exploratory Graphics he differences between graphics for presentation and graphics for exploration lie in both form and practice. Presentation graphics are generally static, and a single
Introduction 5
Figure .. A barchart of the number of authors per paper, a histogram of the number of pages per
paper, and parallel boxplots of length by number of authors. Papers with more than three authors have been selected
graphic is drawn to summarize the information to be presented. hese displays should be of high quality and include complete definitions and explanations of the variables shown and of the form of the graphic. Presentation graphics are like proofs of mathematical theorems; they may give no hint as to how a result was reached, but they should offer convincing support for its conclusion. Exploratory graphics, on the other hand, are used for looking for results. Very many of them may be used, and they should be fast and informative rather than slow and precise. hey are not intended for presentation, so that detailed legends and captions are unnecessary. One presentation graphic will be drawn for viewing by potentially thousands of readers while thousands of exploratory graphics may be drawn to support the data investigations of one analyst. Books on visualization should make use of graphics. Figure . shows some simple summaries of data about the chapters in this volume, revealing that over half the chapters had more than one author and that more authors does not always mean longer papers.
Graphics and Computing Developments in computing power have been of great benefit to graphics in recent years. It has become possible to draw precise, complex displays with great ease and to print them with impressive quality at high resolution. hat was not always the case, and initially computers were more a disadvantage for graphics. Computing screens and printers could at best produce clumsy line-driven displays of low resolution without colour. hese offered no competition to careful, hand-drawn displays. Furthermore, even early computers made many calculations much easier than before and allowed fitting of more complicated models. his directed attention away from graphics, and it is only in the last years that graphics have come into their own again.
1.1.3
6
Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle
hese comments relate to presentation graphics, that is, graphics drawn for the purpose of illustrating and explaining results. Computing advances have benefitted exploratory graphics, that is, graphics drawn to support exploring data, far more. Not just the quality of graphic representation has improved but also the quantity. It is now trivial to draw many different displays of the same data or to riffle through many different versions interactively to look for information in data. hese capabilities are only gradually becoming appreciated and capitalized on. he importance of sotware availability and popularity in determining what analyses are carried out and how they are presented will be an interesting research topic for future historians of science. In the business world, no one seems to be able to do without the spreadsheet Excel. If Excel does not offer a particular graphic form, then that form will not be used. (In fact Excel offers many graphic forms, though not all that a statistician would want.) Many scientists, who only rarely need access to computational power, also rely on Excel and its options. In the world of statistics itself, the packages SAS and SPSS were long dominant. In the last years, first S and S-plus and now R have emerged as important competitors. None of these packages currently provide effective interactive tools for exploratory graphics, though they are all moving slowly in that direction as well as extending the range and flexibility of the presentation graphics they offer. Data visualization is a new term. It expresses the idea that it involves more than just representing data in a graphical form (instead of using a table). he information behind the data should also be revealed in a good display; the graphic should aid readers or viewers in seeing the structure in the data. he term data visualization is related to the new field of information visualization. his includes visualization of all kinds of information, not just of data, and is closely associated with research by computer scientists. Up till now the work in this area has tended to concentrate just on presenting information, rather than on what may be deduced from it. Statisticians tend to be concerned more with variability and to emphasize the statistical properties of results. he closer linking of graphics with statistical modelling can make this more explicit and is a promising research direction that is facilitated by the flexible nature of current computing sotware. Statisticians have an important role to play here.
1.2
The Chapters Needless to say, each Handbook chapter uses a lot of graphic displays. Figure . is a scatterplot of the number of figures against the number of pages. here is an approximate linear relationship with a couple of papers having somewhat more figures per page and one somewhat less. he scales have been chosen to maximize the dataink ratio. An alternative version with equal scales makes clearer that the number of figures per page is almost always less than one. he Handbook has been divided into three sections: Principles, Methodology, and Applications. Needless to say, the sections overlap. Figure . is a binary matrix visualization using Jaccard coefficients for both chapters (rows) and index entries
Introduction 7
Figure .. A scatterplot of the number of figures against the number of pages for the Handbook’s
chapters
(columns) to explore links between chapters. In the raw data map (lower-let portion of Fig. .) there is a banding of black dots from the lower-let to upper-right corners indicating a possible transition of chapter/index combinations. In the proximity map of indices (upper portion of Fig. .), index groups A, B, C, D, and E are overlapped with each other and are dominated by chapters of Good Graphics, History, Functional Data, Matrix Visualization, and Regression by Parts respectively.
Summary and Overview; Part II he ten chapters in Part II are concerned with principles of data visualization. First there is an historical overview by Michael Friendly, the custodian of the Internet Gallery of Data Visualization, outlining the developments in graphical displays over the last few hundred years and including many fine examples. In the next chapter Antony Unwin discusses some of the guidelines for the preparation of sound and attractive data graphics. he question mark in the chapter title sums it up well: whatever principles or recommendations are followed, the success of a graphic is a matter of taste; there are no fixed rules. he importance of sotware for producing graphics is incontrovertible. Paul Murrell in his chapter summarizes the requirements for producing accurate and exact static graphics. He emphasizes both the need for flexibility in customizing standard plots and the need for tools that permit the drawing of new plot types. Structure in data may be represented by mathematical graphs. George Michailidis pursues this idea in his chapter and shows how this leads to another class of graphic displays associated with multivariate analysis methods.
1.2.1
8
Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle
Figure .. Matrix visualizations of the Handbook with chapters in the rows and index entries in the
columns
Lee Wilkinson approaches graph-theoretic visualizations from another point of view, and his displays are concerned predominantly, though by no means exclusively, with trees, directed graphs and geometric graphs. He also covers the layout of graphs, a tricky problem for large numbers of vertices, and raises the intriguing issue of graph matching. Most data displays concentrate on one or two dimensions. his is frequently sufficient to reveal striking information about a dataset. To gain insight into multivariate structure, higher-dimensional representations are required. Martin heus discusses the main statistical graphics of this kind that do not involve dimension reduction and compares their possible range of application. Everyone knows about Chernoff faces, though not many ever use them. he potential of data glyphs for representing cases in informative and productive ways has not been fully realized. Matt Ward gives an overview of the wide variety of possible forms and of the different ways they can be utilized.
Introduction 9
here are two chapters on linking. Adalbert Wilhelm describes a formal model for linked graphics and the conceptual structure underlying it. He is able to encompass different types of linking and different representations. Graham Wills looks at linking in a more applied context and stresses the importance of distinguishing between views of individual cases and aggregated views. He also highlights the variety of selection possibilities there are in interactive graphics. Both chapters point out the value of linking simple data views over linking complicated ones. he final chapter in this section is by Simon Urbanek. He describes the graphics that have been introduced to support tree models in statistics. he close association between graphics and the models (and collections of models in forests) is particularly interesting and has relevance for building closer links between graphics and models in other fields.
Summary and Overview; Part III he middle and largest section of the Handbook concentrates on individual area of graphics research. Geographical data can obviously benefit from visualization. Much of Bertin’s work was directed at this kind of data. Juergen Symanzik and Daniel Carr write about micromaps (multiple small images of the same area displaying different parts of the data) and their interactive extension. Projection pursuit and the grand tour are well known but not easy to use. Despite the availability of attractive free sotware, it is still a difficult task to analyse datasets in depth with this approach. Dianne Cook, Andreas Buja, Eun-Kyung Lee and Hadley Wickham describe the issues involved and outline some of the progress that has been made. Multidimensional scaling has been around for a long time. Michael Cox and Trevor Cox (no relation, but an MDS would doubtless place them close together) review the current state of research. Advances in high-throughput techniques in industrial projects, academic studies and biomedical experiments and the increasing power of computers for data collection have inevitably changed the practice of modern data analysis. Real-life datasets become larger and larger in both sample size and numbers of variables. Francesco Palumbo, Alain Morineau and Domenico Vistocco illustrate principles of visualization for such situations. Some areas of statistics benefit more directly from visualization than others. Density estimation is hard to imagine without visualization. Michael Minnotte, Steve Sain and David Scott examine estimation methods in up to three dimensions. Interestingly there has not been much progress with density estimation in even three dimensions. Sets of graphs can be particularly useful for revealing the structure in datasets and complement modelling efforts. Richard Heiberger and Burt Holland describe an approach primarily making use of Cartesian products and the Trellis paradigm. WeiYin Loh describes the use of visualization to support the use of regression models, in particular with the use of regression trees.
1.2.2
10
Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle
Instead of visualizing the structure of samples or variables in a given dataset, researchers may be interested in visualizing images collected with certain formats. Usually the target images are collected with various types of noise pattern and it is necessary to apply statistical or mathematical modelling to remove or diminish the noise structure before the possible genuine images can be visualized. Jörg Polzehl and Vladimir Spokoiny present one such novel adaptive smoothing procedure in reconstructing noisy images for better visualization. he continuing increase in computer power has had many different impacts on statistics. Computationally intensive smoothing methods are now commonplace, although they were impossible only a few years ago. Adrian Bowman gives an overview of the relations between smoothing and visualization. Yuan-chin Chang, Yuh-Jye Lee, Hsing-Kuo Pao, Mei-Hsien Lee and Su-Yun Huang investigate the impact of kernel machine methods on a number of classical techniques: principal components, canonical correlation and cluster analysis. hey use visualizations to compare their results with those from the original methods. Cluster analyses have oten been a bit suspect to statisticians. he lack of formal models in the past and the difficulty of judging the success of the clusterings were both negative factors. Fritz Leisch considers the graphical evaluation of clusterings and some of the possibilities for a sounder methodological approach. Multivariate categorical data were difficult to visualize in the past. he chapter by David Meyer, Achim Zeileis and Kurt Hornik describes fairly classical approaches for low dimensions and emphasizes the link to model building. Heike Hofmann describes the powerful tools of interactive mosaicplots that have become available in recent years, not least through her own efforts, and discusses how different variations of the plot form can be used for gaining insight into multivariate data features. Alfred Inselberg, the original proposer of parallel coordinate plots, offers an overview of this approach to multivariate data in his usual distinctive style. Here he considers in particular classification problems and how parallel coordinate views can be adapted and amended to support this kind of analysis. Most analyses using graphics make use of a standard set of graphical tools, for example, scatterplots, barcharts, and histograms. Han-Ming Wu, ShengLi Tzeng and Chun-houh Chen describe a different approach, built around using colour approximations for individual values in a data matrix and applying cluster analyses to order the matrix rows and columns in informative ways. For many years Bayesians were primarily theoreticians. hanks to MCMC methods they are now able to also apply their ideas to great effect. his has led to new demands in assessing model fit and the quality of the results. Jouni Kerman, Andrew Gelman, Tian Zheng and Yuejing Ding discuss graphical approaches for tackling these issues in a Bayesian framework. Without sotware to draw the displays, graphic analyis is almost impossible nowadays. Junji Nakano, Yamamoto Yoshikazu and Keisuke Honda are working on Javabased sotware to provide support for new developments, and they outline their approach here. Many researchers are interested in providing tools via the Web. Yoshiro Yamamoto, Masaya Iizuka and Tomokazu Fujino discuss using XML for interactive statistical graphics and explain the issues involved.
Introduction 11
Summary and Overview; Part IV
1.2.3
he final section contains seven chapters on specific applications of data visualization. here are, of course, individual applications discussed in earlier chapters, but here the emphasis is on the application rather than principles or methodology. Genetic networks are obviously a promising area for informative graphic displays. Grace Shieh and Chin-Yuan Guo describe some of the progress made so far and make clear the potential for further research. Modern medical imaging systems have made significant contributions to diagnoses and treatments. Henry Lu discusses the visualization of data from positron emission tomography, ultrasound and magnetic resonance. Two chapters examine company bankruptcy datasets. In the first one, Antony Unwin, Martin heus and Wolfgang Härdle use a broad range of visualization tools to carry out an extensive exploratory data analysis. No large dataset can be analysed cold, and this chapter shows how effective data visualization can be in assessing data quality and revealing features of a dataset. he other bankruptcy chapter employs graphics to visualize SVM modelling. Wolfgang Härdle, Rouslan Moro and Dorothea Schäfer use graphics to display results that cannot be presented in a closed analytic form. he astonishing growth of eBay has been one of the big success stories of recent years. Wolfgang Jank, Galit Shmueli, Catherine Plaisant and Ben Shneiderman have studied data from eBay auctions and describe the role graphics played in their analyses. Krzysztof Burnecki and Rafal Weron consider the application of visualization in insurance. his is another example of how the value of graphics lies in providing insight into the output of complex models.
The Authors he editors would like to thank the authors of the chapters for their contributions. It is important for a collective work of this kind to cover a broad range and to gather many experts with different interests together. We have been fortunate in receiving the assistance of so many excellent contributors. he mixture at the end remains, of course, a mixture. Different authors take different approaches and have different styles. It early became apparent that even the term data visualization means different things to different people! We hope that the Handbook gains rather than loses by this eclecticism. Figures . and . earlier in the chapter showed that the chapter form varied between authors in various ways. Figure . reveals another aspect. he scatterplot shows an outlier with a very large number of references (the historical survey of Michael Friendly) and that some papers referenced the work of their own authors more than others. he histogram is for the rate of self-referencing.
1.2.4
12
Antony Unwin, Chun-houh Chen, Wolfgang K. Härdle
Figure .. A scatterplot of the number of references to papers by a chapter’s authors against the total
number of references and a histogram of the rate of self-referencing
1.3
Outlook here are many open issues in data visualization and many challenging research problems. he datasets to be analysed tend to be more complex and are certainly becoming larger all the time. he potential of graphical tools for exploratory data analysis has not been fully realized, and the complementary interplay between statistical modelling and graphics has not yet been fully exploited. Advances in computer sotware and hardware have made producing graphics easier, but they have also contributed to raising the standards expected. Future developments will undoubtedly include more flexible and powerful sotware and better integration of modelling and graphics. here will probably be individual new and innovative graphics and some improvements in the general design of displays. Gradual gains in knowledge about the perception of graphics and the psychological aspects of visualization will lead to improved effectiveness of graphic displays. Ideally there should be progress in the formal theory of data visualization, but that is perhaps the biggest challenge of all.
Part II Principles
A Brief History of Data Visualization
II.1
Michael Friendly
1.1 1.2
1.3
1.4
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . Milestones Tour .. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
17
Pre-17th Century: Early Maps and Diagrams . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1600–1699: Measurement and Theory . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1700–1799: New Graphic Forms . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1800–1850: Beginnings of Modern Graphics . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1850–1900: The Golden Age of Statistical Graphics . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1900–1950: The Modern Dark Ages .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1950–1975: Rebirth of Data Visualization . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . 1975–present: High-D, Interactive and Dynamic Data Visualization . . . .. . .. . . .
17 19 22 25 28 37 39 40
Statistical Historiography . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
42
History as ‘Data’ . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Analysing Milestones Data . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . What Was He Thinking? – Understanding Through Reproduction . . .. . . .. . .. . . .
42 43 45
Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
48
16
16
Michael Friendly
It is common to think of statistical graphics and data visualization as relatively modern developments in statistics. In fact, the graphic representation of quantitative information has deep roots. hese roots reach into the histories of the earliest map making and visual depiction, and later into thematic cartography, statistics and statistical graphics, medicine and other fields. Along the way, developments in technologies (printing, reproduction), mathematical theory and practice, and empirical observation and recording enabled the wider use of graphics and new advances in form and content. his chapter provides an overview of the intellectual history of data visualization from medieval to modern times, describing and illustrating some significant advances along the way. It is based on a project, called the Milestones Project, to collect, catalogue and document in one place the important developments in a wide range of areas and fields that led to modern data visualization. his effort has suggested some questions concerning the use of present-day methods of analysing and understanding this history, which I discuss under the rubric of ‘statistical historiography.’
1.1
Introduction he only new thing in the world is the history you don’t know.
– Harry S Truman
It is common to think of statistical graphics and data visualization as relatively modern developments in statistics. In fact, the graphic portrayal of quantitative information has deep roots. hese roots reach into the histories of the earliest map-making and visual depiction, and later into thematic cartography, statistics and statistical graphics, with applications and innovations in many fields of medicine and science which are oten intertwined with each other. hey also connect with the rise of statistical thinking and widespread data collection for planning and commerce up through the th century. Along the way, a variety of advancements contributed to the widespread use of data visualization today. hese include technologies for drawing and reproducing images, advances in mathematics and statistics, and new developments in data collection, empirical observation and recording. From above ground, we can see the current fruit and anticipate future growth; we must look below to understand their germination. Yet the great variety of roots and nutrients across these domains, which gave rise to the many branches we see today, are oten not well known and have never been assembled in a single garden to be studied or admired. his chapter provides an overview of the intellectual history of data visualization from medieval to modern times, describing and illustrating some significant advances along the way. It is based on what I call the Milestones Project, an attempt to provide a broadly comprehensive and representative catalogue of important developments in all fields related to the history of data visualization.
A Brief History of Data Visualization
17
here are many historical accounts of developments within the fields of probability (Hald, ), statistics (Pearson, ; Porter, ; Stigler, ), astronomy (Riddell, ) and cartography (Wallis and Robinson, ), which relate to, inter alia, some of the important developments contributing to modern data visualization. here are other, more specialized, accounts which focus on the early history of graphic recording (Hoff and Geddes, , ), statistical graphs (Funkhouser, , ; Royston, ; Tilling, ), fitting equations to empirical data (Farebrother, ), economics and time-series graphs (Klein, ), cartography (Friis, ; Kruskal, ) and thematic mapping (Robinson, ; Palsky, ) and so forth; Robinson (Robinson, , Chap. ) presents an excellent overview of some of the important scientific, intellectual and technical developments of the th–th centuries leading to thematic cartography and statistical thinking. Wainer and Velleman () provide a recent account of some of the history of statistical graphics. But there are no accounts which span the entire development of visual thinking and the visual representation of data and which collate the contributions of disparate disciplines. Inasmuch as their histories are intertwined, so too should be any telling of the development of data visualization. Another reason for interweaving these accounts is that practitioners in these fields today tend to be highly specialized and unaware of related developments in areas outside their domain, much less of their history.
Milestones Tour Every picture tells a story.
1.2 – Rod Stewart,
In organizing this history, it proved useful to divide history into epochs, each of which turned out to be describable by coherent themes and labels. his division is, of course, somewhat artificial, but it provides the opportunity to characterize the accomplishments in each period in a general way before describing some of them in more detail. Figure ., discussed in Sect. .., provides a graphic overview of the epochs I describe in the subsections below, showing the frequency of events considered milestones in the periods of this history. For now, it suffices to note the labels attached to these epochs, a steady rise from the early th century to the late th century, with a curious wiggle thereater. In the larger picture – recounting the history of data visualization – it turns out that many of the milestone items have a story to be told: What motivated this development? What was the communication goal? How does it relate to other developments – What were the precursors? How has this idea been used or re-invented today? Each section below tries to illustrate the general themes with a few exemplars. In particular, this account attempts to tell a few representative stories of these periods, rather than to try to be comprehensive. For reasons of economy, only a limited number of images could be printed here, and these only in black and white. Others are referred to by Web links, mostly from
18
Michael Friendly
Figure .. Time distribution of events considered milestones in the history of data visualization, shown
by a rug plot and density estimate
the Milestones Project, http://www.math.yorku.ca/SCS/Gallery/milestone/, where a colour version of this chapter will also be found. 1.2.1
Pre-17th Century: Early Maps and Diagrams he earliest seeds of visualization arose in geometric diagrams, in tables of the positions of stars and other celestial bodies, and in the making of maps to aid in navigation and exploration. he idea of coordinates was used by ancient Egyptian surveyors in laying out towns, earthly and heavenly positions were located by something akin to latitude and longitude by at least B.C., and the map projection of a spherical earth into latitude and longitude by Claudius Ptolemy [c. –c. ] in Alexandria would serve as reference standards until the th century. Among the earliest graphical depictions of quantitative information is an anonymous th-century multiple time-series graph of the changing position of the seven most prominent heavenly bodies over space and time (Fig. .), described by Funkhouser () and reproduced in Tute (, p. ). he vertical axis represents the inclination of the planetary orbits; the horizontal axis shows time, divided into intervals. he sinusoidal variation with different periods is notable, as is the use of a grid, suggesting both an implicit notion of a coordinate system and something akin to graph paper, ideas that would not be fully developed until the –s. In the th century, the idea of plotting a theoretical function (as a proto bar graph) and the logical relation between tabulating values and plotting them appeared in
A Brief History of Data Visualization
19
Figure .. Planetary movements shown as cyclic inclinations over time, by an unknown astronomer,
appearing in a th-century appendix to commentaries by A.T. Macrobius on Cicero’s In Somnium Sciponis. Source: Funkhouser (, p. )
a work by Nicole Oresme [–] Bishop of Liseus (Oresme, , ), followed somewhat later by the idea of a theoretical graph of distance vs. speed by Nicolas of Cusa. By the th century, techniques and instruments for precise observation and measurement of physical quantities and geographic and celestial position were well developed (for example, a ‘wall quadrant’ constructed by Tycho Brahe [–], covering an entire wall in his observatory). Particularly important were the development of triangulation and other methods to determine mapping locations accurately (Frisius, ; Tartaglia, ). As well, we see initial ideas for capturing images directly (the camera obscura, used by Reginer Gemma-Frisius in to record an eclipse of the sun), the recording of mathematical functions in tables (trigonometric tables by Georg Rheticus, ) and the first modern cartographic atlas (heatrum Orbis Terrarum by Abraham Ortelius, ). hese early steps comprise the beginnings of data visualization.
1600–1699: Measurement and Theory Amongst the most important problems of the th century were those concerned with physical measurement – of time, distance and space – for astronomy, survey
Funkhouser (, p. ) was sufficiently impressed with Oresme’s grasp of the relation between functions and graphs that he remarked, ‘If a pioneering contemporary had collected some data and presented Oresme with actual figures to work upon, we might have had statistical graphs four hundred years before Playfair.’
1.2.2
20
Michael Friendly
ing, map making, navigation and territorial expansion. his century also saw great new growth in theory and the dawn of practical application – the rise of analytic geometry and coordinate systems (Descartes and Fermat), theories of errors of measurement and estimation (initial steps by Galileo in the analysis of observations on Tycho Brahe’s star of (Hald, , §.)), the birth of probability theory (Pascal and Fermat) and the beginnings of demographic statistics (John Graunt) and ‘political arithmetic’ (William Petty) – the study of population, land, taxes, value of goods, etc. for the purpose of understanding the wealth of the state. Early in this century, Christopher Scheiner (–, recordings from ) introduced an idea Tute () would later call the principle of ‘small multiples’ to show the changing configurations of sunspots over time, shown in Fig. .. he multiple images depict the recordings of sunpots from October until December of that year. he large key in the upper let identifies seven groups of sunspots by the letters A–G. hese groups are similarly identified in the smaller images, arrayed let to right and top to bottom below. Another noteworthy example (Fig. .) shows a graphic by Michael Florent van Langren[–], a Flemish astronomer to the court of Spain, believed to be the first visual representation of statistical data (Tute, , p. ). At that time, lack of
Figure .. Scheiner’s representation of the changes in sunspots over time. Source: Scheiner
(–)
A Brief History of Data Visualization
21
Figure .. Langren’s graph of determinations of the distance, in longitude, from Toledo to Rome.
he correct distance is ′ . Source: Tute (, p. )
a reliable means to determine longitude at sea hindered navigation and exploration. his -D line graph shows all known estimates of the difference in longitude between Toledo and Rome and the name of the astronomer (Mercator, Tycho Brahe, Ptolemy, etc.) who provided each observation. What is notable is that van Langren could have presented this information in various tables – ordered by author to show provenance, by date to show priority, or by distance. However, only a graph shows the wide variation in the estimates; note that the range of values covers nearly half the length of the scale. Van Langren took as his overall summary the centre of the range, where there happened to be a large enough gap for him to inscribe ‘ROMA.’ Unfortunately, all of the estimates were biased upwards; the true distance ( ′ ) is shown by the arrow. Van Langren’s graph is also a milestone as the earliest known exemplar of the principle of ‘effect ordering for data display’ (Friendly and Kwan, ). In the s, the systematic collection and study of social data began in various European countries, under the rubric of ‘political arithmetic’ (John Graunt, and William Petty, ), with the goals of informing the state about matters related to wealth, population, agricultural land, taxes and so forth, as well as for commercial purposes such as insurance and annuities based on life tables (Jan de Witt, ). At approximately the same time, the initial statements of probability theory around (see Ball, ) together with the idea of coordinate systems were applied by Christiaan Huygens in to give the first graph of a continuous distribution function (from Graunt’s based on the bills of mortality). he mid-s saw the first bivariate plot derived from empirical data, a theoretical curve relating barometric pressure to altitude, and the first known weather map, showing prevailing winds on a map of the earth (Halley, ). By the end of this century, the necessary elements for the development of graphical methods were at hand – some real data of significant interest, some theory to make
For navigation, latitude could be fixed from star inclinations, but longitude required accurate measurement of time at sea, an unsolved problem until with the invention of a marine chronometer by John Harrison. See Sobel () for a popular account. For example, Graunt () used his tabulations of London births and deaths from parish records and the bills of mortality to estimate the number of men the king would find available in the event of war (Klein, , pp. –). Image: http://math.yorku.ca/SCS/Gallery/images/huygens-graph.gif Image: http://math.yorku.ca/SCS/Gallery/images/halleyweathermap-.jpg
22
Michael Friendly
sense of them, and a few ideas for their visual representation. Perhaps more importantly, one can see this century as giving rise to the beginnings of visual thinking, as illustrated by the examples of Scheiner and van Langren. 1.2.3
1700–1799: New Graphic Forms With some rudiments of statistical theory, data of interest and importance, and the idea of graphic representation at least somewhat established, the th century witnessed the expansion of these aspects to new domains and new graphic forms. In cartography, mapmakers began to try to show more than just geographical position on a map. As a result, new data representations (isolines and contours) were invented, and thematic mapping of physical quantities took root. Towards the end of this century, we see the first attempts at the thematic mapping of geologic, economic and medical data. Abstract graphs, and graphs of functions became more widespread, along with the early stirrings of statistical theory (measurement error) and systematic collection of empirical data. As other (economic and political) data began to be collected, some novel visual forms were invented to portray them, so the data could ‘speak to the eyes.’ For example, the use of isolines to show contours of equal value on a coordinate grid (maps and charts) was developed by Edmund Halley (). Figure ., showing isogons – lines of equal magnetic declination – is among the first examples of thematic cartography, overlaying data on a map. Contour maps and topographic maps were introduced somewhat later by Philippe Buache () and Marcellin du CarlaBoniface (). Timelines, or ‘cartes chronologiques,’ were first introduced by Jacques BarbeuDubourg in the form of an annotated chart of all of history (from Creation) on a foot scroll (Ferguson, ). Joseph Priestley, presumably independently, used a more convenient form to show first a timeline chart of biography (lifespans of famous people, B.C. to A.D. , Priestley, ), and then a detailed chart of history (Priestley, ). he use of geometric figures (squares or rectangles) and cartograms to compare areas or demographic quantities by Charles de Fourcroy () and August F.W. Crome () provided another novel visual encoding for quantitative data using superimposed squares to compare the areas of European states. As well, several technological innovations provided necessary ingredients for the production and dissemination of graphic works. Some of these facilitated the reproduction of data images, such as three-colour printing, invented by Jacob le Blon in , and lithography, invented by Aloys Senefelder in . Of the latter, Robinson (, p. ) says “the effect was as great as the introduction [of the Xerox machine].” Yet, likely due to expense, most of these new graphic forms appeared in publications with limited circulation, unlikely to attract wide attention.
Image: http://math.yorku.ca/SCS/Gallery/images/palsky/defourcroy.jpg
A Brief History of Data Visualization
23
Figure .. A portion of Edmund Halley’s New and Correct Sea Chart Shewing the Variations in the
Compass in the Western and Southern Ocean, . Source: Halley (), image from Palsky (, p. )
A prodigious contributor to the use of the new graphical methods, Johann Lambert [–] introduced the ideas of curve fitting and interpolation from empirical data points. He used various sorts of line graphs and graphical tables to show periodic variation in, for example, air and soil temperature. William Playfair [–] is widely considered the inventor of most of the graphical forms used today – first the line graph and barchart (Playfair, ), later the
Image: http://www.journals.uchicago.edu/Isis/journal/demo/vn//fg.gif
24
Michael Friendly
Figure .. Redrawn version of a portion of Playfair’s pie-circle-line chart, comparing population
and taxes in several nations
piechart and circle graph (Playfair, ). Figure . shows a creative combination of different visual forms: circles, pies and lines, redrawn from Playfair (, Plate ). he use of two separate vertical scales for different quantities (population and taxes) is today considered a sin in statistical graphics (you can easily jiggle either scale to show different things). But Playfair used this device to good effect here to try to show taxes per capita in various nations and argue that the British were overtaxed, compared with others. But, alas, showing simple numbers by a graph was hard enough for Playfair – he devoted several pages of text in Playfair () describing how to read and understand a line graph. he idea of calculating and graphing rates and other indirect measurements was still to come. In this figure, the let axis and line on each circle/pie graph shows population, while the right axis and line shows taxes. Playfair intended that the slope of the line connecting the two would depict the rate of taxation directly to the eye; but, of course, the slope also depends on the diameters of the circles. Playfair’s graphic sins can perhaps be forgiven here, because the graph clearly shows the slope of the line for Britain to be in the opposite direction of those for the other nations. A somewhat later graph (Playfair, ), shown in Fig. ., exemplifies the best that Playfair had to offer with these graphic forms. Playfair used three parallel time series to show the price of wheat, weekly wages and reigning ruler over a -year span from to and used this graph to argue that workers had become better off in the most recent years. By the end of this century (), the utility of graphing in scientific applications prompted a Dr Buxton in London to patent and market printed coordinate paper; curiously, a patent for lined notepaper was not issued until . he first known
A Brief History of Data Visualization
25
Figure .. William Playfair’s time-series graph of prices, wages and reigning ruler over a -year
period. Source: Playfair (), image from Tute (, p. )
published graph using coordinate paper is one of periodic variation in barometric pressure (Howard, ). Nevertheless, graphing of data would remain rare for another or so years, perhaps largely because there wasn’t much quantitative information (apart from widespread astronomical, geodetic and physical measurement) of sufficient complexity to require new methods and applications. Official statistics, regarding population and mortality, and economic data were generally fragmentary and oten not publicly available. his would soon change.
1800–1850: Beginnings of Modern Graphics With the fertilization provided by the previous innovations of design and technique, the first half of the th century witnessed explosive growth in statistical graphics and thematic mapping, at a rate which would not be equalled until modern times. In statistical graphics, all of the modern forms of data display were invented: barand piecharts, histograms, line graphs and time-series plots, contour plots, scatterplots and so forth. In thematic cartography, mapping progressed from single maps to comprehensive atlases, depicting data on a wide variety of topics (economic, social, moral, medical, physical, etc.), and introduced a wide range of novel forms of symbolism. During this period graphical analysis of natural and physical phenomena (lines of magnetism, weather, tides, etc.) began to appear regularly in scientific publications as well. In , the first geological maps were introduced in England by William Smith [–], setting the pattern for geological cartography or ‘stratigraphic geology’
William Herschel (), in a paper that describes the first instance of a modern scatterplot, devoted three pages to a description of plotting points on a grid.
1.2.4
26
Michael Friendly
(Smith, ). hese and other thematic maps soon led to new ways of showing quantitative information on maps and, equally importantly, to new domains for graphically based inquiry. In the s, Baron Charles Dupin [–] invented the use of continuous shadings (from white to black) to show the distribution and degree of illiteracy in France (Dupin, ) – the first unclassed choropleth map, and perhaps the first modern-style thematic statistical map (Palsky, , p. ). Later given the lovely title ‘Carte de la France obscure et de la France éclairée,’ it attracted wide attention, and was also perhaps the first application of graphics in the social realm. More significantly, in , the ministry of justice in France instituted the first centralized national system of crime reporting, collected quarterly from all departments and recording the details of every charge laid before the French courts. In , André-Michel Guerry, a lawyer with a penchant for numbers, used these data (along with other data on literacy, suicides, donations to the poor and other ‘moral’ variables) to produce a seminal work on the moral statistics of France (Guerry, ) – a work that (along with Quételet, , ) can be regarded as the foundation of modern social science. Guerry used maps in a style similar to Dupin to compare the ranking of departments on pairs of variables, notably crime vs. literacy, but other pairwise variable comparisons were made. He used these to argue that the lack of an apparent (negative) relation between crime and literacy contradicted the armchair theories of some social reformers who had argued that the way to reduce crime was to increase education. Guerry’s maps and charts made somewhat of an academic sensation both in France and the rest of Europe; he later exhibited several of these at the London Exhibition and carried out a comparative study of crime in England and France (Guerry, ) for which he was awarded the Moynton Prize in statistics by the French Academy of Sciences. But Guerry’s systematic and careful work was unable
Image: http://math.yorku.ca/SCS/Gallery/images/dupin-map_.jpg Guerry showed that rates of crime, when broken down by department, type of crime, age and gender of the accused and other variables, remained remarkably consistent from year to year, yet varied widely across departments. He used this to argue that such regularity implied the possibility of establishing social laws, much as the regularity of natural phenomena implied physical ones. Guerry also pioneered the study of suicide, with tabulations of suicides in Paris, –, by sex, age, education, profession, etc., and a content analysis of suicide notes as to presumed motives. Today, one would use a scatterplot, but that graphic form had only just been invented (Herschel, ) and would not enter common usage for another years; see Friendly and Denis (). Guerry seemed reluctant to take sides. He also contradicted the social conservatives who argued for the need to build more prisons or impose more severe criminal sentences. See Whitt (). Among the plates in this last work, seven pairs of maps for England and France each included sets of small line graphs to show trends over time, decompositions by subtype of crime and sex, distributions over months of the year, and so forth. he final plate, on general causes of crime, is an incredibly detailed and complex multivariate semi-graphic display attempting to relate various types of crimes to each other, to various social and moral aspects (instruction, religion, population) as well as to their geographic distribution.
A Brief History of Data Visualization
27
Figure .. A portion of Dr Robert Baker’s cholera map of Leeds, , showing the districts affected by
cholera. Source: Gilbert (, Fig. )
to shine in the shadows cast by Adolphe Quételet, who regarded moral and social statistics as his own domain. In October , the first case of asiatic cholera occurred in Great Britain, and over people died in the epidemic that ensued over the next months or so (Gilbert, ). Subsequent cholera epidemics in – and – produced similarly large death tolls, but the water-borne cause of the disease was unknown until when Dr John Snow produced his famous dot map (Snow, ) showing deaths due to cholera clustered around the Broad Street pump in London. his was indeed a landmark graphic discovery, but it occurred at the end of the period, roughly – , which marks a high point in the application of thematic cartography to human (social, medical, ethnic) topics. he first known disease map of cholera (Fig. .), due to Dr Robert Baker (), shows the districts of Leeds ‘affected by cholera’ in the particularly severe outbreak. I show this figure to make another point – why Baker’s map did not lead to a ‘eureka’ experience, while John Snow’s did. Baker used a town plan of Leeds that had been divided into districts. Of a population of in all of Leeds, Baker mapped
Image: http://www.math.yorku.ca/SCS/Gallery/images/snow.jpg
28
Michael Friendly
the cholera cases by hatching in red ‘the districts in which the cholera had prevailed.’ In his report, he noted an association between the disease and living conditions: ‘how exceedingly the disease has prevailed in those parts of the town where there is a deficiency, oten an entire want of sewage, drainage and paving’ (Baker, , p. ). Baker did not indicate the incidence of disease on his map, nor was he equipped to display rates of disease (in relation to population density), and his knowledge of possible causes, while definitely on the right track, was both weak and implicit (not analysed graphically or by other means). It is likely that some, perhaps tenuous, causal indicants or evidence were available to Baker, but he was unable to connect the dots or see a geographically distributed outcome in relation to geographic factors in even the simple ways that Guerry had tried. At about the same time, –, the use of graphs began to become recognized in some official circles for economic and state planning – where to build railroads and canals? What is the distribution of imports and exports? his use of graphical methods is no better illustrated than in the works of Charles Joseph Minard [–], whose prodigious graphical inventions led Funkhouser () to call him the Playfair of France. To illustrate, we choose (with some difficulty) an ‘tableau-graphique’ (Fig. .) by Minard, an early progenitor of the modern mosaicplot (Friendly, ). On the surface, mosaicplots descend from bar charts, but Minard introduced two simultaneous innovations: the use of divided and proportional-width bars so that area had a concrete visual interpretation. he graph shows the transportation of commercial goods along one canal route in France by variable-width, divided bars (Minard, ). In this display the width of each vertical bar shows distance along this route; the divided-bar segments have height proportional to amount of goods of various types (shown by shading), so the area of each rectangular segment is proportional to the cost of transport. Minard, a true visual engineer (Friendly, ), developed such diagrams to argue visually for setting differential price rates for partial vs. complete runs. Playfair had tried to make data ‘speak to the eyes,’ but Minard wished to make them ‘calculer par l’œil’ as well. It is no accident that, in England, outside the numerous applications of graphical methods in the sciences, there was little interest in or use of graphs amongst statisticians (or ‘statists’ as they called themselves). If there is a continuum ranging from ‘graph people’ to ‘table people,’ British statisticians and economists were philosophically more table-inclined and looked upon graphs with suspicion up to the time of William Stanley Jevons around (Maas and Morgan, ). Statistics should be concerned with the recording of ‘facts relating to communities of men which are capable of being expressed by numbers’ (Mouat, , p. ), leaving the generalization to laws and theories to others. Indeed, this view was made abundantly clear in the logo of the Statistical Society of London (now the Royal Statistical Society): a banded
he German geographer Augustus Petermann produced a ‘Cholera map of the British Isles’ in using national data from the – epidemic (image: http://images.rgs.org/webimages//////S.jpg) shaded in proportion to the relative rate of mortality using class intervals (< , , , . . . ). No previous disease map had allowed determination of the range of mortality in any given area.
A Brief History of Data Visualization
29
Figure .. Minard’s Tableau Graphique, showing the transportation of commercial goods along the
Canal du Centre (Chalon–Dijon). Intermediate stops are spaced by distance, and each bar is divided by type of goods, so the area of each tile represents the cost of transport. Arrows show the direction of transport. Source: ENPC:/C (Col. et cliché ENPC; used by permission)
sheaf of wheat, with the motto Aliis Exterendum – to others to flail the wheat. Making graphs, it seemed, was too much like breadmaking.
1850–1900: The Golden Age of Statistical Graphics By the mid-s, all the conditions for the rapid growth of visualization had been established – a ‘perfect storm’ for data graphics. Official state statistical offices were established throughout Europe, in recognition of the growing importance of numerical information for social planning, industrialization, commerce and transportation. Statistical theory, initiated by Gauss and Laplace and extended to the social realm by Guerry and Quételet, provided the means to make sense of large bodies of data. What started as the Age of Enthusiasm (Funkhouser, ; Palsky, ) for graphics ended with what can be called the Golden Age, with unparalleled beauty and many innovations in graphics and thematic cartography. So varied were these developments that it is difficult to be comprehensive, but a few themes stand out.
1.2.5
30
Michael Friendly
Escaping Flatland Although some attempts to display more than two variables simultaneously had occurred earlier in multiple time series (Playfair, ; Minard, ), contour graphs (Vauthier, ) and a variety of thematic maps, (e.g. Berghaus ()) a number of significant developments extended graphics beyond the confines of a flat piece of paper. Gustav Zeuner [–] in Germany (Zeuner, ), and later Luigi Perozzo [–] in Italy (Perozzo, ) constructed -D surface plots of population data. he former was an axonometric projection showing various slices, while the latter (a -D graph of population in Sweden from – by year and age group) was printed in red and black and designed as a stereogram. Contour diagrams, showing isolevel curves of -D surfaces, had also been used earlier in mapping contexts (Nautonier, –; Halley, ; von Humboldt, ), but the range of problems and data to which they were applied expanded considerably over this time in attempts to understand relations among more than two data-based variables, or where the relationships are statistical, rather than functional or measured with little error. It is more convenient to describe these under Galton, below. By , the idea of visual and imaginary worlds of varying numbers of dimensions found popular expression in Edwin Abbott’s () Flatland, implicitly suggesting possible views in four and more dimensions.
Graphical Innovations With the usefulness of graphical displays for understanding complex data and phenomena established, many new graphical forms were invented and extended to new areas of inquiry, particularly in the social realm. Minard () developed the use of divided circle diagrams on maps (showing both a total, by area, and subtotals, by sectors, with circles for each geographic region on the map). Later he developed to an art form the use of flow lines on maps of width proportional to quantities (people, goods, imports, exports) to show movement and transport geographically. Near the end of his life, the flow map would be taken to its highest level in his famous depiction of the fate of the armies of Napoleon and Hannibal, in what Tute () would call the ‘best graphic ever produced.’ See Friendly () for a wider appreciation of Minard’s work. he social and political uses of graphics is also evidenced in the polar area charts (called ‘rose diagrams’ or ‘coxcombs’) invented by Florence Nightingale [–] to wage a campaign for improved sanitary conditions in battlefield treatment of soldiers (Nightingale, ). hey let no doubt that many more soldiers died from disease and the consequences of wounds than at the hands of the enemy. From around the same time, Dr John Snow [–] is remembered for his use of a dot map of deaths from cholera in an outbreak in London. Plotting the residence of each
Image: http://math.yorku.ca/SCS/Gallery/images/stereo.jpg Zeuner used one axis to show year of birth and another to show present age, with number of surviving persons on the third, vertical, axis giving a -D surface. One set of curves thus showed the distribution of population for a given generation; the orthogonal set of curves showed the distributions across generations at a given point in time, e.g. at a census.
A Brief History of Data Visualization
31
deceased provided the insight for his conclusion that the source of the outbreak could be localized to contaminated water from a pump on Broad Street, the founding innovation for modern epidemiological mapping. Scales and shapes for graphs and maps were also transformed for a variety of purposes, leading to semi-logarithmic graphs (Jevons, , ) to show percentage change in commodities over time, log-log plots to show multiplicative relations, anamorphic maps by Émile Cheysson (Palsky, , Figs. –) using deformations of spatial size to show a quantitative variable (e.g. the decrease in time to travel from Paris to various places in France over years) and alignment diagrams or nomograms using sets of parallel axes. We illustrate this slice of the Golden Age with Fig. ., a tour-de-force graphic for determination of magnetic deviation at sea in relation to latitude and longitude without calculation (‘L’ Abaque Triomphe’) by Charles Lallemand (), director general of the geodetic measurement of altitudes throughout France, which combines many variables into a multifunction nomogram, using -D, juxtaposition of anamorphic maps, parallel coordinates and hexagonal grids.
Figure .. Lallemand’s L’ abaque du bateau “Le Triomphe”, allowing determination of magnetic
deviation at sea without calculation. Source: courtesy Mme Marie-Noëlle Maisonneuve, Les fonds anciens de la bibliothèque de l’École des Mines de Paris
32
Michael Friendly
Galton’s Contributions Special note should be made of the varied contributions of Francis Galton [–] to data visualization and statistical graphics. Galton’s role in the development of the ideas of correlation and regression are well known. Less well known is the role that visualization and graphing played in his contributions and discoveries. Galton’s statistical insight (Galton, ) – that, in a bivariate (normal) distribution, (say, height of a child against height of parents), (a) he isolines of equal frequency would appear as concentric ellipses and (b) he locus of the (regression) lines of means of yx and of xy were the conjugate diameters of these ellipses – was based largely on visual analysis from the application of smoothing to his data. Karl Pearson would later say ‘that Galton should have evolved all this from his observations is to my mind one of the most noteworthy scientific discoveries arising from pure analysis of observations.’ (Pearson, , p. ). his was only one of Galton’s discoveries based on graphical methods. In earlier work, Galton had made wide use of isolines, contour diagrams and smoothing in a variety of areas. An paper showed the use of ‘isodic curves’ to portray the joint effects of wind and current on the distance ships at sea could travel in any direction. An ‘isochronic chart’ (Galton, ) showed the time it took to reach any destination in the world from London by means of coloured regions on a world map. Still later, he analysed rates of fertility in marriages in relation to the ages of father and mother using ‘isogens,’ curves of equal percentage of families having a child (Galton, ). But perhaps the most notable non-statistical graphical discovery was that of the “anti-cyclonic” (anticlockwise) pattern of winds around low-pressure regions, combined with clockwise rotations around high-pressure zones. Galton’s work on weather patterns began in and was summarized in Meteorographica (). It contained a variety of ingenious graphs and maps (over illustrations in total), one of which is shown in Fig. .. his remarkable chart, one of a two-page Trellis-style display, shows observations on barometric pressure, wind direction, rain and temperature from days in December . For each day, the grid shows schematic maps of Europe, mapping pressure (row ), wind and rain (row ) and temperature (row ), in the morning, aternoon and evening (columns). One can clearly see the series of black areas (low pressure) on the barometric charts for about the first half of the month, corresponding to the anticlockwise arrows in the wind charts, followed by a shit to red areas (high pressure) and more clockwise arrows. Wainer (, p. ) remarks, ‘Galton did for the collectors of weather data what Kepler did for Tycho Brahe. his is no small accomplishment.’
Statistical Atlases he collection, organization and dissemination of official government statistics on population, trade and commerce, social, moral and political issues became wide
In July , Galton distributed a circular to meterologists throughout Europe, asking them to record these data synchonously, three times a day for the entire month of December . About weather stations supplied the data; see Pearson (–, pp. –).
A Brief History of Data Visualization
33
Figure .. One page of Galton’s multivariate weather chart of Europe showing barometric
pressure, wind direction, rain and temperature for the month of December . Source: Pearson (–, pl. )
spread in most of the countries of Europe from about to (Westergaard, ). Reports containing data graphics were published with some regularity in France, Germany, Hungary and Finland, and with tabular displays in Sweden, Holland, Italy and elsewhere. At the same time, there was an impetus to develop standards for graphical presentation at the International Statistical Congresses which had begun in in Belgium (organized by Quételet), and these congresses were closely linked with state statistical bureaus. he main participants in the graphics section included Georg von Mayr, Hermann Schwabe, Pierre Émile Levasseur and Émile Cheysson. Among other recommendations was one from the th Statistical Congress in that official publications be accompanied by maps and diagrams. he statesponsored statistical atlases that ensued provide additional justification to call this period the golden age of graphics, and some of its most impressive exemplars. he pinnacle of this period of state-sponsored statistical albums is undoubtedly the Albums de statistique graphique published annually by the French ministry of public works from to under the direction of Émile Cheysson. hey were
Cheysson had been one of the major participants in committees on the standardization of graphical methods at the International Statistical Congresses from on. He was trained
34
Michael Friendly
published as large-format books (about in.), and many of the plates folded out to four or six times that size, all printed in colour and with great attention to layout and composition. We concur with Funkhouser (, p. ) that “the Albums present the finest specimens of French graphic work in the century and considerable pride was taken in them by the French people, statisticians and laymen alike.” he subject matter of the albums largely concerned economic and financial data related to the planning, development and administration of public works – transport of passengers and freight, by rail, on inland waterways and through seaports, but also included such topics as revenues in the major theaters of Paris, attendance at the universal expositions of , and , changes in populations of French departments over time and so forth. More significantly for this account the Albums can also be viewed as an exquisite sampler of all the graphical methods known at the time, with significant adaptations to the problem at hand. he majority of these graphs used and extended the flow map pioneered by Minard. Others used polar forms – variants of pie and circle diagrams, star plots and rose diagrams, oten overlaid on a map and extended to show additional variables of interest. Still others used subdivided squares in the manner of modern mosaic displays (Friendly, ) to show the breakdown of a total (passengers, freight) by several variables. It should be noted that in almost all cases the graphical representation of the data was accompanied by numerical annotations or tables, providing precise numerical values. he Albums are discussed extensively by Palsky (), who includes seven representative illustrations. It is hard to choose a single image here, but my favourites are surely the recursive, multimosaic of rail transportation for the – volumes, the first of which is shown in Fig. .. his cartogram uses one large mosaic (in the lower let) to show the numbers of passengers and tons of freight shipped from Paris from the four principal train stations. Of the total leaving Paris, the amounts going to each main city are shown by smaller mosaics, coloured according to railway lines; of those amounts, the distribution to smaller cities is similarly shown, connected by lines along the rail routes. Among the many other national statistical albums and atlases, those from the US Census bureau also deserve special mention. he Statistical Atlas of the Ninth Census, produced in – under the direction of Francis A. Walker [–], contained plates, including several novel graphic forms. he ambitious goal was to present a graphic portrait of the nation, and it covered a wide range of physical and human topics: geology, minerals and weather; population by ethnic origin, wealth, illiteracy, school attendance and religious affiliation; death rates by age, sex, race and cause; prevalence of blindness, deaf mutism and insanity; and so forth. ‘Age pyramids’ (back-to-back, bilateral frequency histograms and polygons) were used effectively to compare age distributions of the population for two classes (gender, married/single, etc.). Subdivided squares and area-proportional pies of various forms were also used to provide comparisons among the states on multiple dimensions simultaneously as an engineer at the ENPC and later became a professor of political economy at the École des Mines.
A Brief History of Data Visualization
35
Figure .. [his figure also appears in the color insert.] Mouvement des voyageurs et des marchandises
dans les principales stations de chemins de fer en . Scale: mm = passengers or tons of freight. Source: Album, , Plate (author’s collection)
(employed/unemployed, sex, schooling, occupational categories). he desire to provide for easy comparisons among states and other categorizations was expressed by arranging multiple subfigures as ‘small multiples’ in many plates.
36
Michael Friendly
Following each subsequent decennial census for to , reports and statistical atlases were produced with more numerous and varied graphic illustrations. he volume from the Eleventh Census (), under the direction of Henry Gannett [–], contained over graphs, cartograms and statistical diagrams. here were several ranked parallel coordinate plots comparing states and cities over all censuses from –. Trellis-like collections of shaded maps showed interstate migration, distributions of religious membership, deaths by known causes and so forth. he and volumes produced under Gannett’s direction are also notable for (a) the multimodal combination of different graphic forms (maps, tables, barcharts, bilateral polygons) in numerous plates and (b) the consistent use of effectorder sorting (Friendly and Kwan, ) to arrange states or other categories in relation to what was to be shown, rather than for lookup (e.g. Alabama–Wyoming). For example, Fig. . shows interstate immigration in relation to emigration for the states and territories in . he right side shows population loss sorted by emigration, ranging from New York, Ohio, Pennsylvania and Illinois at the top to Idaho, Wyoming and Arizona at the bottom. he let side shows where the emigrants went: Illinois, Missouri, Kansas and Texas had the biggest gains, Virginia the biggest net loss. It is clear that people were leaving the eastern states and were attracted to those of the Midwest Mississippi valley. Other plates showed this data in map-based formats. However, the Age of Enthusiasm and the Golden Age were drawing to a close. he French Albums de statistique graphique were discontinued in due to the high cost of production; statistical atlases appeared in Switzerland in and , but never again. he final two US Census atlases, issued ater the and censuses, ‘were both routinized productions, largely devoid of colour and graphic imagination’ (Dahmann, ).
Figure .. Interstate migration shown by back-to-back barcharts, sorted by emigration. Source:
Statistical Atlas of the Eleventh Census, , diagram , p. (author’s collection)
A Brief History of Data Visualization
37
1900–1950: The Modern Dark Ages If the late s were the ‘golden age’ of statistical graphics and thematic cartography, the early s can be called the ‘modern dark ages’ of visualization (Friendly and Denis, ). here were few graphical innovations, and by the mid-s the enthusiasm for visualization which characterized the late s had been supplanted by the rise of quantification and formal, oten statistical, models in the social sciences. Numbers, parameter estimates and, especially, those with standard errors were precise. Pictures were – well, just pictures: pretty or evocative, perhaps, but incapable of stating a ‘fact’ to three or more decimals. Or so it seemed to many statisticians. But it is equally fair to view this as a time of necessary dormancy, application and popularization rather than one of innovation. In this period statistical graphics became mainstream. Graphical methods entered English textbooks (Bowley, ; Peddle, ; Haskell, ; Karsten, ), the curriculum (Costelloe, ; Warne, ) and standard use in government (Ayres, ), commerce (Gantt charts and Shewart’s control charts) and science. hese textbooks contained rather detailed descriptions of the graphic method, with an appreciative and oten modern flavour. For example, Sir Arthur Bowley’s () Elements of Statistics devoted two chapters to graphs and diagrams and discussed frequency and cumulative frequency curves (with graphical methods for finding the median and quartiles), effects of choice of scales and baselines on visual estimation of differences and ratios, smoothing of time-series graphs, rectangle diagrams in which three variables could be shown by height, width and area of bars, and ‘historical diagrams’ in which two or more time series could be shown on a single chart for comparative views of their histories. Bowley’s (, pp. –) example of smoothing (Fig. .) illustrates the character of his approach. Here he plotted the total value of exports from Britain and Ireland over the period –. At issue was whether exports had become stationary in the most recent years, and the conclusion by Sir Robert Giffen (), based solely on tables of averages for successive -year periods, that ‘the only sign of stationariness is an increase at a less rate in the last periods than in the earlier periods’ (p. ). To answer this, he graphed the raw data, together with curves of the moving average over -, - and -year periods. he - and -year moving averages show strong evidence of an approximately -year cycle, and he noted, ‘no argument can stand which does not take account of the cycle of trade, which is not eliminated until we take decennial averages’ (p. ). To this end, he took averages of successive -year periods starting and drew a freehand curve ‘keeping as close [to the points] as possible,
he first systematic attempt to survey, describe and illustrate available graphic methods for experimental data was that of Étienne Jules Marey’s () La Méthode Graphique. Marey [–] also invented several devices for visual recording, including the sphymograph and chronophotography to record the motion of birds in flight, people running and so forth. Giffen, an early editor of he Statist, also wrote a statistical text published posthumously in ; it contained an entire chapter on constructing tables, but not a single graph (Klein, , p. ).
1.2.6
38
Michael Friendly
Figure .. Arthur Bowley’s demonstration of methods of smoothing a time-series graph. Moving averages of , and years are compared with a freehand curve drawn through four points
representing the averages of successive -year periods. Source: Bowley (, opposite p. )
without making sudden changes in curvature,’ giving the thick curve in Fig. .. Support for Sir Robert’s conclusion and the evidence for a -year cycle owe much to this graphical treatment. Moreover, perhaps for the first time, graphical methods proved crucial in a number of new insights, discoveries and theories in astronomy, physics, biology and other sciences. Among these, one may refer to (a) E.W. Maunder’s () ‘butterfly diagram’ to study the variation of sunspots over time, leading to the discovery that they were markedly reduced in frequency from –; (b) the Hertzsprung–Russell diagram (Hertzsprung, ; Spence and Garrison, ), a log-log plot of luminosity as a function of temperature for stars, used to explain the changes as a star evolves and laying the groundwork for modern stellar physics; (c) the discovery of the concept of atomic number by Henry Moseley () based largely on graphical analysis. See Friendly and Denis () for more detailed discussion of these uses.
A reanalysis of the data using a loess smoother shows that this is in fact oversmoothed and corresponds closely to a loess window width of f = .. he optimal smoothing parameter, minimizing AIC C is f = ., giving a smooth more like Bowley’s - and -year moving averages.
A Brief History of Data Visualization
39
As well, experimental comparisons of the efficacy of various graphics forms were begun (Eells, ; von Huhn, ; Washburne, ), a set of standards and rules for graphic presentation was finally adopted by a joint committee (Joint Committee on Standards for Graphic Presentation, ) and a number of practical aids to graphing were developed. In the latter part of this period, new ideas and methods for multidimensional data in statistics and psychology would provide the impetus to look beyond the -D plane. Graphic innovation was also awaiting new ideas and technology: the development of the machinery of modern statistical methodology, and the advent of the computational power and display devices which would support the next wave of developments in data visualization.
1950–1975: Rebirth of Data Visualization Still under the influence of the formal and numerical zeitgeist from the mid-s on, data visualization began to rise from dormancy in the mid-s. his was spurred largely by three significant developments: In the USA, John W. Tukey [–], in a landmark paper, he Future of Data Analysis (Tukey, ), issued a call for the recognition of data analysis as a legitimate branch of statistics distinct from mathematical statistics; shortly later, he began the invention of a wide variety of new, simple and effective graphic displays, under the rubric of ‘exploratory data analysis’ (EDA) – stem-leaf plots, boxplots, hanging rootograms, two-way table displays and so forth, many of which entered the statistical vocabulary and sotware implementation. Tukey’s stature as a statistician and the scope of his informal, robust and graphical approach to data analysis were as influential as his graphical innovations. Although not published until , chapters from Tukey’s EDA book (Tukey, ) were widely circulated as they began to appear in – and began to make graphical data analysis both interesting and respectable again. In France, Jacques Bertin [–] published the monumental Sémiologie graphique (Bertin, ). To some, this appeared to do for graphics what Mendeleev had done for the organization of the chemical elements, that is, to organize the visual and perceptual elements of graphics according to the features and relations in data. In a parallel but separate stream, an exploratory and graphical approach to multidimensional data (‘L’analyse des données’) begun by Jean-Paul Benzécri [–] provided French and other European statisticians with an alternative, visually based view of what statistics was about. Other graphically minded schools of data-thought would later arise in the Netherlands (Gifi), Germany and elsewhere in Europe. But the skills of hand-drawn maps and graphics had withered during the dormant ‘modern dark ages’ of graphics (though nearly every figure in Tukey’s EDA (Tukey, ) was, by intention, hand-drawn). Computer processing of statistical data began in with the creation of Fortran, the first high-level language for computing. By the late s, widespread mainframe university computers offered the possibility to construct old and new graphic forms by computer
1.2.7
40
Michael Friendly
programs. Interactive statistical applications, e.g. Fowlkes (); Fishkeller et al. (), and true high-resolution graphics were developed but would take a while to enter common use. By the end of this period significant intersections and collaborations would begin: (a) Computer science research (sotware tools, C language, UNIX, etc.) at Bell Laboratories (Becker, ) and elsewhere would combine forces with (b) Developments in data analysis (EDA, psychometrics, etc.) and (c) Display and input technology (pen plotters, graphic terminals, digitizer tablets, the mouse, etc.). hese developments would provide new paradigms, languages and sotware packages for expressing statistical ideas and implementing data graphics. In turn, they would lead to an explosive growth in new visualization methods and techniques. Other themes began to emerge, mostly as initial suggestions: (a) Various novel visual representations of multivariate data (Andrews’ () Fourier function plots, Chernoff () faces, star plots, clustering and tree representations); (b) he development of various dimension-reduction techniques (biplot (Gabriel, ), multidimensional scaling, correspondence analysis), providing visualization of multidimensional data in a -D approximation; (c) Animations of a statistical process; and (d) Perceptually based theory and experiments related to how graphic attributes and relations might be rendered to better convey data visually. By the close of this period, the first exemplars of modern GIS and interactive systems for -D and -D statistical graphics would appear. hese would set goals for future development and extension.
1.2.8
1975–present: High-D, Interactive and Dynamic Data Visualization During the last quarter of the th century data visualization blossomed into a mature, vibrant and multidisciplinary research area, as may be seen in this Handbook, and sotware tools for a wide range of visualization methods and data types are available for every desktop computer. Yet it is hard to provide a succinct overview of the most recent developments in data visualization because they are so varied and have occurred at an accelerated pace and across a wider range of disciplines. It is also more difficult to highlight the most significant developments which may be seen as such in a subsequent history focusing on this recent period. With this disclaimer, a few major themes stand out. he development of highly interactive statistical computing systems. Initially, this meant largely command-driven, directly programmable systems (APL, S), as opposed to compiled, batch processing; New paradigms of direct manipulation for visual data analysis (linking, brushing (Becker and Cleveland, ), selection, focusing, etc.); New methods for visualizing high-dimensional data (the grand tour (Asimov, ), scatterplot matrix (Tukey and Tukey, ), parallel coordinates plot (Inselberg, ; Wegman, ), spreadplots (Young, a), etc.);
A Brief History of Data Visualization
41
he invention (or re-invention) of graphical techniques for discrete and categorical data; he application of visualization methods to an ever-expanding array of substantive problems and data structures; and Substantially increased attention to the cognitive and perceptual aspects of data display. hese developments in visualization methods and techniques arguably depended on advances in theoretical and technological infrastructure, perhaps more so than in previous periods. Some of these are: Large-scale statistical and graphics sotware engineering, both commercial (e.g. SAS) and non-commercial (e.g. Lisp-Stat, the R project). hese have oten been significantly leveraged by open-source standards for information presentation and interaction (e.g. Java, Tcl/Tk); Extensions of classical linear statistical modelling to ever-wider domains (generalized linear models, mixed models, models for spatial/geographical data and so forth); Vastly increased computer processing speed and capacity, allowing computationally intensive methods (bootstrap methods, Bayesian MCMC analysis, etc.), access to massive data problems (measured in terabytes) and real-time streaming data. Advances in this area continue to press for new visualization methods. From the early s to mid-s, many of the advances in statistical graphics concerned static graphs for multidimensional quantitative data, designed to allow the analyst to see relations in progressively higher dimensions. Older ideas of dimensionreduction techniques (principal component analysis, multidimensional scaling, discriminant analysis, etc.) led to generalizations of projecting a high-dimensional dataset to ‘interesting’ low-dimensional views, as expressed by various numerical indices that could be optimized (projection pursuit) or explored interactively (grand tour). he development of general methods for multidimensional contingency tables began in the early s, with Leo Goodman (), Shelly Haberman () and others (Bishop et al., ) laying out the fundamentals of log-linear models. By the mids, some initial, specialized techniques for visualizing such data were developed (four-fold display (Fienberg, ), association plot (Cohen, ), mosaicplot (Hartigan and Kleiner, ) and sieve diagram (Riedwyl and Schüpbach, )), based on the idea of displaying frequencies by area (Friendly, ). Of these, extensions of the mosaicplot (Friendly, , ) have proved most generally useful and are now widely implemented in a variety of statistical sotware, most completely in the vcd package (Meyer et al., ) in R and interactive sotware from the Augsburg group (MANET, Mondrian). It may be argued that the greatest potential for recent growth in data visualization came from the development of interactive and dynamic graphic methods, allowing instantaneous and direct manipulation of graphical objects and related statistical properties. One early instance was a system for interacting with probability plots (Fowlkes, ) in real time, choosing a shape parameter of a reference distribution
42
Michael Friendly
and power transformations by adjusting a control. he first general system for manipulating high-dimensional data was PRIM-, developed by Fishkeller, Friedman and Tukey (), and providing dynamic tools for projecting, rotating (in -D), isolating (identifying subsets) and masking data in up to dimensions. hese were quite influential, but remained one-of-a-kind, ‘proof-of-concept’ systems. By the mid-s, as workstations and display technology became cheaper and more powerful, desktop sotware for interactive graphics became more widely available (e.g. MacSpin, Xgobi). Many of these developments to that point are detailed in the chapters of Dynamic Graphics for Statistics (Cleveland and McGill, ). In the s, a number of these ideas were brought together to provide more general systems for dynamic, interactive graphics, combined with data manipulation and analysis in coherent and extensible computing environments. he combination of all these factors was more powerful and influential than the sum of their parts. Lisp-Stat (Tierney, ) and its progeny (Arc, Cook and Weisberg, ; ViSta, Young, b), for example, provided an easily extensible object-oriented environment for statistical computing. In these systems, widgets (sliders, selection boxes, pick lists, etc.), graphs, tables, statistical models and the user all communicated through messages, acted upon by whoever was a designated ‘listener,’ and had a method to respond. Most of the ideas and methods behind present-day interactive graphics are described and illustrated in Young et al. (). Other chapters in this Handbook provide current perspectives on other aspects of interactive graphics.
1.3
Statistical Historiography As mentioned at the outset, this review is based on the information collected for the Milestones Project, which I regard (subject to some caveats) as a relatively comprehensive corpus of the significant developments in the history of data visualization. As such, it is of interest to consider what light modern methods of statistics and graphics can shed on this history, a self-referential question we call ‘statistical historiography’ (Friendly, ). In return, this offers other ways to view this history.
1.3.1
History as ‘Data’ Historical events, by their nature, are typically discrete, but marked with dates or ranges of dates, and some description – numeric, textual, or classified by descriptors (who, what, where, how much and so forth). Amongst the first to recognize that history could be treated as data and portrayed visually, Joseph Priestley (; ) developed the idea of depicting the lifespans of famous people by horizontal lines along a time scale. His enormous ( t., or . m) and detailed Chart of Biography showed two thousand names from B.C. to A.D. by horizontal lines from birth to death, using dots at either end to indicate ranges of uncertainty. Along the vertical dimension, Priestly classified these individuals, e.g., as statesmen or men of learning. A small fragment of this chart is shown in Fig. ..
A Brief History of Data Visualization
43
Figure .. A specimen version of Priestley’s Chart of Biography. Source: Priestley ()
Priestley’s graphical representations of time and duration apparently influenced Playfair’s introduction of time-series charts and barcharts (Funkhouser, , p. ). But these inventions did not inspire the British statisticians of his day, as noted earlier; historical events and statistical facts were seen as separate, rather than as data arrayed along a time dimension. In , at the Jubilee meeting of the Royal Statistical Society, Alfred Marshall () argued that the causes of historical events could be understood by the use of statistics displayed by ‘historical curves’ (time-series graphs): ‘I wish to argue that the graphic method may be applied as to enable history to do this work better than it has hitherto’ (p. ). Maas and Morgan () discuss these issues in more detail.
Analysing Milestones Data he information collected in the Milestones Project is rendered in print and Web forms as a chronological list but is maintained as a relational database (historical items, references, images) in order to be able to work with it as ‘data.’ he simplest analyses examine trends over time. Figure . shows a density estimate for the distribution of milestone items from to the present, keyed to the labels for the periods in history. he bumps, peaks and troughs all seem interpretable: note particularly the steady rise up to about , followed by a decline through the ‘modern dark ages’ to , then the steep rise up to the present. In fact, it is slightly surprising to see that the peak in the Golden Age is nearly as high as that at present, but this probably just reflects underrepresentation of the most recent events.
Technical note: In this figure an optimal bandwidth for the kernel density estimate was selected (using the Sheather–Jones plug-in estimate) for each series separately. he smaller range and sample size of the entries for Europe vs. North America gives a smaller bandwidth for the former, by a factor of about . Using a common bandwidth fixed to that determined for the whole series (Fig. .) undersmoothes the more extensive data on European develop-
1.3.2
44
Michael Friendly
Figure .. he distribution of milestone items over time, comparing trends in Europe and North
America
Other historical patterns can be examined by classifying the items along various dimensions (place, form, content and so forth). If we classify the items by place of development (Europe vs. North America, ignoring Other), interesting trends appear (Fig. .). he greatest peak in Europe around – coincided with a smaller peak in North America. he decline in Europe following the Golden Age was accompanied by an initial rise in North America, largely due to popularization (e.g. textbooks) and significant applications of graphical methods, then a steep decline as mathematical statistics held sway. Finally, Fig. . shows two mosaicplots for the milestone items classified by Epoch, Subject matter and Aspect. Subject was classed as having to do with human (e.g. mortality, disease), physical or mathematical characteristics of what was represented in the innovation. Aspect classed each item according to whether it was primarily map-based, a diagram or statistical innovation or a technological one. he let mosaic shows the shits in Subject over time: most of the early innovations concerned physical subjects, while the later periods shit heavily to mathematical ones. Human topics are not prevalent overall but were dominant in the th century. he right mosaic, for Subject Aspect, indicates that, unsurprisingly, map-based innovations were mainly about physical and human subjects, while diagrams and statistical ones were largely about mathematical subjects. Historical classifications clearly rely on more ments and oversmoothes the North American ones. he details differ, but most of the points made in the discussion about what was happening when and where hold.
A Brief History of Data Visualization
45
Figure .. [his figure also appears in the color insert.] Mosaic plots for milestones items, classified by Subject, Aspect and Epoch. Cells with greater (less) frequency than expected under independence are
coloured blue (red), with intensity proportional to the deviation from independence
detailed definitions than described here; however, it seems reasonable to suggest that such analyses of history as ‘data’ are a promising direction for future work.
What Was He Thinking? – Understanding Through Reproduction Historical graphs were created using available data, methods, technology and understanding current at the time. We can oten come to a better understanding of intellectual, scientific and graphical questions by attempting a re-analysis from a modern perspective. Earlier, we showed Playfair’s time-series graph (Fig. .) of wages and prices and noted that Playfair wished to show that workers were better off at the end of the period shown than at any earlier time. Presumably he wished to draw the reader’s eye to the narrowing of the gap between the bars for prices and the line graph for wages. Is this what you see? What this graph shows directly is quite different from Playfair’s intention. It appears that wages remained relatively stable while the price of wheat varied greatly. he inference that wages increased relative to prices is indirect and not visually compelling. We cannot resist the temptation to give Playfair a helping hand here – by graphing the ratio of wages to prices (labour cost of wheat), as shown in Fig. .. But this would not have occurred to Playfair because the idea of relating one time series to another by ratios (index numbers) would not occur for another half-century (due to Jevons). See Friendly and Denis () for further discussion of Playfair’s thinking. As another example, we give a brief account of an attempt to explore Galton’s discovery of regression and the elliptical contours of the bivariate normal surface,
1.3.3
46
Michael Friendly
Figure .. Redrawn version of Playfair’s time-series graph showing the ratio of price of wheat to
wages, together with a loess smoothed curve
Figure .. Galton’s smoothed correlation diagram for the data on heights of parents and children,
showing one ellipse of equal frequency. Source: (Galton, , Plate X)
A Brief History of Data Visualization
47
Figure .. Contour plot of Galton’s smoothed data, showing the curves of y¯x (filled circles, solid line),
¯ (open circles, solid line) and the corresponding regression lines (dashed) xy
treated in more detail in Friendly and Denis (). Galton’s famous graph showing these relations (Fig. .) portrays the joint frequency distribution of the height of children and the average height of their parents. It was produced from a ‘semigraphic table’ in which Galton averaged the frequencies in each set of four adjacent cells, drew isocurves of equal smoothed value and noted that these formed ‘concentric and similar ellipses.’ A literal transcription of Galton’s method, using contour curves of constant average frequency and showing the curves of the means of yx and xy, is shown in Fig. .. It is not immediately clear that the contours are concentric ellipses, nor that the curves of means are essentially linear and have horizontal and vertical tangents to the contours. A modern data analyst following the spirit of Galton’s method might substitute a smoothed bivariate kernel density estimate for Galton’s simple average of adjacent cells. he result, using jittered points to depict the cell frequencies, and a smoothed loess curve to show E(yx) is shown in Fig. .. he contours now do emphatically suggest concentric similar ellipses, and the regression line is near the points of vertical tangency. A reasonable conclusion from these figures is that Galton did not slavishly interpolate isofrequency values as is done in the contour plot shown in
48
Michael Friendly
Figure .. Bivariate kernel density estimate of Galton’s data, using jittered points for the data, and
a smoothed loess curve for E(yx) (solid) and regression line (dashed)
Fig. .. Rather, he drew his contours to the smoothed data by eye and brain (as he had done earlier with maps of weather patterns), with knowledge that he could, as one might say today, trade some increase in bias for a possible decrease in variance, and so achieve a greater smoothing.
1.4
Final Thoughts his chapter is titled ‘A brief history. . . ’ out of recognition that it it impossible to do full justice to the history of data visualization in such a short account. his is doubly so because I have attempted to present a broad view spanning the many areas of application in which data visualization took root and developed. hat being said, it is hoped that this overview will lead modern readers and developers of graphical methods to appreciate the rich history behind the latest hot new methods. As we have seen, almost all current methods have a much longer history than is commonly thought. Moreover, as I have surveyed this work and travelled to many libraries to view original works and read historical sources, I have been struck by the exquisite
A Brief History of Data Visualization
49
beauty and attention to graphic detail seen in many of these images, particularly those from the th century. We would be hard-pressed to recreate many of these today. From this history one may also see that most of the innovations in data visualization arose from concrete, oten practical, goals: the need or desire to see phenomena and relationships in new or different ways. It is also clear that the development of graphic methods depended fundamentally on parallel advances in technology, data collection and statistical theory. Finally, I believe that the application of modern methods of data visualization to its own history, in this self-referential way I call ‘statistical historiography,’ offers some interesting views of the past and challenges for the future. Acknowledgement. his work is supported by Grant from the National Sciences and Engineering Research Council of Canada. I am grateful to the archivists of many libraries and to les Chevaliers des Albums de Statistique Graphique: Antoine de Falguerolles, Ruddy Ostermann, Gilles Palsky, Ian Spence, Antony Unwin, and Howard Wainer for historical information, images, and helpful suggestions.
References Abbott, E.A. (). Flatland: A Romance of Many Dimensions, Buccaneer Books, Cutchogue, NY. ( reprint of the edition). Andrews, D.F. (). Plots of high dimensional data, Biometrics, :–. Asimov, D. (). Grand tour, SIAM Journal of Scientific and Statistical Computing, ():–. Ayres, L.P. (). he War with Germany, A Statistical Summary, U.S. Government Printing Office, Washington, D.C. Commonly known as the Ayres report; reprinted: Arno Press, NY, . Baker, R. (). Report of the Leeds Board of Health. British Library, London: .ee.(). Ball, W.W.R. (). A Short Account of the History of Mathematics, edn, Macmillan & Co., London. (re-published in , N.Y.: Dover). Becker, R.A. (). A brief history of S, in P. Dirschedl and R. Ostermann (eds), Computational Statistics, Physica Verlag, Heidleberg, pp. –. Becker, R.A. and Cleveland, W.S. (). Brushing scatterplots, Technometrics, :–. Berghaus, H. (). Physikalischer Atlas, Justus Perthes, Gotha. vols., published –. Bertin, J. (). Sémiologie Graphique: Les diagrammes, les réseaux, les cartes, Gauthier-Villars, Paris. Bishop, Y.M.M., Fienberg, S.E. and Holland, P.W. (). Discrete Multivariate Analysis: heory and Practice, MIT Press, Cambridge, MA. Bowley, A.L. (). Elements of Statistics, P.S. King and Son, London. Buache, P. (). Essai de géographie physique, Mémoires de l’Académie Royale des Sciences pp. –. Bibliothèque Nationale de France, Paris (Tolbiac): Ge.FF–.
50
Michael Friendly
Chernoff, H. (). he use of faces to represent points in k-dimensional space graphically, Journal of the American Statistical Association, :–. Cleveland, W.S. and McGill, M.E. (eds) (). Dynamic Graphics for Statistics, CRC Press, Boca Raton. FL. Cohen, A. (). On the graphical display of the significant components in a two-way contingency table, Communications in Statistics – heory and Methods, A:–. Cook, R.D. and Weisberg, S. (). Applied Regression Including Computing and Grapics, Wiley, New York. Costelloe, M.F.P. (). Graphic methods and the presentation of this subject to first year college students, Nebraska Blue Print . Crome, A.F.W. (). Über die Grösse and Bevölkerung der Sämtlichen Europäschen Staaten, Weygand, Leipzig. Dahmann, D.C. (). Presenting the nation’s cultural geography [in census atlases], At American Memory: Historical Collections for the National Digital Library Internet. http://memory.loc.gov/ammem/gmdhtml/census.html. de Fourcroy, C. (). Essai d’une table poléométrique, ou amusement d’un amateur de plans sur la grandeur de quelques villes, Dupain-Triel, Paris. de Witt, J. (). he Worth of Life Annuities Compared to Redemption Bonds, n.p., Leiden. du Carla-Boniface, M. (). Expression des nivellements; ou, méthode nouvelle pour marquer sur les cartes terrestres et marines les hauteurs et les configurations du terrain. In François de Dainville, “From the Depths to the Heights,” translated by Arthur H. Robinson, Surveying and Mapping, , :–, on page . de Nautonier, G. (–) Mecometrie de l’eymant, c’est a dire la maniere de mesurer les longitudes par le moyen de l’eymant, n.p., Paris. British Library, London: .k.. Dupin, C. (). Carte figurative de l’instruction populaire de la France, Jobard. Bibliothèque Nationale de France, Paris (Tolbiac): Ge C (Funkhouser (, p. ) incorrectly dates this as ). Eells, W.C. (). he relative merits of circles and bars for representing component parts, Journal of the American Statistical Association, :–. Farebrother, R.W. (). Fitting Linear Relationships: A History of the Calculus of Observations –, Springer, New York. Ferguson, S. (). he carte chronographique of Jacques Barbeu-Dubourg, Princeton University Library Chronicle, :–. Fienberg, S.E. (). Perspective Canada as a social report, Technical report, Department of Applied Statistics, University of Minnesota. unpublished paper. Fishkeller, M.A., Friedman, J.H. and Tukey, J.W. (). PRIM-: An interactive multidimensional data display and analysis system, Technical Report SLAC-PUB-, Stanford Linear Accelerator Center, Stanford, CA. Fowlkes, E.B. (). User’s manual for a system for interactive probability plotting on Graphic-, Technical report, Bell Laboratories. Friendly, M. (). Mosaic displays for multi-way contingency tables, Journal of the American Statistical Association, :–.
A Brief History of Data Visualization
51
Friendly, M. (). Conceptual and visual models for categorical data, he American Statistician, :–. Friendly, M. (). Extending mosaic displays: Marginal, conditional, and partial views of categorical data, Journal of Computational and Graphical Statistics, ():–. Friendly, M. (). Re-Visions of Minard, Statistical Computing & Statistical Graphics Newsletter, ():, –. Friendly, M. (). Visions and re-visions of Charles Joseph Minard, Journal of Educational and Behavioral Statistics, ():–. Friendly, M. (). Milestones in the history of data visualization: A case study in statistical historiography, in C. Weihs and W. Gaul (eds), Classification: he Ubiquitous Challenge, Springer, New York, pp. –. Friendly, M. and Denis, D. (). he roots and branches of statistical graphics, Journal de la Société Française de Statistique, ():–. (published in ). Friendly, M. and Denis, D. (). he early origins and development of the scatterplot, Journal of the History of the Behavioral Sciences, ():–. Friendly, M. and Kwan, E. (). Effect ordering for data displays, Computational Statistics and Data Analysis, ():–. Friis, H.R. (). Statistical cartography in the United States prior to and the role of Joseph C.G. Kennedy and the U.S. Census Office, American Cartographer, :–. Frisius, R.G. (). Libellus de locorum describendorum ratione, Antwerp. Funkhouser, H.G. (). A note on a tenth century graph, Osiris, :–. Funkhouser, H.G. (). Historical development of the graphical representation of statistical data, Osiris, ():–. reprinted Brugge, Belgium: St. Catherine Press, . Gabriel, K.R. (). he biplot graphic display of matrices with application to principal components analysis, Biometrics, ():–. Galton, F. (). Meteorographica, or, Methods of Mapping the Weather, Macmillan, London. British Library, London: Maps..b.. Galton, F. (). On the construction of isochronic passage charts, Proceedings of the Geographical Section of the British Association, n.v.(XI):, . Read Sept. , ; also published: Roy. Geog. Soc. Proc., , -. Galton, F. (). Regression towards mediocrity in hereditary stature, Journal of the Anthropological Institute, :–. Galton, F. (). Results derived from the natality table of Körösi by employing the method of contours or isogens, Journal of the Royal Statistical Society, ():– . Giffen, R. (). he excess of imports, Journal of the Royal Statistical Society, ():– . Gilbert, E.W. (). Pioneer maps of health and disease in England, Geographical Journal, :–. Goodman, L.A. (). he multivariate analysis of qualitative data: Interactions among multiple classifications, Journal of the American Statistical Association, :–.
52
Michael Friendly
Graunt, J. (). Natural and Political Observations Mentioned in a Following Index and Made Upon the Bills of Mortality, Martin, Allestry, and Dicas, London. Guerry, A.-M. (). Essai sur la statistique morale de la France, Crochard, Paris. English translation: Hugh P. Whitt and Victor W. Reinking, Lewiston, N.Y. : Edwin Mellen Press, . Guerry, A.-M. (). Statistique morale de l’ Angleterre comparée avec la statistique morale de la France, d’après les comptes de l’ administration de la justice criminelle en Angleterre et en France, etc., J.-B. Baillière et fils, Paris. Bibliothèque Nationale de France, Paris (Tolbiac): GR FOL-N-; SG D/; British Library, London: Maps .e.; Staatsbibliothek zu Berlin: Fe ; Library of Congress: . Haberman, S.J. (). he analysis of residuals in cross-classified tables, Biometrics, :–. Hald, A. (). A History of Probability and Statistics and their Application before , John Wiley and Sons, New York. Halley, E. (). On the height of the mercury in the barometer at different elevations above the surface of the earth, and on the rising and falling of the mercury on the change of weather, Philosophical Transactions pp. –. Halley, E. (). he description and uses of a new, and correct sea-chart of the whole world, shewing variations of the compass, London. Hartigan, J.A. and Kleiner, B. (). Mosaics for contingency tables, in W.F. Eddy (ed), Computer Science and Statistics: Proceedings of the th Symposium on the Interface, Springer-Verlag, New York, NY, pp. –. Haskell, A.C. (). How to Make and Use Graphic Charts, Codex, New York. Herschel, J.F.W. (). On the investigation of the orbits of revolving double stars, Memoirs of the Royal Astronomical Society, :–. Hertzsprung, E. (). Publikationen des Astrophysikalischen Observatoriums zu Potsdam. Num. . Hoff, H.E. and Geddes, L.A. (). Graphic recording before Carl Ludwig: An historical summary, Archives Internationales d’Histoire des Sciences, :–. Hoff, H.E. and Geddes, L.A. (). he beginnings of graphic recording, Isis, :– . Pt. . Howard, L. (). On a periodical variation of the barometer, apparently due to the influence of the sun and moon on the atmosphere, Philosophical Magazine, :– . Inselberg, A. (). he plane with parallel coordinates, he Visual Computer, :– . Jevons, W.S. (). A serious fall in the value of gold ascertained, and its social effects set fourth, London. Jevons, W.S. (). Graphical method, Principles of Science: A Treatise on Logic and Scientific Method, rd edn, Dover, New York, pp. –. First ed.: ; page numbers from rd Ed. Dover reprint (). Joint Committee on Standards for Graphic Presentation (). Preliminary report published for the purpose of inviting suggestions for the benefit of the committee, Publications of the American Statistical Association, ():–.
A Brief History of Data Visualization
53
Karsten, K.G. (). Charts and Graphs. An Introduction to Graphic Methods in the Control and and Analysis of Statistics, Prentice Hall, New York. Klein, J.L. (). Statistical Visions in Time: A History of Time Series Analysis, – , Cambridge University Press, Cambridge, UK. Kruskal, W. (). Visions of maps and graphs, Proceedings of the International Symposium on Computer-Assisted Cartography, Auto-Carto II, pp. –. . Lallemand, C. (). Les abaques hexagonaux: Nouvelle méthode générale de calcul graphique, avec de nombreux exemples d’application, Ministère des travaux publics, Comité du nivellement général de la France, Paris. Maas, H. and Morgan, M.S. (). Timing history: he introduction of graphical analysis in th century british economics, Revue d’Histoire des Sciences Humaines, :–. Marey, E.-J. (). La méthode graphique dans les sciences expérimentales et principalement en physiologie et en médecine, G. Masson, Paris. Marshall, A. (). On the graphic method of statistics, Journal of the Royal Statistical Society, Jubille volume, –. Maunder, E.W. (). Note on the distribution of sun-spots in heliographic latitude, to , Royal Astronomical Society Monthly Notices, :–. Meyer, D., Zeileis, A. and Hornik, K. (). vcd: Visualizing Categorical Data. R package version .–. Minard, C.J. (). Projet de canal et de chemins de fer pour le transport de pavés à Paris, précédé d’un tableau des progrès de la dépense du pavé de Paris pendant les deux derniers siècles, A. Pihan Delaforest, Paris. École Nationale des Ponts et Chaussées, Paris: ./C. Minard, C.J. (). Tableaux figuratifs de la circulation de quelques chemins de fer, lith. (n.s.). École Nationale des Ponts et Chaussées, Paris: /C, /C. Minard, C.J. (). Des tableaux graphiques et des cartes figuratives, E. hunot et Cie, Paris. École Nationale des Ponts et Chaussées, Paris: /C; Bibliothèque Nationale de France, Paris (Tolbiac): V-. Moseley, H. (). he high frequency spectra of the elements, Philosophical Magazine pp. . (continued, , pp. ). Mouat, F.J. (). History of the Statistical Society of London, Journal of the Royal Statistical Society, Jubille volume, –. Nightingale, F. (). Mortality of the British Army, Harrison and Sons, London. Oresme, N. (). Tractatus de latitudinibus formarum, Padova. British Library, London: IA Q. Oresme, N. (). Nicole Oresme and the Medieval Geometry of Qualities and Motions: A Treatise on the Uniformity and Difformity Known as Tractatus de configurationibus qualitatum et motuum, University of Wisconsin Press, Madison WI. tr.: M. Clagget. Palsky, G. (). Des chiffres et des cartes: naissance et développement de la cartographie quantitative française au XIX e siècle, Comité des Travaux Historiques et Scientifiques (CTHS), Paris.
54
Michael Friendly
Pearson, E.S. (ed) (). he History of Statistics in the th and th Centuries Against the Changing Background of Intellectual, Scientific and Religeous hought, Griffin & Co. Ltd., London. Lectures by Karl Pearson given at University College London during the academic sessions –. Pearson, K. (–). he life, letters and labours of Francis Galton, Cambridge: University Press. Pearson, K. (). Notes on the history of correlation, Biometrika, ():–. Peddle, J.B. (). he Construction of Graphical Charts, McGraw-Hill, New York. Perozzo, L. (). Della rappresentazione grafica di una collettività di individui nella successione del tempo, Annali di Statistica, :–. British Library, London: S.. Petty, W. (). he economic writings of Sir William Petty: together with the observations upon the bills of mortality, he University Press, Cambridge. C.H. Hall (ed) (more probably by Captain John Graunt). Playfair, W. (). Commercial and Political Atlas: Representing, by Copper-Plate Charts, the Progress of the Commerce, Revenues, Expenditure, and Debts of England, during the Whole of the Eighteenth Century, Corry, London. Re-published in Wainer, H. and Spence, I. (eds), he Commercial and Political Atlas and Statistical Breviary, , Cambridge University Press, ISBN ---. Playfair, W. (). Statistical Breviary; Shewing, on a Principle Entirely New, the Resources of Every State and Kingdom in Europe, Wallis, London. Re-published in Wainer, H. and Spence, I. (eds), he Commercial and Political Atlas and Statistical Breviary, , Cambridge University Press, ISBN ---. Playfair, W. (). Letter on our agricultural distresses, their causes and remedies; accompanied with tables and copperplate charts shewing and comparing the prices of wheat, bread and labour, from to . British Library, London: .c.. Porter, T.M. (). he Rise of Statistical hinking –, Princeton University Press, Princeton, NJ. Priestley, J. (). A chart of biography, London. British Library, London: .I.. Priestley, J. (). A New Chart of History, homas Jeffreys, London. British Library, London: Cup..e. (). Quételet, A. (). Recherches sur le penchant au crime aux différens âges, Hayez, Brussels. English translation by Sawyer F. Sylvester. Cincinnati, OH: Anderson Publishing Company, . Quételet, A. (). Sur l’homme et le développement de ses facultés, ou Essai de physique sociale, Bachelier, Paris. English translation, by R. Knox and T. Smilbert, A Treatise on Man and the Development of his Faculties. Edinburgh, ; New York: Burt Franklin, . Riddell, R.C. (). Parameter disposition in pre-Newtonain planetary theories, Archives Hist. Exact Sci., :–. Riedwyl, H. and Schüpbach, M. (). Siebdiagramme: Graphische Darstellung von Kontingenztafeln, Technical Report , Institute for Mathematical Statistics, University of Bern, Bern, Switzerland. Robinson, A.H. (). Early hematic Mapping in the History of Cartography, University of Chicago Press, Chicago.
A Brief History of Data Visualization
55
Royston, E. (). Studies in the history of probability and statistics, III. a note on the history of the graphical presentation of data, Biometrika pp. –. , Pts. and (December ); reprinted In Studies in the History Of Statistics and Probability heory, eds. E.S. Pearson and M.G. Kendall, London: Griffin. Scheiner, C. (–) Rosa ursina sive sol ex admirando facularum & macularum suarum phoenomeno varius, Andream Phaeum, Bracciano, Italy. British Library, London: .l.. Smith, W. (). A delineation of the strata of England and Wales, with part of Scotland; exhibiting the collieries and mines, the marshes and fenlands originally overflowed by the sea, and the varieties of soil according to the substrata, illustrated by the most descriptive names, John Cary, London. British Library, London: Maps .(). Snow, J. (). On the Mode of Communication of Cholera, edn, (n.p.), London. Sobel, D. (). Longitude: he True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time, Penguin, New York. Spence, I. and Garrison, R.F. (). A remarkable scatterplot, he American Statistician, ():–. Stigler, S.M. (). he History of Statistics: he Measurement of Uncertainty before , Harvard University Press, Cambridge, MA. Tartaglia, N.F. (). General Trattato di Numeri et Misure, Vinegia, Venice. British Library, London: .n.–; .e.. Tierney, L. (). LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics, John Wiley and Sons, New York. Tilling, L. (). Early experimental graphs, British Journal for the History of Science, :–. Tute, E.R. (). he Visual Display of Quantitative Information, Graphics Press, Cheshire, CT. Tute, E.R. (). Visual Explanations, Graphics Press, Cheshire, CT. Tukey, J.W. (). he future of data analysis, Annals of Mathematical Statistics, :– and . Tukey, J.W. (). Exploratory Data Analysis, Addison-Wesley, Reading, MA. Tukey, P.A. and Tukey, J.W. (). Graphical display of data sets in or more dimensions, in V. Barnett (ed), Interpreting Multivariate Data, Wiley and Sons, Chichester, U.K. Vauthier, L.L. (). Note sur une carte statistique figurant la répartition de la population de Paris, Comptes Rendus des Séances de l’Académie des Sciences, :–. École Nationale des Ponts et Chaussées, Paris: C. von Huhn, R. (). A discussion of the Eells’ experiment, Journal of the American Statistical Association, :–. von Humboldt, A. (). Sur les lignes isothermes, Annales de Chimie et de Physique, :–. Wainer, H. (). Graphic Discovery: A Trout in the Milk and Other Visual Adventures, Princeton University Press, Princeton, NJ. Wainer, H. and Velleman, P.F. (). Statistical graphics: Mapping the pathways of science, Annual Review of Psychology, :–.
56
Michael Friendly
Wallis, H.M. and Robinson, A.H. (). Cartographical Innovations: An International Handbook of Mapping Terms to , Map Collector Publications, Tring, Herts. Warne, F.J. (). Warne’s Book of Charts, A Special Feature of Warne’s Elementary Course in Chartography, F.J. Warne, Washington, D.C. p. l., charts. cm. Washburne, J.N. (). An experimental study of various graphic, tabular and textual methods of presenting quantitative material, Journal of Educational Psychology, :–, –. Wegman, E.J. (). Hyperdimensional data analysis using parallel coordinates, Journal of the American Statistical Association, ():–. Westergaard, H. (). Contributions to the History of Statistics, P.S. King & Son, London. Whitt, H. (). Inventing sociology: André-Michel Guerry and the Essai sur la statistique morale de la France, Edwin Mellen Press, Lewiston, NY, pp. ix–xxxvii. English translation: Hugh P. Whitt and Victor W. Reinking, Lewiston, N.Y. : Edwin Mellen Press, . Young, F.W. (a). ViSta: he visual statistics system, Technical Report RM -, L.L. hurstone Psychometric Laboratory, UNC. Young, F.W. (b). ViSta: he visual statistics system, Technical Report RM -, L.L. hurstone Psychometric Laboratory, UNC. Young, F.W., Valero-Mora, P. and Friendly, M. (). Visual Statistics: Seeing Data with Dynamic Interactive Graphics, Wiley, New York. Zeuner, G. (). Abhandlungen aus der mathematischen Statistik, Leipzig. British Library, London: .f..
Good Graphics?
II.2
Antony Unwin
2.1
2.2
2.3 2.4
2.5
2.6
2.7
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
58
Content, Context and Construction .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Presentation Graphics and Exploratory Graphics . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
58 59
Background .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
60
History .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Literature . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . The Media and Graphics . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
60 61 62
Presentation (What to Whom, How and Why) . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . Scientiic Design Choices in Data Visualization .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
62 63
Choice of Graphical Form . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Graphical Display Options . . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
64 64
Higher-dimensional Displays and Special Structures .. . . . . . . . . . .. . . . . . . . .
70
Scatterplot Matrices (Sploms) .. .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Parallel Coordinates . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Mosaic Plots . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Small Multiples and Trellis Displays ... . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Time Series and Maps . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
70 70 71 72 74
Practical Advice .. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
76
Software .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Bad Practice and Good Practice (Principles) . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
76 77
And Finally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
77
58
Antony Unwin
Graphical excellence is nearly always multivariate.
2.1
– Edward Tute
Introduction his chapter discusses drawing good graphics to visualize the information in data. Graphics have been used for a long time to present data. Figure . is a scanned image from Playfair’s Commercial and Political Atlas of , reproduced in Playfair (). he fairly continuous increase of both imports and exports, and the fact that the balance was in favour of England from on, can be seen easily. Some improvements might be made, but overall the display is effective and well drawn. Data graphics are used extensively in scientific publications, in newspapers and in the media generally. Many of those graphics do not fully convey the information in the data they are supposed to be presenting and may even obscure it. What makes a graphic display of data bad? More importantly, what makes one good? In any successful graphic there must be an effective blending of content, context, construction and design.
2.1.1
Content, Context and Construction What is plotted comes first, and without content no amount of clever design can bring meaning to a display. A good graphic will convey information, but a graphic is always part of a larger whole, the context, which provides its relevance. So a good graphic will complement other related material and fit in, both in terms of content and also
Figure .. Playfair’s chart of trade between England and Ireland from to
Good Graphics?
59
Figure .. Church attendance (DDB Life Style Survey –)
with respect to style and layout. Finally, if a graphic is constructed and drawn well, it will look good. Figure . shows two similar displays of the same data from the DDB social survey used in Robert Putnam’s book Bowling Alone (Putnam, ). Every year for years, different groups of people were surveyed. Amongst other questions, they were asked how oten they had attended church in the last year. he let-hand graph includes gridlines and a coloured background and uses -D columns to represent the data counts. he right-hand graph sticks to basics. In general, the right-hand display is to be preferred (-D columns can cross gridlines, and zero values would be misleadingly displayed). For these data there is not much to choose between the two representations; both convey the same overall information. he potential weakness in both graphics is the set of categories. Grouping the data together in different ways could give quite different impressions. For a given dataset there is not a great deal of advice which can be given on content and context. hose who know their own data should know best for their specific purposes. It is advisable to think hard about what should be shown and to check with others if the graphic makes the desired impression. Design should be let to designers, though some basic guidelines should be followed: consistency is important (sets of graphics should be in similar style and use equivalent scaling); proximity is helpful (place graphics on the same page, or on the facing page, of any text that refers to them); and layout should be checked (graphics should be neither too small nor too large and be attractively positioned relative to the whole page or display). Neither content nor context nor design receives much attention in books offering advice on data graphics; quite properly they concentrate on construction. his chapter will, too.
Presentation Graphics and Exploratory Graphics here are two main reasons for using graphic displays of datasets: either to present or to explore data. Presenting data involves deciding what information you want to convey and drawing a display appropriate for the content and for the intended audience. You have to think about how the plot might be perceived and whether it will be understood as you wish. Plots which are common in one kind of publication may be unfamiliar to the readers of another. here may only be space for one plot and it may
2.1.2
60
Antony Unwin
be available in print for a very long time, so great care should be taken in preparing the most appropriate display. Exploring data is a much more individual matter, using graphics to find information and to generate ideas. Many displays may be drawn. hey can be changed at will or discarded and new versions prepared, so generally no one plot is especially important, and they all have a short life span. Clearly principles and guidelines for good presentation graphics have a role to play in exploratory graphics, but personal taste and individual working style also play important roles. he same data may be presented in many alternative ways, and taste and customs differ as to what is regarded as a good presentation graphic. Nevertheless, there are principles that should be respected and guidelines that are generally worth following. No one should expect a perfect consensus where graphics are concerned.
2.2
Background
2.2.1
History Data graphics may be found going very far back in history, but most experts agree that they really began with the work of Playfair a little more than years ago. He introduced some modern basic plots (including the barchart and the histogram) and produced pertinent and eye-catching displays (Fig. .). Wainer and Spence recently republished a collection of his works (Playfair, ). Not all his graphics could be described as good, but most were. In the second half of the th century Minard prepared impressive graphics, including his famous chart of Napoleon’s advance on and retreat from Moscow. he French Ministry of Public Works used his ideas to attractive, and presumably pertinent, effect in an annual series of publications (Album de Statistique Graphique) from to , presenting economic data geographically for France. Examples can be found in Michael Friendly’s chapter in this book. In the first half of the last century graphics were not used in statistics as much as they might have been. Interestingly, the second chapter in Fisher’s Statistical Methods for Research Workers in was on diagrams for data, so he, at least, thought graphics important. In Vienna there was a group led by Otto Neurath which worked extensively on pictograms in the s and early s. hey produced some well-crated displays, which were forerunners of the modern infographics. (Whether Fig. . is improved by including the symbols at the top to represent the USA is a matter of taste.) With the advent of computers, graphics went into a relative decline. Computers were initially bad for graphics for two reasons. Firstly, much more complex analytic models could be evaluated and, quite naturally, modelling received a great deal more attention than displaying data. Secondly, only simple and rather ugly graphics could be drawn by early computers. he development of hardware and sotware has turned all this around. In recent years it has been very easy to produce graphics, and far more can be seen than before. Which is, of course, all the more reason to be concerned that graphics be drawn well.
Good Graphics?
61
Figure .. Pictogram by Otto Neurath of the number of cars in the USA and the rest of the world in
, , and
Literature Several authors have written excellent books on drawing good statistical graphics, the best known, justifiably, being Edward Tute. His books (e.g. Tute, ) include many splendid examples (and a few dreadful ones) and describe important principles on how to draw good graphics. Tute criticizes unsuitable decoration and data misrepresentation, but his advice is restricted to representing data properly. Cleveland’s books [for instance, Cleveland () another useful source of advice on preparing data displays] are equally valuable. And this is the way it should be. Statisticians should concentrate on getting the basic statistical display right, and designers may be consulted to produce a polished final version. While there is a place for applied books full of sound practical advice [other useful references include Burn (), Kosslyn (), and Robbins ()], there is also a need for theory to provide formal structures for understanding practice and to provide a foundation from which progress can be made. Graphics must be one of the few areas in statistics where there is little such theory. Bertin’s major work (Semiologie Graphique) from contains a number of interesting ideas and is oten cited, but it is difficult to point to later work that directly extends it. Wilkinson’s Grammar of Graphics has received a lot of attention and been quickly revised in a substantially expanded second edition (Wilkinson, ). If there is little theory, then examples become particularly important to show what can be achieved. he two books by Wainer (, ) contain collections of columns first published in Chance and offer instructive and entertaining examples. Friendly’s Gallery of Statistical Visualization (http://www.math.yorku.ca/SCS/Gallery/) includes many examples, both good and bad, chronicling the history of graphical developments. he websites ASK E.T. (http://www.edwardtute.com) and Junk Charts (http://junkcharts.typepad.com) provide lively discussions and sage advice for par-
2.2.2
62
Antony Unwin
ticular examples. It would be invidious, and perhaps unfair, to single out egregious examples here. Readers should be able to find plenty for themselves without having to look far. 2.2.3
The Media and Graphics Graphical displays of data appear in the press very oten. hey are a subset of infographics, graphical displays for conveying information of any kind, usually discussed under the heading information visualization (Spence, ). Many impressive examples can be found in the New York Times. While there are guidelines which apply to all infographics, this chapter is restricted to the construction of data visualizations. Data displays in the media are used to present summary information, such as the results of political surveys (what proportion of the people support which party), the development of financial measures over time (a country’s trade balance or stock market indices) or comparisons between population groups (average education levels of different sections of the community). here are many other examples. hese topics only require fairly basic displays, so it is not surprising that in the media they are commonly embellished with all manner of decoration and ornamentation, sometimes effectively drawing attention both to the graphic and to its subject, sometimes just making it more difficult to interpret the information being presented. What is surprising is that the graphics are oten misleading or flawed.
2.3
Presentation (What to Whom, How and Why) How is it possible to make a mess of presenting simple statistical information? Surely there is little that can go wrong. It is astonishing just what distortion can be introduced: misleading scales may be employed; -D displays of -D data make it difficult to make fair comparisons; areas which are apparently intended to be proportional to values are not; so much information is crammed into a small space that nothing can be distinguished. While these are some of the technical problems that can arise, there are additional semantic ones. A graphic may be linked to three pieces of text: its caption, a headline and an article it accompanies. Ideally, all three should be consistent and complement each other. In extreme cases all four can tell a different story! A statistician cannot do much about headlines (possibly added or amended by a subeditor at the last minute) or about accompanying articles if he or she is not the first author (in the press the journalist chooses the graphic and may have little time to find something appropriate), but the caption and the graphic itself should be “good”. Some displays in the media highlight a news item or provide an illustration to lighten the text. hese are oten prepared by independent companies at short notice and sold to the media as finished products. Fitting the graphic to its context may be
Good Graphics?
63
awkward. here are displays in scientific publications which are prepared by the authors and should be the product of careful and thorough preparation. In this situation a graphic should match its context well. Whatever kind of data graphic is produced, a number of general principles should be followed to ensure that the graphic is at least correct. Whether or not a graphic is then successful as a display depends on its subject, on its context and on aesthetic considerations. It depends on what it is supposed to show, on what form is chosen for it and on its audience. Readers familiar with one kind of graphic will have no trouble interpreting another example of the same kind. On the other hand, a graphic in a form which is new to readers may lead to unanticipated interpretation difficulties. When someone has spent a long time on a study and further time on the careful preparation of a graphic display to illustrate the conclusions, they are usually astonished when others do not see what they can see. [his effect is, of course, not restricted to drawing graphics. Designers are frequently shocked by how people initially misunderstand their products. How oten have you stared at the shower in a strange hotel wondering how you can get it to work without its scalding or freezing you? Donald Norman’s book (Norman, ) is filled with excellent examples.] Other factors have to be considered as well. A graphic may look different in print than on a computer screen. Complex graphics may work successfully in scientific articles where the reader takes time to fully understand them. hey will not work well as a brief item in a television news programme. On the other hand, graphics which are explained by a commentator are different from graphics in print. If graphics displayed on the Web can be queried (as with some of the maps on http://www.cancer.gov/ atlasplus/, discussed in Sect. ..), then more information can be provided without cluttering the display.
Scientiic Design Choices in Data Visualization Plotting a single variable should be fairly easy. he type of variable will influence the type of graphic chosen. For instance, histograms or boxplots are right for continuous variables, while barcharts or piecharts are appropriate for categorical variables. In both cases other choices are possible too. Whether the data should be transformed or aggregated will depend on the distribution of the data and the goal of the graphic. Scaling and captioning should be relatively straightforward, though they need to be chosen with care. It is a different matter with multivariate graphics, where even displaying the joint distribution of two categorical variables is not simple. he main decision to be taken for a multivariate graphic is the form of display, though the choice of variables and their ordering are also important. In general a dependent variable should be plotted last. In a scatterplot it is traditional to plot the dependent variable on the vertical axis.
2.4
64
2.4.1
Antony Unwin
Choice of Graphical Form here are barcharts, piecharts, histograms, dotplots, boxplots, scatterplots, roseplots, mosaicplots and many other kinds of data display. he choice depends on the type of data to be displayed (e.g. univariate continuous data cannot be displayed in a piechart and bivariate categorical data cannot be displayed in a boxplot) and on what is to be shown (e.g. piecharts are good for displaying shares for a small number of categories and boxplots are good for emphasizing outliers). A poor choice graph type cannot be rectified by other means, so it is important to get it right at the start. However, there is not always a unique optimal choice and alternatives can be equally good or good in different ways, emphasizing different aspects of the same data. Provided an appropriate form has been chosen, there are many options to consider. Simply adopting the default of whatever computer sotware is being used is unlikely to be wise.
2.4.2
Graphical Display Options Scales Defining the scale for the axis for a categorical variable is a matter of choosing an informative ordering. his may depend on what the categories represent or on their relative sizes. For a continuous variable it is more difficult. he endpoints, divisions and tick marks have to be chosen. Initially it is surprising when apparently reliable sotware produces a really bad scale for some variable. It seems obvious what the scale should have been. It is only when you start trying to design your own algorithm for automatically determining scales that you discover how difficult the task is. In Grammar of Graphics Wilkinson puts forward some plausible properties that ‘nice’ scales should possess and suggests a possible algorithm. he properties (simplicity, granularity and coverage, with the bonus of being called ‘really nice’ if zero is included) are good but the algorithm is easy to outwit. his is not to say that it is a weak algorithm. What is needed is a method which gives acceptable results for as high a percentage of the time as possible, and the user must also check the resulting scale and be prepared to amend it for his or her data. Difficult cases for scaling algorithms arise when data cross natural boundaries, e.g., data with a range of to would be easy to scale, whereas data with a range of to would be more awkward. here is a temptation to choose scales running from the minimum to the maximum of the data, but this means that some points are right on the boundaries and may be obscured by the axes. Unless the limits are set by the meaning of the data (e.g. with exam marks from to , neither negative marks nor marks more than are possible – usually!), it is good practice to extend the scales beyond the observed limits and to use readily understandable rounded values. here is no obligatory requirement to include zero in a scale, but there should always be a reason for not doing so; otherwise it makes the reader wonder if some deception is being practiced. Zero is in fact not the only possible baseline or alignment point for a scale, though it is the most common one. A sensible alignment value for ratios is one, and financial series
Good Graphics?
65
are oten standardized to all start at . In Fig. . the cumulative times for all the riders who finished the Tour de France cycle race in are plotted. he data at the end of each stage have been aligned at their means. he interest lies in the differences in times between the riders, not so much in their absolute times. Figure . shows histograms for the Hidalgo stamp thickness data (Izenman and Sommer, ). he first uses default settings and shows a skew distribution with possibly a second mode around .. he second has rounded endpoints and a rounded binwidth and shows stronger evidence for the second mode. he third is drawn so that each distinct value is in a different bin (the data were all recorded to a thousandth of a millimetre). It suggests that the first mode is actually made of up to two groups and that there may be evidence for several additional modes to the right. It also reveals that rounded values such as ., ., . . . , . occur relatively more frequently. Izenman and Sommer used the third histogram in their paper. What the data repre-
Figure .. hree different histograms of the Hidalgo stamp thickness data, all with the same
anchorpoint but with different binwidths. he horizontal scales are aligned and the total area of each display is the same (note the different frequency scales). Source: Izenman and Sommer ()
66
Antony Unwin
sent and how they are collected should be taken into account when choosing scales. Asymptotic optimality criteria only have a minor role to play. While Fig. . shows the importance of choosing binwidths carefully, it also illustrates some display issues. he horizontal value axis is clearly scaled, but it would surely be nicer if it extended further to the right. More importantly, the comparison in Fig. . ideally requires that all three plots be aligned exactly and have the same total area. Not all sotware provides these capabilities. Graphics should be considered in their context. It may be better to use a scale in one graphic that is directly comparable with that in another graphic instead of individually scaling both. Common scaling is used in one form or another in Figs. ., . and .. It is one thing to determine what scale to use, but quite another to draw and label the axes. Too many labels make a cluttered impression; too few can make it hard for the reader to assess values and differences. (Note that it is not the aim of graphics to provide exact case values; tables are much better for that.) Tick marks in between labels oten look fussy and have little practical value. In some really bad situations, they can obscure data points.
Sorting and Ordering he effect of a display can be influenced by many factors. When more than one variable is to be plotted, the position or order in which they appear in the graphic makes a difference. Examples arise with parallel coordinate plots, mosaicplots and matrix visualizations, all discussed in other chapters. Within a nominal variable with no natural ordering, the order in which the categories are plotted can have a big effect. Alphabetic ordering may be appropriate (a standard default, which is useful for comparison purposes), or a geographic or other grouping (e.g. shares by market sector) might be relevant. he categories could be ordered by size or by a secondary variable. Figure . shows two barcharts of the same data, the numbers in each class and in the crew on the Titanic. he second ordering would be the same in any language, but the first would vary (for instance, Crew, First, Second, hird in English).
Figure .. Numbers of passengers and crew who travelled on the Titanic, by class, ordered
alphabetically (in German) and by status. Source: Dawson ()
Good Graphics?
67
Adding Model or Statistical Information – Overlaying (Statistical) Information Guides may be drawn on a plot as a form of annotation and are useful for emphasizing particular issues, say which values are positive or negative. Sloping guides highlight deviations from linearity. Fitted lines, for instance polynomial regression or smoothers, may be superimposed on data not only to show the hypothesized overall structure but also to highlight local variability and any lack of fit. Figure . plots the times from the first and last stages of a -km road race. A lowess smoother has been drawn. It suggests that there is a linear relationship for the faster runners and a flat one for the slower ones. When multiple measurements are available, it is standard practice in scientific journals to plot point estimates with their corresponding confidence intervals. ( % confidence intervals are most common, though it is wise to check precisely what has been plotted.) Figure . displays the results of a study on the deterioration of a thin plastic over time. Measurements could only be made by destructive testing, so all measurements are of independent samples. he high variability at most of the time points is surprising. Adjacent means have been joined by straight lines. A smoothing function would be a better alternative, but such functions are not common for this kind of plot. As the measurement timepoints are far apart and as there is only one dataset, there is no overlapping here. hat can very oten be a serious problem. Overlaying information, whether guides or annotation, can lead to overlapping and cluttered displays. Good solutions are possible but may require individual adjustments depending on the shape of the data. A well-spaced and informative display at one size may appear unsatisfactory and unclear when shrunk for publication.
Figure .. Times for runners for the last stage of a road race vs. their times for the first stage, with
a lowess smoother. Default scales from R have been used. Source: Everitt ()
68
Antony Unwin
Captions, Legends and Annotations Ideally, captions should fully explain the graphic they accompany, including giving the source for the data. Relying on explanations in the surrounding text rarely works. Ideals cannot always be met and very long captions are likely to put off the reader, but the whole point of a graphic is to present information concisely and directly. A compromise where the caption outlines the information in the graphic and a more detailed description can be found in the text can be a pragmatic solution. Graphics which require extensive commentary may be trying to present too much information at one go. Legends describe which symbols and/or colours refer to which data groups. Tute recommends that this information be directly on the plot and not in a separate legend, so that the reader’s eyes do not have to jump backwards and forwards. If it can be done, it should be. Annotations are used to highlight particular features of a graphic. For reasons of space there cannot be many of them and they should be used sparingly. hey are useful for identifying events in time series, as Playfair did (Playfair, ), or for drawing attention to particular points in scatterplots. Union estimates of protest turnout in Fig. . are larger than the police estimates by roughly the same factor, except for the two extreme exceptions, Marseille and Paris, where the disagreement is much greater.
Positioning in Text Keeping graphics and text on the same page or on facing pages is valuable for practical reasons. It is inconvient to have to turn pages back and forth because graphics and the text relating to them are on different pages. However, it is not always possible to avoid this. Where graphics are placed on a given page is a design issue.
Size, Frames and Aspect Ratio Graphics should be large enough for the reader to see the information in them clearly and not much larger. his is a rough guideline, as much will depend on the surrounding layout. Frames may be drawn to surround graphics. As frames take up space and add to the clutter, they should best only be used for purposes of separation, i.e. separating the graphic from other graphics or from the text. Aspect ratios have a surprisingly strong effect on the perception of graphics. his is especially true of time series. If you want to show gradual change, grow the horizontal axis and shrink the vertical axis. he opposite actions will demonstrate dramatic change. For a scatterplot example, see Fig. ., which displays the same data as Fig. .. here is useful advice on aspect ratios in Cleveland (), especially the idea of ‘banking to degrees’ for straight lines.
Colour Colour should really have been discussed much earlier. It is potentially one of the most effective ways of displaying data. In practice it is also one of the most difficult to get right. A helpful check for colour schemes for maps, Colorbrewer by Cynthia
Good Graphics?
69
Figure .. Average chemical deterioration and % confidence intervals, measured at different time
points. here were either or measurements at each time point. Source: Confidential research data
Figure .. Union estimates (in thousands) of the protester turnout in various French cities in the
spring of plotted against police estimates (also in thousands). Source: Le Monde ..
Figure .. he same plot as Fig. ., drawn with different aspect ratios. Data are times for runners for
the last stage of a road race vs. their times for the first stage
70
Antony Unwin
Brewer, can be found at http://colorbrewer.org. Colorbrewer can give suggestions for colour schemes that both blend well and distinguish between different categories. here remain many factors in the choice of colour which have to be borne in mind: some people are colour blind; colours have particular associations (red for danger or for losses); colours may not be reproduced in print the way they were intended; and colour can be a matter of personal taste. Colour is discussed in more detail in other Handbook chapters.
2.5
Higher-dimensional Displays and Special Structures
2.5.1
Scatterplot Matrices (Sploms) Plotting each continuous variable against every other one is effective for small numbers of variables, giving an overview of possible bivariate results. Figure . displays data from emissions tests of cars sold in Germany. It reveals that engine size, performance and fuel consumption are approximately linearly related, as might be expected, that CO measurements and fuel consumption are negatively correlated in batches, which might not be so expected, and that other associations are less conclusive. Packing so many plots into a small space it is important to cut down on scales. Placing the variable names on the diagonal works well, and histograms of the individual variables could also be placed there.
2.5.2
Parallel Coordinates Parallel coordinate plots (Inselberg, ) are valuable for displaying large numbers of continuous variables simultaneously. Showing so much information at once has several implications: not all information will be visible in any one plot (so that several may be needed); formatting and scaling will have a big influence on what can be seen (so that there are many choices to be made); and some overlapping is inevitable (so that α-blending or more sophisticated density estimation methods are useful). Figure . plots the cumulative times of the cyclists at the ends of the stages of the Tour de France. he axes all have the same scale, so that differences are comparable. he best riders take the shortest time and are at the bottom of the plot. he axes have been aligned at their means, as without some kind of alignment little could be seen. α-blending has been applied to reduce the overprinting in the early sprint stages where all riders had almost the same times. If more α-blending is used, then the individual lines for the riders in the later stages of the race become too faint. his single display conveys a great deal about the race. In the early stages, at most a few minutes separates the riders. On the mountain stages there are much larger differences and individual riders gain both time and places (where a line segment
Good Graphics?
71
Figure .. A scatterplot matrix of the five main continuous variables from a car emissions dataset
from Germany. Source: http://www.adac.de, March
crosses many others downwards). Note that there are relatively few line crossings over the later stages of the race, which means, perhaps surprisingly, that not many riders changed their race ranking. his graphic might be improved in a number of ways: the axes could be labelled (though there is little space for this); the vertical axes could be drawn less strongly; scale information could be added (the range of the vertical axes is about h, though precise values would be better read off a table of results); and the level of α-blending might be varied across the display. Figure . shows a special form of parallel coordinate plot. Usually each axis has its own scale and there is no natural ordering of the axes. Other examples of parallel coordinate plots can be found in other chapters of the Handbook.
Mosaic Plots Mosaic plots display the counts in multivariate contingency tables. here are various types of mosaicplot (Hofmann, ) and a -D example of a doubledecker plot is displayed in Fig. .. he data are from a study of patterns of arrest based on
2.5.3
72
Antony Unwin
Figure .. Cumulative times for riders in Tour de France for the stages. he axes have
a common scale and are aligned by their means. Each vertical line represents a stage, and they have been plotted in date order. Source: http://www.letour.fr
cases in Toronto. Each column represents one combination of the four binary variables Gender, Employed, Citizen and Colour. he width of a column is proportional to the number with that combination of factors. hose stopped who were not released later have been highlighted. Over % of those stopped were male. Some of the numbers of females in the possible eight combinations are too small to draw firm conclusions. Each pair of columns represents the variable colour, and the proportion not released amongst the males is lower amongst the whites for all combinations of other factors. he general decline in the level of highlighting across the male columns shows that the proportion not released is lower if the person is a citizen and lower still if they are employed. Figure . shows the difficulties in displaying data of this kind in a graphic for presentation. Colour, aspect ratio and size can make a big difference, but labelling is the main problem. 2.5.4
Small Multiples and Trellis Displays One way to avoid overloading a single large plot with information is to use a set of smaller, comparable plots instead. his can be effective for subgroup analyses [e.g. trellis displays for conditioning (Becker et al., )] or for geographic data [cf. micromaps Carr ()]. A simple example is given in Fig. .. he boxplots on their
Good Graphics?
73
Figure .. A doubledecker plot of Toronto arrest data. Source: Fox ()
own show that diesel cars have generally lower fuel consumption (in Europe consumption is measured in litres/ km). he barchart on the let shows that little attention should be paid to the (Natural) Gas and Hybrid groups as few of these cars were measured. Should these two groups have been let out or perhaps be replaced by dotplots? Small groups are always a problem. It should also be noted that the units for natural gas cars are different (kg/ km) from the others. Small multiples can work well, but careful captioning is necessary to ensure that it is clear which smaller plot is which, and common scaling is obviously essential. Figure . is a trellis display of emissions data for the petrol or diesel cars. hey have been grouped by engine type (rows) and engine size (columns). An equal count grouping has been used for engine size, which is why the shaded parts of the cc bars have different lengths. Engine size seems to make little difference as the plots in each row are similar to one another. he type of engine makes more difference, with diesel
Figure .. Boxplots of fuel consumption by engine type data from Germany. he barchart shows the
relative numbers of cars involved. he total number was . Source: http://www.adac.de, March
74
Antony Unwin
Figure .. Trellis display of car emissions data from Germany. Each panel is a scatterplot of two pollution measures. Rows: type of engine; columns: engine size. Source: http://www.adac.de, March
engines in particular being different from the other two types. here are a few local outliers amongst the petrol cars. When several plots of the same kind are displayed, they can be plots of subsets of the same data, as in trellis displays, or plots of different variables for the same dataset, as in a parallel coordinates plot. It should always be obvious from the display which is the case. 2.5.5
Time Series and Maps Time Series Time series are special because of the strict ordering of the data, and good displays respect temporal ordering. It is useful to differentiate between value measurements at particular time points (e.g. a patient’s weight or a share price) and summary measurements over a period (e.g. how much the patient ate in the last month or how many shares were traded during the day). Time scales have to be carefully chosen. he choice of time origin is particularly important, as anyone who looks at the advertised performance of financial funds will know. Time points for value measurements may not match the calendar scale (e.g.
Good Graphics?
75
Figure .. Weekly Dow Jones Industrial Average: a Four years from to . b Six months from
July to December . he maximum vertical axis value on the let is over four times the maximum on the right
daily share prices only being available on days the market is open). Time units for summary measurements may be of unequal length (e.g. months). he time period chosen and the aspect ratio used for a time series plot can make a big difference in the interpretation of the data (Fig. .). If several time series are plotted in the same display, then it is necessary to ensure that they are properly aligned in time (e.g. two annual economic series may be published at different times of the year), that their vertical scales are matched (the common origin and the relative ranges) and that they can be distinguished from one another. Depending on the data, this can be tricky to do successfully.
Maps Geographic data are complex to analyse, though graphical displays can be very informative. Bertin discussed many ways of displaying geographic data in his book, and MacEachren’s book contains a lot of sound advice (MacEachren, ), though more from a cartographic point of view. he main problems to be solved lie in the fact that areas do not reflect the relative importance of regions (e.g. Montana has fewer people than New York City but is much bigger) and spatial distance is not directly associated with similarity or nearness (e.g. where countries are divided by natural borders, like mountain ranges). here is a substantial research literature in geography on these and other display issues, such as how to use colour scales to show values (‘choropleth maps’) and how to choose colour schemes (e.g. Colorbrewer referred to above). Some instructive examples can be found in the cancer atlas maps of US health authorities on the Web and in the book by Devesa et al. (). Figure . shows that cancer rates are highest along the East Coast and lowest in the Midwest. State Economic Areas (SEAs) have been chosen because using states oversmooths the data (consider Nevada in the West with its high cancer rate around Las Vegas, but its lower rate elsewhere), while using counties undersmooths. he map on the website is in colour, on a scale from deep red for high rates to dark blue for low. Naturally, this would not reproduce well in a grey-scale view, so the webpage provides the alternative version that is used here. Offering multiple versions of the same image on the Web is readily possible but not oten done. his is one of several reasons why the cancer atlas webpages are exemplary.
76
Antony Unwin
Figure .. Cancer mortality rates for white males in the USA between and by State
Economic Area. he scale has been chosen so that each interval contains % of the SEAs. Source: http://www.cancer.gov/atlasplus/
2.6
Practical Advice
2.6.1
Software For a long time all graphics had to be prepared by dratsmen by hand. he volumes of the Album de Statistique Graphique produced towards the end of the th century contain many exceptional displays which must have taken much painstaking preparation. Such graphics may be individually designed with special features for the particular data involved. Nowadays graphics are produced by sotware, and this has tended to mean that certain default displays are adopted by many as a matter of course. If it takes a few minutes to prepare a graphic that is standard in your field, why bother to prepare something novel? his has advantages – standards avoid possible gross errors and are readily understood by readers familiar with them – and disadvantages – not all data fit the existing standards and interesting new information may be obscured rather than emphasized by a default display. As sotware becomes more sophisticated and user interfaces become more intuitive, this may change. Currently (in ), there are sotware packages which give users substantial control over all aspects of the displays they wish to draw, but these are still only for experts in the
Good Graphics?
77
sotware (Murrell, ). It is reasonable to assume that there will be a steady progression to a situation where even non-experts will be able to draw what they wish. Whether good graphics are the result will depend on the users’ statistical good sense and on their design ability. Like the quality of a scientific article, the quality of a data visualization graphic depends on content and presentation. How has the quality of scientific articles changed since scientists have been able to prepare their own drats with sophisticated text preparation sotware?
Bad Practice and Good Practice (Principles)
2.6.2
Sometimes it is easier to see what has gone wrong than to explain how to do something right. Take the simple task of preparing a barchart to display univariate categorical data. What could possibly go wrong? he bars may be too thin (or too fat); the gaps between the bars may be too narrow (or too wide): the labelling of the bars may be unclear (because it is difficult to fit long category names in); the order of the bars may be confusing; the vertical scale may be poorly chosen; there may be superfluous gridlines; irrelevant -D effects may have been used; colours or shading may have been unnecessarily added; or the title may be misleading and the caption confusing. Doubtless there are even more ways of ruining a barchart. It is not possible to give rules to cover every eventuality. Guiding principles like those outlined in this chapter are needed.
And Finally he lack of formal theory bedevils good graphics. he only way to make progress is through training in principles and through experience in practice. Paying attention to content, context and construction should ensure that sound and reliable graphics are produced. Adding design flair aterwards can add to the effect, so long as it is consistent with the aims of the graphic. Gresham’s Law in economics states that ‘bad money drives out good.’ Fortunately this does not seem to apply to graphics, for while it is true that there are very many bad graphics displays prepared and published, there are also many very good ones. All serious data analysts and statisticians should strive for high standards of graphical display.
References Becker, R., Cleveland, W. and Shyu, M.-J. (). he visual design and control of trellis display, JCGS :–. Burn, D. (). Designing effective statistical graphs, in C. Rao (ed), Handbook of Statistics, Vol. , Elsevier, pp. –. Carr, D.B. (). Designing linked micromap plots for states with many counties, Statistics In Medicine :–.
2.7
78
Antony Unwin
Cleveland, W. (). he Elements of Graphing Data, revised edn, Hobart Press, Summit, New Jersey, USA. Dawson, R. (). he ‘unusual episode’ data revisited, Journal of Statistics Education (Online) (). Devesa, S., Grauman, D., Blot, W., Pennello, G., Hoover, R. and Fraumeni, J.J. (). Atlas of cancer mortality in the United States, -, US Govt Print Off, Washington, DC. Everitt, B. (). Cluster Analysis, rd edn, Edward Arnold, London. Fox, J. (). Effect displays in r for generalised linear models, Journal of Statistical Sotware (). Hofmann, H. (). Exploring categorical data: interactive mosaic plots, Metrika ():–. Inselberg, A. (). Don‘t panic . . . do it in parallel, Computational Statistics ():–. Izenman, A. and Sommer, C. (). Philatelic mixtures and multimodal densities, Journal of the American Statistical Association ():–. Kosslyn, S. (). Elements of Graph Design, Freeman, New York. MacEachren, A. (). How Maps Work, Guildford Press, New York. Murrell, P. (). R Graphics, Chapman & Hall, London. Norman, D. (). he Design of Everyday hings, Doubleday, New York. Playfair, W. (). Playfair’s Commercial and Political Atlas and Statistical Breviary, Cambridge, London. Putnam, R. (). Bowling Alone, Touchstone, New York. Robbins, N. (). Creating More Effective Graphs, John Wiley. Spence, R. (). Information Visualization, Adison-Wesley, New York. Tute, E. () he Visual Display of Quantitative Information, nd edn, Graphic Press, Cheshire, Connecticut. Wainer, H. (). Visual Revelations, Springer, New York. Wainer, H. (). Graphic Discovery: a Trout in the Milk and other Visual Adventures, Princeton UP. Wilkinson, L. (). he Grammar of Graphics, nd edn, Springer, New York.
Static Graphics
II.3
Paul Murrell
3.1
3.2
3.3
3.4
3.5
Complete Plots . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
81
Sensible Defaults .. .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . User Interface . . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
82 84
Customization .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
84
Setting Parameters . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Arranging Plots .. .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Annotation . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . The User Interface . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
84 87 88 92
Extensibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
92
Building Blocks . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Combining Graphical Elements . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . The User Interface . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
93 97 98
Other Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
98
3-D Plots .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Speed . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Output Formats . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Data Handling . . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
98 98 99 99
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
100
80
Paul Murrell
his chapter describes the requirements for a modern statistical graphics system for the production of static plots. here is a discussion of the production of complete plots, customizing plots, adding extra output to plots and creating entirely new plots. Statistical graphics is described as an extension of a general graphics language. here is an emphasis on the importance of support for sophisticated graphics facilities such as semitransparent colours, image compositing operators and the complex arrangement of graphical elements.
Static displays of information continue to be the primary graphical method for the display and analysis of data. his is true both for presentation purposes, where the vast majority of data displays produced for articles and reports are still static in nature, and for data exploration, where many important statistical discoveries have been made based simply on static displays (e.g. Cleveland’s barley data discovery using Trellis plots; Cleveland, ). he recent advances in dynamic and interactive displays (e.g. Swayne et al., ; heus, ) provide us with wonderful additional tools, but static displays still play a fundamental role in the presentation of data. here are very many sotware packages (some of them statistical) that provide ways to produce static displays of data. his is good, because these computer programs allow us to produce more complex graphics, and graphics in greater volumes, than was ever possible when working just with pen and paper. But how good is the sotware for displaying data? More importantly, how good could the sotware be? What should we expect from our statistical graphics sotware? his chapter addresses these questions by discussing the important features which sotware for the static display of data should provide. In addition, there are descriptions of ways to provide those features. For each topic, there will be an abstract discussion of the issue followed by concrete examples implemented in R (R Development Core Team, ). he use of R is natural for me due to my personal familiarity with the system, but it is also justified by the fact that R is widely acknowledged as being pretty good at producing static displays of data, and, to my knowledge, some of the ideas can only be demonstrated in R.
The Grammar of Graphics A comprehensive overview of statistical graphics is provided by Wilkinson’s Grammar of Graphics (Wilkinson, , ). Wilkinson outlines a system in which statistical graphics are described in a high-level, abstract language and which encompasses more than just static graphical displays. his chapter provides a different view, where statistical graphics is seen as an extension of a general graphics language like PostScript (Inc., ) or SVG (Ferraiolo et al., ). his view is lower level, more explicit about the basic graphical elements which are drawn and more focused on static graphics. To emphasize the difference, consider a simple barplot of birth rate for three different types of government (Fig. .). A Grammar of Graphics description (or part
Static Graphics
81
Figure .. A simple barchart of birth rate for three different types of government
thereof) for the barplot would be a statement of the following form (from p. of the Grammar of Graphics, st edn. Wilkinson, ): FRAME: gov*birth GRAPH: bar() A description consistent with this chapter would involve a description of the coordinate systems and graphical shapes that make up the plot. For example, the barplot consists of a plotting region and several graphical elements. he plotting region is positioned to provide margins for axes and has scales appropriate to the range of the data. he graphical elements consist of axes drawn on the edges of the plotting region, plus three rectangles drawn relative to the scales within the plotting region, with the height of the rectangles based on the birth rate data.
Complete Plots We will start with the most obvious feature of statistical graphics sotware: the user should be able to produce graphical output. In other words, the user should be able to draw something. In most cases, the user will want to draw some sort of plot consisting of axes, labels and data symbols or lines to represent the data. Figure . shows an example consisting of a basic scatterplot. his is one distinction between statistical graphics sotware and a more general graphics language such as PostScript. he user does not just want to be able to draw
3.1
82
Paul Murrell
Figure .. A basic scatterplot consisting of axes, labels and data symbols
lines and rectangles (though he may also want to do that; see Sect. ..). he user wants to be able to create an entire plot. To be even more explicit, the user wants to be able to draw an entire plot with a single command (or via a single menu selection). his is so far quite uncontroversial, and all statistical sotware packages provide this feature in one way or another (though they may differ in terms of the range of different sorts of plots that can be produced). In R, the following command usually does the trick (where the variable somedata contains the data values to plot). > plot(somedata)
3.1.1
Sensible Defaults Take another look at the basic plot in Fig. .. As we have mentioned, it consists of a standard set of components: axes, labels and data symbols. But there are other important aspects to this plot. For a start, these components are all in sensible locations; the title is at the top and, very importantly, the data symbols are at the correct locations relative to the axes (and the scales on the axes ensure that there is sufficient room for all of the data points). Some of these aspects are inevitable; no one would use a program that drew data symbols in the wrong locations or created axis scales so that none of the data could be seen. However, there are many aspects that are less obvious.
Static Graphics
83
Why should the title be at the top? Did you notice that the title uses a sans serif font? Why is that? Something else the sotware has done is to position the tick marks at sensible locations within the range of the data. Also, the axes have their tick marks and tick labels pointing away from the region where the data are plotted (other sotware may do this differently). Does that matter? In some of these cases, there are clear reasons for doing things a certain way (e.g. to improve clarity or visual impact; Cleveland, , ; Robbins, ; Tute, ). In other cases, the choice is more subjective or a matter of tradition. he main point is that there are a number of ways that the sotware could do these things. What is important is that the sotware should provide a good default choice.
Trellis Plots A good example of a graphics system that provides sensible defaults is the Trellis system (Becker et al., ). he choice of default values in this system has been guided by the results of studies in human perception (Cleveland and McGill, ) so that the information within a plot will be conveyed quickly and correctly to the viewer. In R, the lattice package (Sarkar, ) implements Trellis plots. Figure . shows a Trellis version of a basic scatterplot. One subtle, but well-founded, difference with Fig. . is the fact that the labels on the tick marks of the y-axis are horizontal so
Figure .. A basic Trellis scatterplot, which has a different default appearance from the scatterplot in
Fig. .
84
Paul Murrell
that they are easier to read. he subtitle of Fig. . is also more heavily emphasized by using a bold font face. he Trellis defaults extend to selections of plotting symbols and colours in plots of multiple data series, which are chosen so that different data series can be easily distinguished by the viewer. 3.1.2
User Interface A sometimes controversial aspect of statistical graphics sotware is the user interface. he choice is between a command line, where the user must type textual commands (or function calls), and a graphical user interface (GUI), consisting of menus and dialogue boxes. A batch system is considered to be a command-line interface; the important point is that the user has to do everything by typing on the keyboard rather than by pointing and clicking with a mouse. Oten both a command line and a GUI will be offered. he interface to a piece of sotware is conceptually orthogonal to the set of features that the sotware provides, which is our main focus here. Nevertheless, in each section of this chapter we will briefly discuss the user interface because there are situations where the interface has a significant impact on the accessibility of certain features. For the purpose of producing complete plots, the choice of user interface is not very important. Where one system might have an option on a GUI menu to produce a histogram, another system can have a command or function to do the same thing. With R, the standard interface is a command line, but a number of GUI options exist, notably Rcmdr (Fox, ), JGR (Helbig et al., ) and the standard GUI on the Mac platform (Iacus and Urbanek, ).
3.2
Customization Let us assume that your statistical sotware allows you to produce a complete plot from a single command and that it provides sensible defaults for the positioning and appearance of the plot. It is still quite unlikely that the plot you end up with will be exactly what you want. For example, you may want a different scale on the axes, or the tick marks in different positions, or no axes at all. Ater being able to draw something, the next most important feature of statistical graphics sotware is the ability to control what gets drawn and how it gets drawn.
3.2.1
Setting Parameters For any particular piece of output, there will be a number of free parameters that must be specified. As a very basic example, it is not sufficient to say something like ‘I want to draw a line’; you must also specify where the line should start and where it should end. You might be surprised how many free parameters there are in even simple cases like this; in order to fully specify the drawing of a single straight line, it is necessary to provide not only a start and end point, but a colour, a line style (perhaps
Static Graphics
85
dashed rather than solid), how thick to draw the line and even a method for how to deal with the ends of the line (should they be rounded or square?). When producing plots, you deal with more complex graphical output than just a single line, and more complex graphical components have their own sets of parameters. For example, when drawing an axis, one parameter might control the number of tick marks on the axis and another might control the text for the axis label. When drawing a complete plot, an important parameter is the data to plot(!), but there may also be parameters to control whether axes are drawn, whether a legend (or key) is provided, and so on. Wherever there is a parameter to control some aspect of graphical output, the user should have the ability to provide a value for that parameter. In R, each graphics function provides a set of parameters to control aspects of the output. he following code shows how a plot can be created with no axes and no labels by specifying arguments for axes and ann respectively. A line is added to the plot with control over its location and its colour, line type and line width. > plot(1:10, axes=FALSE, ann=FALSE) > lines(1:10, col="red", lty="dashed", lwd=3)
Graphical Parameters here is a common set of ‘graphical’ parameters that can be applied to almost any graphical output to affect the appearance of the output. his set includes such things as line colour, fill colour, line width, line style (e.g. dashed or solid) and so on. his roughly corresponds to the concept of graphics state in the PostScript language. In order to be able to have complete control over the appearance of graphical output, it is important that statistical graphics sotware provides a complete set of graphical parameters. Examples of parameters that may sometimes be overlooked
Figure .. Line join and line ending styles. hree thick lines have been drawn with different line end
and line join styles. he top line has ‘square’ ends and ‘mitre’ joins, the middle line has ‘round’ ends and ‘round’ joins, and the bottom line has ‘butt’ ends and ‘bevel’ joins. In each case, the three points that the line goes through are indicated by black circles
86
Paul Murrell
are semitransparent colours, line joins and endings (Fig. .) and full access to a variety of fonts. Edward Tute has recommended (Tute, ) the use of professional graphics sotware such as Adobe Illustrator to achieve quality results, but even better is the ability to provide the control within the statistical graphics sotware itself. In R there is a large set of graphical parameters that allow control over many aspects of graphical output, such as colours, line types and fonts (see the previous example code demonstrating the control of colour, line type and line width), but this could be extended further to include any of the basic drawing parameters and operators that you will find in a sophisticated graphics language such as SVG. Examples are gradient fills (where a region is filled with a smoothly varying colour), general pattern fills and composition of output. An example of the use of composition operators is the addition of a legend to a plot, both of which have a transparent background, but where the plot has grid lines. If we do not want the grid lines to appear in the legend background, one way to achieve
Figure .. Composing graphical elements on a white background. here are three elements being
composed: two legends, one with a transparent background and one with a white background (top let), and a plot with a transparent background (top right). In the bottom let, the legend with a transparent background is drawn over the plot and the grid lines in the plot are visible behind the legend. In the bottom right, the legend with a white background is drawn over the plot and the grid lines in the plot are not visible behind the legend
Static Graphics
87
that is to combine the legend with the plot in such a way that the legend output completely replaces the plot output over the region that the legend is drawn. Figure . shows how just drawing the legend on top of the plot produces the wrong result (there are grid lines visible behind the legend). Using an opaque background in the legend does the job as long as we can anticipate the background colour that the overall plot will be drawn on (Fig. .). However, this is not a good general solution because it fails badly if a different background colour is encountered (Fig. .). A general solution involves more complex image manipulations, such as negating the alpha channel
Figure .. Composing graphical elements on a grey background to show that the use of an opaque
background for a legend (as in Fig. .) is not suitable if the background of the final image is a different colour. here are three elements being composed: two legends, one with a transparent background and one with a white background (top let in Fig. .), and a plot with a transparent background (top right in Fig. .). In the top let in this figure, the legend with a white background is drawn over the plot and the grid lines in the plot are not visible behind the legend, but the white background of the legend does not match the background of the plot, so the result is unpleasant. In the top right, the legend with a transparent background has had its alpha channel (opacity) negated, so that the background is the only part that is opaque. In the bottom let, the negated legend is composited with the plot using an ‘out’ operator, thereby creating a ‘hole’ in the plot. In the bottom right, the (untransformed) legend with a transparent background is drawn over the plot with a hole and the grid lines in the plot are not visible behind the legend, but the background of the final image is still grey, which is the desired result
88
Paul Murrell
(inverting the opacity) of the legend and using a Porter–Duff (Porter and Duff, ) ‘out’ compositing operator to create a ‘hole’ for the legend within the plot (Fig. .). 3.2.2
Arranging Plots Where several plots are produced together on a page, a new set of free parameters becomes available, corresponding to the location and size of each complete plot. It is important that statistical graphics sotware provides some way to specify an arrangement of several plots. In R, it is easy to produce an array of plots all of the same size, as shown by the code below. > par(mfrow=c(2, 2))
It is also possible to produce arrangements where plots have different sizes. he following code gives a simple example (Fig. .): > layout(rbind(c(1, 2), c(0, 0), c(3, 3)), heights=c(1.5, lcm(0.5), 1))
he idea of arranging several plots can be generalized to the arrangement of arbitrary graphical elements; we will discuss this more in Sect. .. 3.2.3
Annotation A more complex sort of customization involves the addition of further graphical output to a plot. For example, it can be useful to add an informative label to one or more data symbols in a plot.
Graphical Primitives he first requirement for producing annotations is the ability to produce very basic graphical output, such as simple text labels. In this way, statistical graphics sotware needs to be able to act like a generic drawing program, allowing the user to draw lines, rectangles, text and so on. In other words, it is not good if the sotware can ‘only’ draw complete plots. In R, there are functions for drawing a standard set of graphical primitives. he following code demonstrates how rectangles, lines, polygons and text can be added to a basic plot (Fig. .): > x <- rnorm(20) > plot(x) > polygon(c(1, 1:20, 20), c(0, x, 0), col="grey", border=NA) > rect(1, -0.5, 20, 0.5, col="white", lty="dotted") > lines(x) > points(x, pch=16) > text(c(0.7, 20.3), 0, c("within", "control"), srt=90)
Static Graphics
89
Figure .. Two arrangements of multiple plots on a page. In the top example, all of the plots have the
same size; in the bottom example, several plots of different sizes are arranged
90
Paul Murrell
Figure .. Basic scatterplot with extra rectangles, lines, polygons and text added to it
Some slightly more complex primitives (not currently natively supported by R) are spline curves, arbitrary paths (as in PostScript or SVG) and polygons with holes, which are useful for drawing maps. An example of a polygon with a hole is an island within a lake within an island, where both islands are part of the same country or state and so are usefully represented as a single polygon.
Coordinate Systems One of the most important and distinctive features of statistical graphics sotware is that it is not only capable of producing many pieces of graphical output at once (lots of lines, text, and symbols that together make up a plot), but that it is also capable of positioning the graphical output within more than one coordinate system. Here are some examples (Fig. .): he title of a plot might be positioned halfway across a page. hat is, the title is positioned relative to a ‘normalized’ coordinate system that covers the entire page, where the location corresponds to the let edge of the page and the location corresponds to the right edge. he data symbols in a scatterplot are positioned relative to a coordinate system corresponding to the range of the data that only covers the area of the page bounded by the plot axes. he axis labels might be positioned halfway along an axis. hat is, the axis labels are positioned relative to a ‘normalized’ coordinate system that only covers the area of the page bounded by the plot axes.
Static Graphics
91
Figure .. Two of the coordinate systems involved in producing a simple scatterplot. A ‘normalized’
coordinate system that covers the whole page is used to centre the plot title, and a coordinate system based on the range of the data that covers the plot region is used to position the data symbols
Many users of statistical graphics sotware produce a plot and then export it to a format which can be easily edited using third-party sotware (e.g. export to WMF and edit using Microsot Office products). his has the disadvantage that the coordinate systems used to produce the plot are lost and cannot be used to locate or size annotations. Furthermore, it makes it much harder to automate or programmatically control the annotation, which is essential if a large number of plots are being produced. When it comes to annotating a plot, it is important that output can be added relative to the coordinate systems which were used to draw the original plot. For example, in Fig. . all additional output is positioned relative to the scales on the plot axes. Because there are several coordinate systems used in the construction of a graph, there must be some way to specify which coordinate system to use when adding further output. In R’s traditional graphics, each function for adding additional output to a plot only works with a single coordinate system. For example, the text() function only positions text relative to the scales on the axes and the mtext() function only positions text relative to the plot margins (where the axis labels and plot titles are drawn). R’s grid package (Murrell, ) provides a more general approach; it is described in more detail in Sect. ..
92
Paul Murrell
Non-Cartesian Coordinates here are many examples of useful plots and diagrams that require non-cartesian coordinates, so it is desirable for statistical graphics sotware to support or at least allow the construction of a variety of coordinate systems. For example, a number of data sources suit polar coordinate displays, such as wind diagrams; when plotting a single categorical variable with exactly three levels, ternary plots can be effective; hierarchical data are naturally displayed as trees or graphs (with nodes and edges). 3.2.4
The User Interface he user interface for providing parameters to control graphical output can be adequately provided by either a command line or GUI. In a command-line environment, function calls can be made with an argument provided for each control parameter; GUIs tend to provide dialog boxes full of various options. One issue that arises with statistical graphics is the ‘explosion’ of parameters for higher-level graphical elements. Consider a matrix of scatterplots: the matrix contains many plots; each plot contains several axes; each axis consists of multiple lines and pieces of text. How can you provide parameters to control each piece of text in every axis on every plot? hat is a lot of parameters. he problem essentially is one of being able to uniquely specify a particular component of an overall plot. A mouse input device provides a very good way of specifying elements in an image. It is very natural to point at the element you want. However, there are issues when selecting components of a plot because there is oten ambiguity due to the hierarchical structure inherent in a plot. If you click on a piece of text on an axis tick mark, it is not clear whether you want to select just the text, or the entire axis, or even the entire plot. he advantage of using a command line to select objects is that, although it may be less convenient, you can typically be more expressive, or more precise. For example, in the grid graphics system in R, the text for a particular axis might be expressed as the following ‘path’: "plot1::xaxis::label". Another problem with the GUI approach is that it is hard to capture a particular editing operation. For example, if the same editing operation is required on another plot, the same series of actions must be repeated by the user. In a command-line environment, operations can be captured and repeated easily.
3.3
Extensibility he ability to produce complete plots, control all aspects of their appearance and add additional output represents a minimum standard for what statistical graphics sotware should provide. A more advanced feature is the ability to extend the system to add new capabilities, such as new types of plots. In some respects, creating a new sort of plot is just an extreme version of customization, but there are two distinguishing features: you are starting from a blank slate rather than building on an existing plot as a starting point (i.e. it is not just annotation) and, more importantly, extensibility means that the new plot that you create
Static Graphics
93
is made available for others to use in exactly the same way as existing plots. To be more explicit, in an extensible system you can create a new menu item or function that other users can access. So what sorts of features are necessary or desirable to support the development of new plots? For a start, the system must allow new functions or menu items to be added, and these must be able to be added by the user. he next most important features are that low-level building blocks must be available and there must be support for combining those building blocks into larger, coherent graphical elements (plots).
Building Blocks What are the fundamental building blocks from which plots are made? At the lowest level, a plot is simply basic graphical shapes and text, so these must be available (see ‘Graphical Primitives’ in Sect. ..). In addition, there must be some way to define coordinate systems so that graphical elements can be conveniently positioned in sensible locations to make up a plot. Surprisingly, that’s about it. Given the ability to draw shapes and locate them conveniently, you can produce a huge variety of results. Controlling coordinate systems is a special case of being able to define arbitrary transformations on output, such as is provided by the current transformation matrix in PostScript or transform attributes on group elements in SVG. We have already seen that R provides basic graphical elements such as lines and text (Sect. ..). R also provides ways to control coordinate systems; this discussion will focus on the features provided by the grid system because they are more flexible. he grid system in R provides the concept of a ‘viewport’, which represents a rectangular region on the page and contains several different coordinate systems. Viewports can be nested (positioned within each other) to produce quite complex arrangements of regions. he following code provides a simple demonstration (Fig. .). First of all, we create a region centred on the page, but only % as wide and high as the page. > pushViewport(viewport(width=0.8, height=0.8, xscale=c(0, 3), yscale=c(0, 10)))
his now is where drawing occurs, so rectangles and axes are drawn relative to this viewport. > grid.rect(gp=gpar(fill="light grey")) > grid.xaxis(at=1:2, gp=gpar(cex=0.5)) > grid.yaxis(gp=gpar(cex=0.5))
Now we define a new viewport, which is located at (, ) relative to the axis scales of the first viewport. his also demonstrates the idea of multiple coordinate systems; the width and height of this new viewport are specified in terms of absolute units, rather than relative to the axis scales of the previous viewport. > pushViewport(viewport(unit(1, "native"), unit(4, "native"), width=unit(1, "cm"), height=unit(1, "inches")))
3.3.1
94
Paul Murrell
Figure .. A demonstration of grid viewports. he overall data region (bounded by the axes) is
a viewport, each overall thermometer is another viewport, and the black region within each thermometer is yet another viewport. he white text within each thermometer is also drawn within its own (clipped) viewport
We draw a rectangle around this new viewport and then draw the word ‘thermometer’. > grid.rect(gp=gpar(fill="white")) > grid.text("thermometer", y=0, just="left", rot=90)
We create yet another viewport, which is just the bottom % of the second viewport, and draw a filled rectangle within that. > pushViewport(viewport(height=0.3, y=0, just="bottom")) > grid.rect(gp=gpar(fill="black"))
Finally, we create a viewport in exactly the same location as the third viewport, but this time with clipping turned; when we draw the word ‘thermometer’ again in white, it is only drawn within the filled black rectangle. > pushViewport(viewport(clip=TRUE)) > grid.text("thermometer", y=0, just="left", rot=90, gp=gpar(col="white"))
A second thermometer has been drawn in a similar manner in Fig. . (code not shown). his sort of facility provides great power and flexibility for producing complex plots such as the Trellis plots produced by the lattice system (Fig. .) and more besides.
Static Graphics
95
Figure .. Example of a complex Trellis plot. he data are yields of several different varieties of barley
at six sites, over years. he plot consists of panels, one for each year at each site. Each panel consists of a dotplot showing yield for a particular site in a particular year and a strip showing the year and the name of the site
96
Paul Murrell
Graphical Layout he production of a complex plot involves positioning multiple elements within multiple coordinate systems. he arrangement of output within a coordinate system is typically very explicit; for example, a data symbol is drawn at a precise location and is a fixed proportion of the plotting region in size. By contrast, the arrangement of coordinate systems (or entire plots) relative to each other is more implicit. hese arrangements are more along the lines of a number of rows and columns of plots (and let the sotware figure out exactly what that means in terms of the size and location of the plots on the page). hese sorts of arrangements have echoes of typesetting or page-layout operations like those in LATEX (Lamport, ) or HTML (Raggett et al., ), or even the generation of GUI components such as Java layout managers (Using Layout Managers, ). It is therefore useful for a statistical graphics system to provide a means for defining implicit arrangements of elements. In R there is the concept of a ‘layout’ (Murrell, ) (a simple example was given in Sect. ..). A layout divides a rectangular region into rows and columns, each with a different height or width if desired. In the grid system, a viewport can be positioned relative to a layout rather than via an explicit location and size. For example, the following code creates a viewport with a layout that defines a central region so that the margins around the central region are guaranteed to be identical on all sides and are one quarter of the minimum of the width and height of the central region. > pushViewport(viewport(layout=grid.layout(3, 3, widths=c(1, 4, 1), heights=c(1, 4, 1), respect=rbind(c(1, 0, 1), c(0, 0, 0), c(1, 0, 1)))))
his next code shows another viewport being positioned in the central region of the layout (Fig. .). he location and size of this viewport will depend on the size and shape of the parent viewport that defined the layout. > pushViewport(viewport(layout.pos.col=2, layout.pos.row=2))
With the ability to nest viewports, it is possible to specify complex implicit arrangements of graphical elements in R (this is how the panels are arranged in a lattice plot).
Transformations in Statistical Graphics An important difference between transformations in a general graphics language and transformations in statistical sotware is that statistical sotware does not apply transformations to all output. his arises from the difference between statistical graphics and general graphics images (art). A good example is that in PostScript or SVG the current transformation applies to text as well as all other shapes. In particular, if the current transformation scales output, all text is scaled. his is not desirable when drawing a statistical plot because we would like the text to be readable, so in statistical graphics, transformations apply to the locations of output and the size of shapes such as rectangles and lines, but text is sized separately (Fig. .).
Static Graphics
97
Figure .. Two views of a layout that defines a central region with equal-sized margins all around
(indicated by the grey rectangles). he location and shape of the central region depend on the size and shape of the ‘page’ which the layout is applied to; the let-hand page is tall and thin and the right-hand page is short and wide
Figure .. he difference between transformations in statistical graphics (let) and a general graphics
language (right). In statistical graphics, the location of the text depends on the coordinate system, but the size of text is controlled separately from coordinate-system transformations. In a general graphics system, all output, including text size, is affected by the current transformation; in this case, the text gets flipped upside down and drawn one-quarter of the size of normal text
Combining Graphical Elements In addition to allowing the user to compose basic graphics shapes and position them flexibly, a statistical graphics system should allow the user to ‘record’ a composition of graphics shapes. For example, the user should be able to write a function that encapsulates a series of drawing operations. his does two things: the complete set of operations becomes easily available for other people to use, and the function represents a higher-level graphical element that can be used as part of further compositions.
3.3.2
98
3.3.3
Paul Murrell
The User Interface Extending a system is one area where the user interface is crucial. In almost all cases, an extensible system must provide a language for developing new graphics. In other words, you must write code (type commands) to extend a system. his is not to say that it is impossible to produce a graphical programming interface (see, for example, the ViSta system; Young, ), but a command line offers by far the best environment for power and flexibility. As an absolute minimum, a GUI must provide some way to record code equivalents of GUI actions. Another detail is that the language for extending the system should ideally be the same language that is used to develop the system. his has two implications: first, the user has full access to the graphics system and, second, a scripting language, such as R or Python, is preferable to a ‘heavy-duty’ language such as C or Java because scripting languages are easier to get started with.
3.4
Other Issues his section draws together a number of issues that overlap with the production of static graphics but are described in more detail elsewhere.
3.4.1
3-D Plots Static -D plots have limited usefulness because -D structures are oten difficult to perceive without motion. Nevertheless, it is important to be able to produce -D images for some purposes. For example, a -D plot can be very effective for visualizing a prediction surface from a model. R provides only simple functionality for drawing -D surfaces via the persp() function, but the rgl (Adler, ) add-on package provides an interface to the powerful OpenGL -D graphics system (Schreiner, ).
3.4.2
Speed In dynamic and interactive statistical graphics, speed is essential. Drawing must be as fast as possible in order to allow the user to change settings and have the graphics update in real time. In static graphics, speed is less of an issue; achievability of a particular result is more important than how long it takes to achieve it. It is acceptable for a plot to take on the order of seconds to draw rather than milliseconds. his speed allowance is particularly important in terms of the user interface. For example, in R a lot of graphics code is written in interpreted R code (which is much slower than C code). his makes it easier for users to see the code behind graphics functions, to possibly modify the code, and even to write their own code for graphics.
Static Graphics
99
Nevertheless, a limit is still required because the time taken to draw a single plot can be multiplied many times when producing plots of a large number of observations and when running batch jobs involving a large number of plots. In R, complex plots, such as Trellis plots produced by the lattice package, can be slow enough to see individual panels being drawn, but most users find this acceptable. he entire suite of figures for a medium-sized book can still be generated in much less than a minute.
Output Formats
3.4.3
When producing plots for reports, it is necessary to produce different formats depending on the format of the report. For example, reports for printing are best produced using PostScript or PDF (Bienz and Cohn, ) versions of plots, but for publication on the World Wide Web, it is still easiest to produce some sort of raster format such as PNG. here are many excellent pieces of sotware for converting between graphics formats, which reduces the need for statistical graphics sotware to produce output in many formats; simply produce whatever format the statistical graphics sotware supports and then convert it externally. Nevertheless, there are still some reasons for statistical graphics sotware to support multiple formats. One example is that sotware can raise the bar for the lowestcommon-denominator format. For example, R performs clipping of output for formats that do not have their own clipping facilities (e.g. the FIG format; Sutanthavibul, ). Another example is that some formats, especially modern ones, provide features that are unavailable in other formats, such as transparency, hyperlinks and animation. It is not possible to convert a more basic format into a more sophisticated format without adding information. Essentially this says that if you are going to aim for a single format, aim high. Finally, it is worth noting that a description of a plot in the original language of a statistical graphics sotware system is a viable and important persistent storage option. For example, when producing plots with R, it is advisable to record the R code that was used to produce the plot in addition to saving the plot in any ‘traditional’ formats such as PDF or PostScript. One important advantage with retaining such a high-level format is that it is then possible to modify the image using high-level statistical graphics concepts. For example, an extra text label can be positioned relative to the scales on a plot by modifying the original R code, but this sort of manipulation would be inconvenient, inaccurate and hard to automate if you had to edit a PDF or PostScript version of the plot.
Data Handling he description of statistical graphics sotware in this chapter has largely ignored the issue of where the data come from. On one hand, this is deliberate because by separating data from graphics there is a greater flexibility to present any data using any sort of graphic.
3.4.4
100
Paul Murrell
However, we should acknowledge the importance of functionality for generating, importing, transforming and analysing data. Without data, there is nothing interesting to plot. In an ideal situation, statistical graphics facilities are provided as part of a larger system with data-handling features, as is the case with R.
3.5
Summary Statistical graphics sotware should provide a straightforward way to produce complete plots. It should be possible to customize all aspects of the plot, add extra output to the plot and extend the system to create new types of plots. Statistical graphics sotware can be thought of as an extension of a sophisticated graphics language, providing a fully featured graphics system, a programming language and extensions to specifically support statistical graphics.
References Adler, D. (). rgl: D visualization device system (OpenGL). R package version .-. http://wsopuppenkiste.wiso.uni-goettingen.de/~dadler/rgl. Becker, R.A., Cleveland, W.S. and Shyu, M.-J. (). he visual design and control of Trellis display, Journal of Computational and Graphical Statistics :–. Bienz, T. and Cohn, R. (). Portable Document Format Reference Manual, Addison-Wesley, Reading, MA, USA. Cleveland, W.S. (). he Elements of Graphing Data, Wadsworth Publ. Co. Cleveland, W.S. (). Visualizing Data, Hobart Press. Cleveland, W.S. and McGill, R. (). Graphical perception: The visual decoding of quantitative information on graphical displays of data (C/R, pp. -), Journal of the Royal Statistical Society, Series A, General :–. Ferraiolo, J., Jun, F. and Jackson, D. (). Scalable vector graphics (SVG) ., http://www.w.org/TR/SVG/. Fox, J. (). Rcmdr: R Commander. R package version .-. http://socserv.socsci. mcmaster.ca/jfox/Misc/Rcmdr/. Helbig, M., Urbanek, S. and heus, M. (). Java GUI for R, http://stats.math. uni-augsburg.de/JGR/index.shtml. Iacus, S. and Urbanek, S. (). Cocoa-based GUI for R for Mac OS X, http://cran. r-project.org/bin/macosx/. Inc., C.A.S. (). PostScript language reference manual (nd ed.), Addison-Wesley, Boston, MA, USA. Lamport, L. (). LATEX: A document preparation system, Addison-Wesley. Murrell, P. (). he grid graphics package, R News ():–. http://CRAN. R-project.org/doc/Rnews/. Murrell, P.R. (). Layouts: A mechanism for arranging plots on a page, Journal of Computational and Graphical Statistics :–.
Static Graphics 101
Porter, T. and Duff, T. (). Compositing digital images, SIGGRAPH ’ Conference Proceedings, Association for Computing Machinery, Vol. . R Development Core Team (). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria. http://www. R-project.org Raggett, D., Hors, A.L. and Jacobs, I. (). HTML . specification, http://stat. stat.auckland.ac.nz/stats/resource/html/cover.html. Robbins, N. (). Creating More Effective Graphs, Wiley. Sarkar, D. (). Lattice, R News ():–. http://CRAN.R-project.org/doc/ Rnews/. Schreiner, D. (). OpenGL Reference Manual: he Official Reference Document to OpenGL, Version ., Addison-Wesley Longman. Sutanthavibul, S. (). FIG: Facility for interactive generation of figures, http:// duke.usask.ca/~macphed/sot/fig/FORMAT..txt. Swayne, D.F., Lang, D.T., Buja, A. and Cook, D. (). GGobi: evolving from XGobi into an extensible framework for interactive data visualization, Computational Statistics and Data Analysis :–. heus, M. (). Interactive data visualization using Mondrian, Journal of Statistical Sotware, (). http://www.jstatsot.org/v/i/. Tute, E. (). Graphing Sotware in ASK E.T., http://www.edwardtute.com/ bboard/q-and-a?topic_id=. Tute, E.R. (). he Visual Display of Quantitative Information, Graphics Press. Using Layout Managers. in he Java Tutorial (). http://java.sun.com/docs/books/ tutorial/uiswing/layout/using.html. Young, F.W. (). ViSta: he visual statistics system, Technical Report -(c), UNC L.L. hurstone Psychometric Laboratory Research Memorandum. Wilkinson, L. (). he Grammar of Graphics, Springer. Wilkinson, L. (). he grammar of graphics, in J. Gentle, W. Härdle and Y. Mori (eds), Handbook of Computational Statistics: concepts and methods, Springer.
Data Visualization Through Their Graph Representations
II.4
George Michailidis
4.1 4.2 4.3
4.4
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . Data and Graphs . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . Graph Layout Techniques . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
106
Force-directed Techniques . . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Multidimensional Scaling . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . The Pulling Under Constraints Model . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . . Bipartite Graphs .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .. . .. . . .
109 110 113 114
Discussion and Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .
118
104 104
104
4.1
George Michailidis
Introduction he amount of data and information collected and retained by organizations and businesses is constantly increasing, due to advances in data collection, computerization of transactions, and breakthroughs in storage technology. Further, many attributes are also recorded, resulting in very high-dimensional data sets. Typically, the applications involve large-scale information banks, such as data warehouses that contain interrelated data from a number of sources. Examples of new technologies giving rise to large, high-dimensional data sets are high-throughput genomic and proteomic technologies, sensor-based monitoring systems, etc. Finally, new application areas such as biochemical pathways, web documents, etc. produce data with inherent structure that cannot be simply captured by numbers. To extract useful information from such large and structured data sets, a first step is to be able to visualize their structure, identifying interesting patterns, trends, and complex relationships between the items. he main idea of visual data exploration is to produce a representation of the data in such a way that the human eye can gain insight into their structure and patterns. Visual data mining techniques have proven to be of particularly high value in exploratory data analysis, as indicated by the research in this area (Eick and Wills a, b). In this exposition, we focus on the visual exploration of data through their graph representations. Specifically, it is shown how various commonly encountered structures in data analysis can be represented by graphs. Special emphasis is paid to categorical data for which many commonly used plotting techniques (scatterplots, parallel coordinate plots, etc.) prove problematic. Further, a rigorous mathematical framework based on optimizing an objective function is introduced that results in a graph layout. Several examples are used to illustrate the techniques.
4.2
Data and Graphs Graphs are useful entities since they can represent relationships between sets of objects. hey are used to model complex systems (e.g., computer and transportation networks, VLSI and Web site layouts, molecules, etc.) and to visualize relationships (e.g., social networks, entity-relationship diagrams in database systems, etc.). In statistics and data analysis, we usually encounter them as dendrograms in cluster analysis, as trees in classification and regression, and as path diagrams in structural equation models and Bayesian belief diagrams. Graphs are also very interesting mathematical objects, and a lot of attention has been paid to their properties. In many instances, the right picture is the key to understanding. he various ways of visualizing a graph provide different insights, and oten hidden relationships and interesting patterns are revealed. An increasing body of literature is considering the problem of how to draw a graph [see for instance the book by Di Battista et al. () on graph drawing, the Proceedings of the Annual Conference on Graph Drawing, and the annotated bibliography by Di Battista et al. ()]. Also, several problems in distance geometry
Data Visualization Through Their Graph Representations 105
Figure .. Graph representation of a small protein interaction network, with nodes corresponding to
proteins and links to their physical interactions
and in graph theory have their origin in the problem of graph drawing in higherdimensional spaces. Of particular interest in this study is the representation of data sets through graphs. his bridges the fields of multivariate statistics and graph drawing. Figure . shows the graph representation of the protein interaction network implicated in the membrane fusion process of vesicular transport for yeast (Ito et al., ), with the nodes representing the proteins and the links the physical interactions between them. However, graphs are also capable of capturing the structure of data commonly encountered in statistics, as the following three examples show. he first example deals with a contingency table (Table . and Fig. .), where the nodes correspond to the categories and the weighted links represent the frequencies. he second example deals with a small correlation matrix (Table . and Fig. .), which can also be represented by a weighted graph, with the nodes representing the variables and the links the strength of the correlation. Table .. Contingency table of school children form Caithness, Scotland, classified according to
two categorical variables, hair and eye color (Fisher, )
Eye color
Hair color Fair
Red
Medium
Dark
Black
Light Blue Medium Dark
106
George Michailidis
Figure .. Weighted graph representation of a contingency table Table .. A small correlation matrix for four variables
Var Var Var Var
Var
Var
Var
Var
. . . .
. . .
. .
.
Figure .. Representation of a small correlation matrix by a weighted graph
Another interesting data structure that can be represented successfully by a graph is that corresponding to a multivariate categorical data set, as the following example attests (Table .). he data on sleeping bags and their characteristics come from Prediger () and have also been discussed in Michailidis and de Leeuw ().
4.3
Graph Layout Techniques he problem of graph drawing/layout has received a lot of attention from various scientific communities. It is defined as follows: given a set of nodes connected by a set
Data Visualization Through Their Graph Representations 107
One Kilo Bag Sund Kompakt Basic Finmark Tour Interlight Lyx Kompakt Touch the Cloud Cat’s Meow Igloo Super Donna Tyin Travellers Dream Yeti Light Climber Viking Eiger Climber light Cobra Cobra Comfort Foxfire Mont Blanc
Good
Synthetic fibers
Down fibers
Expensive
Fiber
Bad
Price
Acceptable
Sleeping Bag
Not expensive
Cheap
Table .. he superindicator matrix representation (Gifi, ) of a categorical data set
Quality
of edges, identify the positions of the nodes in some space and calculate the curves that connect them. Hence, in order to draw a graph, one has to make the following two choices: (i) selection of the space and (ii) selection of the curves. For example, grid layouts position the nodes at points with integer coordinates, while hyperbolic layouts embed the points on a sphere. Most graph drawing techniques use straight lines between connected nodes, but some use curves of a certain degree (Di Battista et al., ). Many layout algorithms are based on a set of aesthetic rules that the drawing needs to adhere to. Popular rules are that nodes and edges must be evenly distributed, edges should have similar lengths, edge crossings must be kept to a minimum, etc. Some of these rules are important in certain application areas. Further, many of these rules lead to a corresponding optimization problem, albeit intractable in certain cases. For example, the edge-crossing minimization is provably NP-hard and hence computationally intractable (Di Battista et al., ). In many cases, a basic layout is obtained by a computationally fast algorithm, and the resulting drawing is postprocessed to
108
George Michailidis
Figure .. Graph representation of the sleeping bag data set presented in Table ., with the let set of
nodes corresponding to the objects (sleeping bags), the right set of nodes to the categories of the three attributes (price, fiber, quality), and selected edges capturing the relationship between objects and categories.
adhere to such aesthetic rules. he latter strategy proves particularly useful in the presence of large graphs and is adopted by several graph drawing systems, such as Nicheworks (Wills, ), GVF (Herman et al., ), and HViewer (Muentzer, ). Many systems also allow manual postprocessing of the resulting layout; see for example the Cytoscape visualization system (www.cytoscape.org). he general problem of graph drawing discussed in this paper is to represent the edges of a graph as points in R p and the vertices as lines connecting the points. Graph drawing is an active area in computer science, and it is very ably reviewed in the recent book by Di Battista et al. (). he choice of R p is due to its attractive underlying geometry and the fact that it renders the necessary computations more manageable.
Data Visualization Through Their Graph Representations 109
here are basically two different approaches to making such drawings. In the metric or embedding approach, one uses the path-length distance defined between the vertices of the graph and tries to approximate these distances by the Euclidean distance between the points. he area of embedding graph-theoretical distances is related to distance geometry, and it has been studied a great deal recently. In this paper, we adopt primarily the adjacency model, i.e., we do not emphasize graph-theoretical distances, but we pay special attention to which vertices are adjacent and which are not. Obviously, this is related to distance, but the emphasis is different. We use objective (loss) functions to measure the quality of the resulting embedding.
Force-directed Techniques
4.3.1
he class of graph-drawing techniques most useful for data visualization are forcedirected techniques. his class of techniques borrows an analogy from classical physics, with the vertices being bodies with masses that attract and repel each other due to the presence of springs, or because the vertices have electric charges. his implies that there are ‘physical’ forces pulling and pushing the vertices apart, and the optimal graph layout will be one in which these forces are in equilibrium. An objective (loss) function that captures this analogy is given next: n
n
n
n
Q(XA, B) = a i j ϕ(d i j (X)) − b i j ψ(d i j (X)) ,
(.)
i= j=
i= j=
where the n p matrix X contains the coordinates of the n vertices in R p and d i j (X) denotes the distances between points with coordinates x i and x j . he weights a i j correspond to those in the adjacency matrix A of the graph G, while the pushing weights B = b i j could be derived either from the adjacency matrix or from an external constraint. Finally, the functions ϕ(ċ) and ψ(ċ) are transformations whose role is to impose some aesthetic considerations on the layout. For example, a convex ϕ function will reinforce large distances by rendering them even larger and thus enable one to detect unique features in the data, while a concave transformation will dampen the effect of isolated vertices. Notice that this framework can accommodate both simple (i.e., a i j , ) and weighted (i.e., a i j ) graphs. A popular forcedirected technique that employs this pull-push framework is discussed in Di Battista et al. (), where the pulling is done by springs obeying Hooke’s law (i.e., the force is proportional to the difference between the distance of the vertices and the zeroenergy length of the spring), while the pushing is done by electrical forces following an inverse square law. Variations on this physical theme are used in several other algorithms (Fruchterman and Reingold and references therein). Another way of incorporating a pushing component in the above objective function is through a normalization constraint. For example, one can require that η(X) = , and then the objective function takes the form by forming the Lagrangian: n
n
Q(XA) = a i j ϕ(d i j (X)) − λ(η(X) − ) . i= j=
(.)
110
George Michailidis
It then becomes clear that the constraint term in the Lagrangian corresponds to the push component of Q(ċ). Examples of η(X) include η(X) = trace(X ′ X) or η(X) = det(X ′ X). Other possibilities include requiring the orthonormality of the points in the layout, such as X ′ X = I p or even fixing some of the Xs (Tutte, ). Finally, this formulation allows one to incorporate into the force-directed framework the metric approach of graph drawing, where one works not with the adjacency matrix of the graph but with a distance matrix defined on the graph G. he goal then becomes to approximate graph-theoretic distances by Euclidean distances. Hence, the goal becomes to minimize n
n
Q(XW) = w i j ρ(d i j (X)) ,
(.)
ρ(d i j (X)) = (η(δ i j ) − η(d i j (X))) ,
(.)
i= j=
where where W = w i j is a set of weights. he δ i j correspond to path-length distances defined on the graph G, whereas the transformation η is usually the identity, the square, or the logarithm. Obviously ρ(d(X)) is not increasing and does not pass through zero; nevertheless, by expanding the square it becomes clear that it is equivalent to minimizing Q(XW) with ϕ(d) = η (d), ψ(d) = η(d), and b i j = η(δ i j )w i j . hus, all points are pulling together, but points with large path-length distances are being pushed apart. Next we examine in more detail the metric or embedding approach and the pulling under constraints model, which have proved particularly useful for drawing graphs obtained from data. 4.3.2
Multidimensional Scaling he metric approach previously discussed corresponds to one version of multidimensional scaling (MDS). MDS is a class of techniques where a set of given distances is approximated by distances in low-dimensional Euclidean space. Formally, let δ i j , i, j = , . . . , n be a set of distances. he goal is to identify the coordinates of n points x i in R p such that the Euclidean distance d(x i , x j ) d i j (X) is approximately equal to δ i j . As mentioned before, for graph-drawing purposes, the δ i j correspond to the shortest path distances defined on a graph G. A discussion of MDS as a graph-drawing technique is provided in Buja and Swayne (), where in addition other choices beyond Euclidean space are studied for the embedding space. he least-squares-loss (fit) function (known in the literature as Stress) introduced in Kruskal () has the form n
n
σ(X) = w i j (δ i j − d i j (X)) i= j=
(.)
Data Visualization Through Their Graph Representations 111
that is minimized over X. he w i j are weights that can be chosen to reflect variability, measurement error, or missing data. his is precisely the objective function (.) derived from the general framework of force-directed techniques previously introduced and discussed. A number of variations of (.) have appeared in the literature. In McGee (), the loss function has weights δ − i j . he loss function is interpreted as the amount of physical work that must be done on elastic springs to stretch or compress them from an initial length δ i j to a final length d i j . On the other hand, the following choice of weights w i j = δ − i j is discussed in Sammon (). Minimization of the loss function (.) can be accomplished either by an iterative majorization algorithm (Borg and Groenen ; De Leeuw and Michailidis ) or by a steepest descent method (Buja and Swayne ). he latter method is used in the implementation of MDS in the GGobi visualization system (Swayne et al., ). A -D MDS solution for the sleeping bag data is shown in Fig. .. It can be seen that the solution spreads the objects in the data set fairly uniformly in the plane, and edge crossings are avoided. We discuss next a fairly recent application of MDS. In many instances, the data exhibit nonlinearities, i.e., they lie on a low-dimensional manifold of some curvature. his has led to several approaches that still rely on the embedding (MDS) approach for visualization purposes but appropriately alter the input distances δ i j . A pop-
Figure .. MDS representation of sleeping bag data set based on χ -distances. Due to the discrete
nature of the data, multiple objects are mapped onto the same location, as shown in the plot. Further, for reference purposes, the categories to which the sleeping bags belong have been added to the plot at the centroids of the object points
112
George Michailidis
Figure .. [his figure also appears in the color insert.] Top panel: original data ( data points)
arranged along a nonlinear surface (Swiss Roll). Middle panel: -D MDS representation based on a complete weighted graph. Bottom panel: -D MDS representation based on nearest-neighbor graph
Data Visualization Through Their Graph Representations 113
ular and fairly successful nonlinear embedding technique is the Isomap algorithm (Tenenbaum et al., ). he main algorithm computes for each point in the data set a K-nearest neighbor graph and then stitches them together in the adjacency matrix. It then calculates distances using the resulting graph and then applies MDS. he main idea in the first step of the construction is to capture well the local geometry of the data. An illustration of the idea based on the Swiss Roll data set is shown next. Specifically, random points lying on a roll have been generated and their Euclidean pairwise distances computed. In addition, a graph that connects data points only with their closest neighbors in the Euclidean metric was computed and shortest path distances calculated. Subsequently, MDS was applied to both distance matrices and a -D embedding obtained. he coloring scheme shows that straightforward MDS does not capture the underlying geometry of the roll (since the points do not follow their progression on the roll, blue, cyan, green, etc.), whereas the first dimension using the Isomap algorithm recovers the underlying structure. he hole in the middle is mostly due to the low density of orange points.
The Pulling Under Constraints Model
4.3.3
In this model, the similarity of the nodes is important. In the case of a simple graph, only connections between nodes are taken into consideration, whereas in a weighted graph, edges with large weights play a more prominent role. However, the normalization constraint, as discussed in Sect. , pushes points apart and avoids the trivial solution of all points collapsing to the origin. his model, under various distance functions, has been studied in a series of papers by Michailidis and de Leeuw (, , ). We examine next the case of squared Euclidean distances, where ϕ(d(X)) d ij (X), which turns out to be particularly interesting from a data visualization point of view. Some algebra shows that the objective function can be written in the following matrix algebra form: Q(XA) = trace(X ′ LX) ,
(.)
where L = D−A is the graph Laplacian (Chung, ), with D being a diagonal matrix containing the row sums of the adjacency matrix A. It can be seen that by minimizing (.), nodes sharing many connections would be pulled together, whereas nodes with few connections would end up on the periphery of the layout. For a weighted graph, the larger the weights, the stronger the bond between nodes and hence the more pronounced the clustering pattern. A normalization constraint that leads to a computationally easy-to-solve problem is X ′ DX = I p . Some routine calculations show that minimizing (.) subject to this constraint corresponds to solving a generalized eigenvalue problem. Further, notice that the solution is not orthogonal in Euclidean space, but in weighted (by D) Euclidean space. Figure . shows the graph layout for the small protein interactions network shown in Fig. .. It can be seen that proteins PIB and BET that have very few interactions are located on the periphery of the layout. Moreover, the ‘hub’ pro-
114
George Michailidis
Figure .. Two-dimensional layout of the small protein interaction network shown in Fig. .
teins TLG and YIP are positioned close to the center of the plot, signifying their central role in this network. he next example comes from the UCI machine learning repository. he data set consists of features of handwritten numerals (–) extracted from a collection of Dutch utility maps. here are patterns per class, and variables characterizing the pixel intensity of the underlying digital image of the digit have been collected. he pixel intensities are categorical and take values in the to range. his is an example where linear techniques such as principal component analysis fail to separate the classes (see top panel of Fig. .). he next set of plots in Fig. . shows the layouts of a few large graphs that have been used for testing graph partitioning algorithms (Walshaw, ). he first graph is comprised of vertices and edges, the second of vertices and edges, and the third of vertices and edges. hey are derived from computational mechanics meshes and characterized by extreme variations in the mesh density and the presence of “holes.” he layouts shown are based on weighted graphs that were built by considering for each vertex its ten nearest neighbors in the Euclidean metric and calculating exponentially decreasing weights. It can be seen that the layouts capture, to a large extent, the underlying structure of the graphs in terms of density and the presence of “holes.” 4.3.4
Bipartite Graphs As noted in Sect. , the graph representation of a contingency table and of a categorical data set has some special features, namely, the node set V can be partitioned into two subsets. For example, in the case of a contingency table, the categories of one variable form the first subset and those of the other variable the second one. Notice that there are only connections between members of these two subsets. An analogous situation arises in the case of categorical data, where the first subset of nodes corresponds to the objects (e.g., the sleeping bags) and the second subset to the categories
Data Visualization Through Their Graph Representations 115
Figure .. [his figure also appears in the color insert.] PCA layout of digits dataset (top panel) and
the -D graph layout (bottom panel)
of all the variables. hese are two instances where the resulting graph representation of the data gives rise to a bipartite graph. A slight modification of the Q(ċ) objective function leads to interesting graph layouts of such data sets. Let X = [Z ′ Y ′ ], where Z contains the coordinates of the first subset of the vertices and Y those of the second subset. he objective function for squared Euclidean distances can then be written as (given the special block structure of the adjacency matrix A) Q(Z, YA) = trace(Z ′ D Z Z + Y ′ DY Y − Y ′ AZ) ,
(.)
where DY is a diagonal matrix containing the column sums of A and D Z another diagonal matrix containing the row sums of A. In the case of a contingency table, both DY and D Z contain the marginal frequencies of the two variables, while for a multivariate categorical data set DY contains again the univariate marginals of all the categories of all the variables and D Z = JI is a constant multiple of the identity matrix, with J denoting the number of variables in the data set. A modification of
116
George Michailidis
Figure .. Layouts of large graphs derived from computational mechanics meshes and characterized
by varying degrees of mesh density
Data Visualization Through Their Graph Representations 117
the normalization constraint to this setting, namely, Z ′ D Z Z = I p , gives the following solution, which can be obtained through a block relaxation algorithm (Michailidis and de Leeuw, ): ′ Z = J − AY and Y = D− Y A XZ.
Hence, the optimal solution satisfies the centroid principle (Gifi, ), which says that the category points in the optimal layout are at the center of gravity of the objects that belong to them. he above graph-drawing solution is known in multivariate analysis for contingency tables as correspondence analysis and for multivariate categorical data sets as multiple correspondence analysis (Michailidis and de Leeuw, ). Figure . shows the graph layout of the sleeping bags data set. he solution captures the basic patterns in the data set, namely, that there are good-quality, expensive sleeping bags filled with down fibers and cheap, bad-quality sleeping bags filled with synthetic fibers. Further, there exist some sleeping bags of intermediate quality and price filled with either down or synthetic fibers. Notice that the centroid principle resulting from the partitioning of the vertex set proves useful in the interpretation of the layout. Further, the resulting layout is less ‘uniform’ than the one obtained through MDS and thus better captures features of the data. It is interesting to note that the choice of the distance function coupled with a particular normalization has a significant effect on the aesthetic quality of the resulting
Figure .. Graph layout of sleeping bags data based on objective function (.). Due to the discrete
nature of the data, multiple objects are mapped on the same location, as shown in the plot. Further, for reference purposes, the categories to which the sleeping bags belong have been added to the plot at the centroids of the object points
118
George Michailidis
graph layout. An extreme case occurs when ϕ(d(X)) d i j (X) corresponds to Euclidean distances. hen, under the orthonormality constraint, the solution is rather uninteresting and consists of exactly p + points, where p is the dimensionality of the embedding space. A mathematical explanation of this result is given for the -D case (p = ) in de Leeuw and Michailidis () and is also illustrated for the higherdimensional case (p ) in Michailidis and de Leeuw (). 4.4
Discussion and Concluding Remarks In this paper, the problem of data visualization through layouts of their graph representations is considered. A mathematical framework for graph drawing based on force-directed techniques is introduced, and several connections to well-known multivariate analysis techniques such as multidimensional scaling, correspondence, and multiple correspondence analysis are made. Several extensions that may improve the quality of the graph layout are possible within this general framework. For example, logistic loss functions are explored in de Leeuw (), together with the arrangement of the nodes along Voronoi cells. he visualization of several related data sets through multilevel extensions of multiple correspondence analysis are explored in Michailidis and de Leeuw (). Finally, a version of multidimensional scaling for data that change over time is discussed in Costa et al. (). Acknowledgement. his work was partially supported by NIH grant PRR-. he author would like to thank Jan de Leeuw for many fruitful discussions and suggestions on the topic over the course of the last years and the editors of the Handbook for many comments that improved the presentation of the material.
References Borg, I. and Groenen, P. (). Modern Multidimensional Scaling: heory and Applications. Springer, Berlin Heidelberg New York. Buja, A. and Swayne, D. (). Visualization methodology for multidimensional scaling. J Classificat, :–. Chung, F.R.K. (). Spectral Graph heory. American Mathematical Society, Providence, RI. Costa, J., Patwari, N. and Hero, A.O. (). Distributed multidimensional scaling with adaptive weighting for node localization in sensor networks. ACM J Sensor Network, ():–. www.cytoscape.org. De Leeuw, J. (). Nonlinear Principal Component Analysis and Related Techniques. Preprint # , Department of Statistics, University of California, Los Angeles. De Leeuw, J. and Michailidis, G. (). Graph layout techniques and multidimensional data analysis. In: Bruss, F.T., Le Cam, L. (eds) Game heory, Optimal Stop-
Data Visualization Through Their Graph Representations 119
ping, Probability and Statistics. IMS Lecture Notes – Monograph Series, :– . De Leeuw, J. and Michailidis, G. (). Weber correspondence analysis: the onedimensional case. J Comput Graph Stat, :–. Di Battista, G., Eades, P., Tamassia, R. and Tollis, I. (). Algorithms for drawing graphs: an annotated bibliography. Comput Geom heory Appl, :–. Di Battista, G., Eades, P., Tamassia, R. and Tollis, I. (). Graph Drawing: Algorithms for Geometric Representation of Graphs. Prentice-Hall, Upper Saddle River, NJ. Eick, S.G. and Wills, G.J. (a) Navigating large networks with hierarchies. In: Proceedings of Visualization Conference, pp. –. Eick, S.G. and Wills, G.J. (b) High interaction graphics. Eur J Operat Res, :–. Fisher, R.A. (). he precision of discriminant functions. Ann Eugen, :–. Fruchterman, T.M.J. and Reingold, E.M. (). Graph drawing by force-directed placement. Sotware: practice and engineering, :–. Gifi, A. (). Nonlinear Multivariate Analysis. Wiley, Chichester. www.ggobi.org. Herman, I., Melancon, G. and Marshall, M.S. (). Graph visualization and navigation in information visualization. IEEE Trans Visualizat Comput Graph, :–. Ito, T., Tashiro, K., Muta, S., Ozawa, R., Chiba, T., Nishizawa, M., Yamamoto, K., Kuhara, S. and Sakaki, Y. (). Toward a protein-protein interaction map of the budding yeast: a comprehensive system to examine two-hybrid interactions in all possible combinations between the yeast interactions. Proc Natl Acad Sci USA, :–. Kruskal, J.B. (). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, :–. McGee, V.E. (). he multidimensional analysis of elastic distances. Br J Math Stat Psychol, :–. Michailidis, G. and de Leeuw, J. (). he Gifi system of descriptive multivariate analysis. Stat Sci, :–. Michailidis, G. and de Leeuw, J. (). Multilevel homogeneity analysis with differential weighting. Comput Stat Data Anal, :–. Michailidis, G. and de Leeuw, J. (). Data visualization through graph drawing. Comput Stat, :–. Michailidis, G. and de Leeuw, J. (). Homogeneity analysis using absolute deviations. Comput Stat Data Anal, :–. Muentzer, T. (). Drawing large graphs with HViewer and Site Manager. In: Proceedings of Graph Drawing . Lecture Notes in Computer Science, vol . Springer, Berlin, Heidelberg, New York, pp. –. Prediger, S. (). Symbolic objects in formal concept analysis. In: Mineau, G., Fall, A. (eds) Proceedings of the nd international symposium on knowledge, retrieval, use and storage for efficiency. Sammon, J.W. (). A nonlinear mapping for data structure analysis. IEEE Trans Comput, :–.
120
George Michailidis
Swayne, D.F., Cook, D. and Buja, A. (). XGobi: interactive dynamic data visualization in the X Window system. J Comput Graph Stat, :–. Tenenbaum, J.B., da Silva, V. and Langford, J.C. (). A global geometric framework for nonlinear dimensionality reduction. Science, :–. Tutte, W.T. (). How to draw a graph. Proc Lond Math Soc, :–. Walshaw, C. (). A multilevel algorithm for force-directed graph drawing J Graph Algor Appl, :–. Wills, G.J. (). NicheWorks – interactive visualization of very large graphs