Reading and Writing the Electronic Book
iii
Synthesis Lectures on Information Concepts, Retrieval, and Services Editor Gary Marchionini, University of North Carolina, Chapel Hill Reading and Writing the Electronic Book Catherine C. Marshall 2010 Understanding User – Web Interactions via Web Analytics Bernard J. ( Jim) Jansen 2009 XML Retrieval Mounia Lalmas 2009 Faceted Search Daniel Tunkelang 2009 Introduction to Webometrics: Quantitative web research for the social sciences Michael Thelwall 2009 Automated Metadata in Multimedia Information Systems: Creation, Refinement, Use in Surrogates, and Evaluation Michael G. Christel 2009 Exploratory Search: Beyond the Query-Response Paradigm Ryen W. White and Resa A. Roth 2009 New Concepts in Digital Reference R. David Lankes 2009
Copyright © 2010 by Morgan & Claypool All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopy, recording, or any other except for brief quotations in printed reviews, without the prior permission of the publisher. Reading and Writing the Electronic Book Catherine C. Marshall www.morganclaypool.com ISBN: 9781598299052 paperback ISBN: 9781598299069 ebook DOI: 10.2200/S00215ED1V01Y200907ICR009 A Publication in the Morgan & Claypool Publishers series Synthesis Lectures on Information Concepts, Retrieval, and Services Lecture #9 Series Editor: Gary Marchionini, University of North Carolina, Chapel Hill Series ISSN ISSN 1947-945X
print
ISSN 1947-9468
electronic
Reading and Writing the Electronic Book Catherine C. Marshall Microsoft Research
SYNTHESIS LECTURES ON INFORMATION CONCEPTS, RETRIEVAL, AND SERVICES #9
vi
ABSTRACT Developments over the last 20 years have fueled considerable speculation about the future of the book and of reading itself. This book begins with a gloss over the history of electronic books, including the social and technical forces that have shaped their development. The focus then shifts to reading and how we interact with what we read: basic issues such as legibility, annotation, and navigation are examined as aspects of reading that eBooks inherit from their print legacy. Because reading is fundamentally communicative, I also take a closer look at the sociality of reading: how we read in a group and how we share what we read. Studies of reading and eBook use are integrated throughout the book, but Chapter 5 “goes meta” to explore how a researcher might go about designing his or her own reading-related studies. No book about eBooks is complete without an explicit discussion of content preparation, i.e., how the electronic book is written. Hence, Chapter 6 delves into the underlying representation of eBooks and efforts to create and apply markup standards to them. This chapter also examines how print genres have made the journey to digital and how some emerging digital genres might be realized as eBooks. Finally, Chapter 7 discusses some beyond-the-book functionality: how can eBook platforms be transformed into portable personal libraries? In the end, my hope is that by the time the reader reaches the end of this book, he or she will feel equipped to perform the next set of studies, write the next set of articles, invent new eBook functionality, or simply engage in a heated argument with the stranger in seat 17C about the future of reading.
Keywords eBooks, e-book, electronic publications, field studies, ethnography, reading, personal digital libraries, personal information management, annotation, bookmarking, navigation, study design, e-book design.
vii
Preface Questions about the future of reading have fueled considerable speculation over the past 20 years. To frame the topics covered in this book, the book begins with a gloss over the history of electronic books, including the social and technical forces that have shaped their development. From this background, Chapters 2 and 3 then move into reading itself and how we interact with what we read. Topics covered in Chapter 2 include layout, legibility, and eBook display hardware. Chapter 3 moves on to annotation, navigation, linking, clipping, and bookmarking—in short, the interactions people have with ordinary print books. Reading is above all a social activity, so Chapter 4 takes a close look at the sociality of reading—how we read in a group and how we share what we read. Because qualitative and quantitative study results are integrated throughout the book, studies of reading or eBook use are not relegated to their own chapter. Instead, Chapter 5 talks about how a researcher might go about designing some types of studies that are particularly appropriate for exploring reading and for evaluating the results of eBook field deployments. No book about eBooks is complete without an explicit discussion of content, i.e., how the electronic book is written. Hence, Chapter 6 delves into the underlying representation of content and efforts to create and apply markup standards to texts. This chapter also examines how print genres have made the journey to digital and how emerging digital genres including blogs, wikis, and hypertext fiction might be realized as eBooks. Finally, Chapter 7 discusses how we can do more with electronic books than just read them cover to cover. Support for different kinds of reading that are part and parcel of different disciplines—law, education, analysis, and so on—may be implemented within eBooks. We devote the concluding sections of the chapter to managing and using collections, such as tools for gathering, triage, and analysis. We focus particularly on what it will take to make eBook platforms into more than just books rendered electronically: how can eBook platforms be transformed into portable personal digital libraries? In the end, some basic philosophical questions remain. My hope is that by the time the reader reaches the end of this book, he or she will feel equipped to perform the next set of studies, write the next set of articles, invent new beyond-print functionality, vet the next generation of eBook business plans, or engage in a heated argument with the stranger in seat 17C about the future of reading.
Contents Preface���������������������������������������������������������������������������������������������������������������������� vii Figure Credits............................................................................................................ xiii 1.
Introduction........................................................................................................1 1.1 Generation 1: A New World of Hypermedia....................................................... 1 1.2 Generation 2: EBook Hardware Arrives.............................................................. 3 1.3 Generation 3: ePaper and the Quiet Revolution.................................................. 6 1.4 This Book............................................................................................................. 8
2.
Reading............................................................................................................. 11 2.1 Reading.............................................................................................................. 14 2.1.1 Assumptions About Reading.................................................................. 14 2.1.2 Purposes of Reading............................................................................... 18 2.1.3 Types of Reading.................................................................................... 19 2.2 Layout, Typography, and Legibility.................................................................... 21 2.3 Studies of the Effect of Layout on Readers’ Performance.................................. 29 2.4 Reading Hardware and Display Technologies.................................................... 33
3.
Interaction......................................................................................................... 37 3.1 Annotation......................................................................................................... 38 3.1.1 Representing Annotations...................................................................... 40 3.1.2 Anatomy of an Annotation..................................................................... 42 3.1.3 Linking................................................................................................... 45 3.1.4 Functions of Annotation........................................................................ 46 3.1.5 Status and Value of Annotations............................................................ 49 3.2 Navigation.......................................................................................................... 51 3.2.1 Three Navigation Scenarios.................................................................... 53 3.2.2 Moving................................................................................................... 56 3.2.3 Orienting................................................................................................ 58 3.3 Clipping.............................................................................................................. 62
reading and writing the electronic book
3.4 Bookmarking...................................................................................................... 67 3.5 Hardware for Interacting with EBooks.............................................................. 69 3.5.1 Hardware That Supports Navigation..................................................... 69 3.5.2 Pen-Based Interaction............................................................................ 71 3.6 Essential but Insufficient.................................................................................... 72 4.
Reading as a Social Activity................................................................................ 73 4.1 Reading Together............................................................................................... 74 4.1.1 Shared Focus.......................................................................................... 75 4.1.2 Collaborative Search and Reference Following...................................... 77 4.1.3 Reading Together as an Informal Act..................................................... 78 4.1.4 Peer-to-Peer Sharing.............................................................................. 79 4.2 Sharing the Artifacts of Reading........................................................................ 80 4.2.1 Reading to Know What Other People Know......................................... 81 4.2.2 Sharing Annotations.............................................................................. 82 4.2.3 Aggregating Annotations: The Wisdom of Crowds............................... 85 4.2.4 Sharing Encountered Information......................................................... 89 4.2.5 Information Brokering........................................................................... 91 4.2.6 Sharing and Recommending Books....................................................... 92
5.
Studying Reading............................................................................................... 95 5.1 Types of Studies................................................................................................. 98 5.2 Quantitative/Laboratory Studies........................................................................ 99 5.2.1 Performance Metrics for Reading........................................................... 99 5.2.2 Eye-Tracking........................................................................................ 100 5.3 Field Studies..................................................................................................... 101 5.3.1 Interview Studies.................................................................................. 104 5.3.2 Diary Studies........................................................................................ 105 5.3.3 Observational Studies........................................................................... 105 5.3.4 Surveys and Questionnaires.................................................................. 106 5.3.5 Instrumenting Software........................................................................ 108 5.4 Performing a Field Study of Reading............................................................... 109 5.4.1 Research Questions and Study Design�������������������������������������������������109 5.4.2 Finding Participants............................................................................. 111 5.4.3 Developing an Interview Script............................................................ 112 5.4.4 Preparing Materials.............................................................................. 113 5.4.5 In the Field........................................................................................... 114 5.4.6 Data Analysis....................................................................................... 115
contents xi
6.
Content: Markup and Genres........................................................................... 117 6.1 Content Representation.................................................................................... 118 6.1.1 Page Description Languages................................................................ 123 6.1.2 Markup Languages............................................................................... 123 6.1.3 Packaging Files..................................................................................... 124 6.1.4 Accessibility.......................................................................................... 125 6.1.5 Digital Rights Management................................................................. 125 6.1.6 DRM Technologies.............................................................................. 128 6.1.7 DRM in Use......................................................................................... 128 6.1.8 Standards Efforts.................................................................................. 130 6.2 Content Preparation......................................................................................... 130 6.3 Paper Genres Reborn....................................................................................... 133 6.3.1 eNewspapers......................................................................................... 133 6.3.2 eMagazines........................................................................................... 135 6.3.3 eTextbooks and Course Packs............................................................... 136 6.3.4 Electronic Journals................................................................................ 138 6.4 New Digital Genres.......................................................................................... 138 6.5 EBooks and Libraries....................................................................................... 140 6.5.1 A Pilot EBook Program in a Public Library........................................ 140 6.5.2 EBook Experiences in Other Libraries................................................. 143 6.6 Sustainability and Digital Preservation............................................................. 144
7.
Beyond the Book.............................................................................................. 147 7.1 Beyond Paper Capabilities................................................................................ 147 7.1.1 Domain- and Practice-Specific Capabilities......................................... 149 7.1.2 Within-Book Search............................................................................ 151 7.2 Portable Personal Libraries and Collection-Level Functionality...................... 155 7.2.1 Search at the Collection Level.............................................................. 156 7.2.2 Re-encountering................................................................................... 157 7.2.3 Gathering and Triage........................................................................... 160 7.2.4 Supporting Browsing with Computed Visualizations.......................... 163 7.2.5 Metadata for Personal Digital Libraries............................................... 164 7.3 Conclusion....................................................................................................... 166
References................................................................................................................ 169 Author Biography�����������������������������������������������������������������������������������������������������185
xiii
Figure Credits Figure 1.2 Zippy “Paper Trail” comic strip Copyright © 2001 Bill Griffith. Used with permission. Figure 2.3
from Marshall, C.C. and Ruotolo. Reading-in-the-Small: a study of reading on small form factor devices. Proceedings of JCDL’02, New York, pp. 56–64. Copyright © 2002, Association for Computing Machinery. Reprinted by permission.
Figure 3.2
from Marshall, C.C. and Bly, S. 2005b. Turning the Page on Navigation. Proceedings of JCDL’05, ACM Press, New York, pp. 225–234. Copyright © 2005, Association for Computing Machinery. Reprinted by permission.
Figure 3.3
Courtesy of the British Library.
Figure 4.2a
from Marshall, C.C., Price, M.N., Golovchinsky, G., and Schilit, B.N. Introducing a Digital Library Reading Appliance into a Reading Group. Proceedings of Digital Libraries 99, ACM Press, New York, pp. 77–84. Copyright © 1999, Association for Computing Machinery. Reprinted by permission.
Figure 4.6
based on Marshall, C.C. and Brush, A.J. 2004. Exploring the Relationship between Personal and Public Annotations. Proceedings of JCDL’04, ACM Press, New York, pp. 349–357.
Figure 4.7
based on Marshall, C.C., Price, M.N., Golovchinsky, G., and Schilit, B.N. 1999. Introducing a Digital Library Reading Appliance into a Reading Group. Proceedings of Digital Libraries 99, ACM Press, New York, pp. 77–84.
Figure 7.1
from Marshall, C.C., Price, M., Golovchinsky, G., and Schilit, B.N. Designing e-Books for Legal Research. Proceedings of JCDL’01, ACM Press, New York, pp. 41–48. Copyright © 2001, Association for Computing Machinery. Reprinted by permission.
xiv reading and writing the electronic book
Figure 7.2
from Marshall, C.C., Price, M., Golovchinsky, G., and Schilit, B.N. Designing e-Books for Legal Research. Proceedings of JCDL’01, ACM Press, New York, pp. 41–48. Copyright © 2001, Association for Computing Machinery. Reprinted by permission.
Figure 7.3
based on Marshall, C.C.. Collection-level Analysis Tools for Books Online, 2008, Books Online Workshop, 30 October 2008, Napa, California.
chapter 1
Introduction Publishers and technologists have been promising us electronic books of one sort or another for well over two decades. Indeed, many of them delivered products in the marketplace in what amounted to at least three distinct waves of technology and content. Each wave was heralded by hyperbolic claims about the disappearance of the print book, the death of text, and other ways in which literacy would be forever changed—claims that excited critics’ wrath and sometimes their derision in the wake of the products’ disappointing showings in the marketplace. Not only did the products fall short of readers’ expectations, but also the changes were slower to arrive than the more optimistic forecasts predicted and the products themselves demonstrated limited economic viability. For those of us whose lives have been inexorably intertwined with electronic texts all along, the immediate source of the disappointment has varied: reading on the screen just was not engaging enough; there were not enough titles in enough genres to satisfy our most immediate and fundamental textual desires; and beyond-the-book functionality was inadequate. More importantly, the hardware was too clunky and battery life was too short to make electronic books any more than a novelty purchased by early adopters, then stored unused in a drawer or forgotten in a hotel room. Yet—like Charlie Brown running at the football one last time, hoping to pull off one solid kick at the goalposts—we kept on trying to read and write the electronic book. Each wave of electronic books was spurred by somewhat different social and technical forces. Each anticipated different changes and each foundered on a new set of challenges that the products quickly revealed. It is instructive to look at the waves one by one if we are to understand the separate (but not separable) facets of the new medium.
1.1
GENERATION 1: A NEW WORLD OF HYPERMEDIA
The first wave of electronic publishing focused on what we could do with the new medium using pixels and fonts instead of ink and moveable type, screens in lieu of paper, and hypertext links in place of page flips. Companies like Voyager and Eastgate Systems not only developed electronic book software and figured out a way to distribute titles and sell them like most publishers would;
Voyager is now defunct; Eastgate can be found at http://www.eastgate.com/.
reading and writing the electronic book
they also worked through an initial economic model for the endeavor and nurtured a stable of writers and content preparation technologies. A small number of first-wave titles emerged. Writers learned how to create in the new medium. Dissemination depended on (relatively) inexpensive digital media such as laser disks (for high-end multimedia titles) or floppy disks (for experimental electronic literature, hypertext fiction, or documentary projects). Multimedia was Voyager’s strong suit as an early publisher of electronic content; the company initially adopted the laser disk as a primary means of distributing content and developed a series of titles that were well suited to multimedia. Consumer-friendly titles included history books such as Who Built America?; the work of more accessible performance artists such as Laurie Anderson; an electronic version of Art Spiegelman’s graphic novel about the Holocaust, Maus; and A Hard Day’s Night, which drew on the Beatles’ music. Laser disks were quickly replaced by CD-ROMs, but the focus of these companies remained on multimedia. The ability to transcend other intellectual limitations of linear text—limitations initially explored in Gedanken experiments by writers like Jorge Luis Borges (Garden of Forking Paths) and Julio Cortázar (Hopscotch)—appealed to a small set of literary pioneers, and hypertext fiction publisher Eastgate Systems was born. Research prototypes like Bellcore’s SuperBook System transformed marked-up files into hypertext that included a dynamically generated table of contents and index to facilitate what were then new kinds of reader navigation (Remde, Gomez, & Landauer 1987). Thus, the initial wave of electronic books produced a highly diverse (but sparse) selection of titles and a handful of research systems developed to explore issues associated with reading on the screen. In some sense, if you wanted to read an electronic book, you would have to adjust your reading preferences to what was available. Availability was further constrained by hardware platforms (some electronic titles were only available for Macs; others were only available for PCs), by limitations in how the titles were delivered ( just how much content fits on a floppy?), and by primitive reading hardware (color monitors were new, heavy, and expensive, and their resolution seems so low by today’s standards as to be utterly unusable). Furthermore, readers were tethered to their desktop computers; laptops, although available, were far from ubiquitous and were only portable if your arms were strong. In a 1992 New York Times article, Brown University professor (and a novelist himself ) Robert Coover forecast the end of books (Coover 1992), thus setting off what amounted to the first electronic book backlash; the best-known examples include The Gutenberg Elegies (Birkerts 1994) and several of the essays included in The Size of Thoughts (Baker 1997). The page versus pixel debate spurred further interest in the nature of documents themselves: Scrolling Forward (Levy 2001) is a thoughtful and reflective exploration of the document and the emergence of modern document technologies; The Myth of the Paperless Office (Sellen & Harper 2001) takes a more practical look at
introduction
why paper persists in an electronic world. A set of influential essays, The Future of the Book (Nunberg 1996), which puts the emergence of the information age into a historical context. Finally, The Order of Books (Chartier 1994) provides a cultural backdrop for understanding the role of books and how we read. Taken together, these works form a scaffolding for our subsequent thinking about electronic books and reading. By the end of the first wave of what we see now as eBooks, there were flagship electronic publications in many areas: the Journal of Postmodern Culture paved the road for a new era of scholarly publishing; titles like Afternoon ( Joyce 1990), Victory Garden (Moulthrop 1993), and Its Name Was Penelope (Malloy 1991) pioneered electronic literature; corpora like Perseus demonstrated that there may be more fruitful avenues for teaching and learning than traditional textbooks; and many early digital libraries like the University of Virginia’s eText Center showed that it was possible to offer the traditional canon of Western literature and special collections online. Although their coverage was by no means complete, these projects developed processes for the transition from print resources to digital. At the same time, the World Wide Web was emerging as a new mode of distribution for the burgeoning born-digital content, starting with high energy physics preprints, but moving quickly into an unimaginable sea of electronic genres. Although individual efforts maintained a measure of internal consistency, at this early date, there was little thought given to the standardization and sustainability of electronic content.
1.2
GENERATION 2: EBook HARDWARE ARRIVES
The World Wide Web and the early electronic texts that it delivered were not called eBooks. eBook was a word coined when companies like Nuvomedia introduced the Rocket eBook and
Naturally I have forgotten or have otherwise omitted volumes that have had a tremendous influence on the thinking that has taken place as the computer screen has become our primary venue for reading. The adventurous reader is urged to pay close attention to the bibliographies each of these authors has painstakingly assembled—there is much interesting reading to be pursued.
http://www.perseus.tufts.edu/hopper/.
http://www2.lib.virginia.edu/etext/index.html.
The Web had a set of standards with a momentum of its own; native Web documents were marked up with HTML, which was standardized to some extent. But the early multimedia titles I have referred to so far conformed to the storage formats required by their own presentation software.
The Rocket eBook and its parent company Nuvomedia were acquired by Gemstar TV Guide International for what was then considered a shockingly high price tag of 200 million dollars. The Softbook Reader and its parent company, Softbook Press, were also acquired that same year. The hardware was consolidated and offered as the REB1200 product, which limped along in the marketplace for several more years.
reading and writing the electronic book
SoftBook Press offered its comparable SoftBook platform; both eBook readers hit the market in 1998 amid greater fanfare than adoption. The eBook platforms only displayed eBooks that were prepared specially for them. Figure 1.1 shows these competing readers. Both the Rocket eBook and the SoftBook Reader were noble forays into purpose-built hardware, but they were clunky: the Rocket eBook weighed in at a bulky 22 ounces and the SoftBook was not exactly svelte either at 2.9 pounds. Furthermore, neither reader stored enough content to meet the promise of a portable personal digital library: the Rocket eBook held about 10 titles and the SoftBook Reader held about eight. The emphasis of these products was to provide the consumer with reading hardware that was book-like in its size and shape. Both companies nurtured partnerships with traditional publishers and developed digital rights management (DRM) mechanisms; eventually, they were compliant with the emerging Open eBook (OEB) standard (discussed in Chapter 6). Immediately there were complaints about battery life (which in the case of the SoftBook Reader was insufficient to handle a coast-to-coast airplane flight); consumers balked at carrying an extra device that, while not as heavy as a laptop of the time, was still significantly bulky. Consumers and public libraries (some of which were enticed into sponsoring ambitious eBook programs) were not broadly satisfied with the range of content and the ease with which they could prepare and transfer their own documents onto the platforms. Some libraries, both public and academic, reported on their experiences with these early devices (e.g., see McKnight & Dearnley 2003); we report on a California public library’s pilot eBook program in Chapter 6.
FIGURE 1.1: The second generation of reading platforms. (a) Nuvomedia’s Rocket eBook. (b) SoftBook Press’ SoftBook Reader.
introduction
The second generation of eBooks also saw the introduction of reading software, generally aimed at providing functionality that would allow the user to annotate, bookmark, look up words, search within the text, and interact with the material in basic ways. Reading software ran on conventional PCs and laptops, but it was also designed to run on handhelds such as Pocket PCs, Palm Pilots, and other small mobile devices. This software included Microsoft Reader, which in 2000 rolled out an extensive advertising campaign that once again heralded the end of reading as we know it. The ads crowed about the coming demise of “the pBook”, which prompted another round of backlash from scholars and critics. Figure 1.2 shows a Bill Griffith comic strip that appeared not long after the first Microsoft Reader ad was published in The New Yorker. At the second generation’s onset, in 1998, the Research Library Group’s Walt Crawford published an essay in Online Magazine called “Paper Persists: Why Physical Library Collections Still Matter,” in which he said: Reading from digital devices . . . suffers in several areas—among them light, resolution, speed, and impact on the reader—and there has been essentially no improvement in any of these areas in the last five years . . . It’s just too hard to read from a computer, and it doesn’t seem likely to get a lot easier (Crawford 1998). Four years later, the new hardware and software had not made a significant dent in critics’ skepticism. For example, Jimmy Guterman wrote a short critical piece on CNN.com in late 2002 that claimed readers were deriving insufficient benefit from eBooks, that publishers just did not get it:
FIGURE 1.2: Backlash prompted by the launch of a second generation of eBook products (ca. 2001). Copyright © 2001 Bill Griffith. Used with permission.
reading and writing the electronic book
Publishers need to add value. Current onscreen magazine systems . . . simply load a bulky software program and copy-protected PDFs of the magazines onto your system. There’s some rudimentary searching, zooming, and annotating, but that’s all . . . There’s . . . no desire to do anything but replicate a flat print publication (Guterman 2002). Research projects, such as Fuji Xerox Palo Alto Laboratory’s (FXPAL) active readingoriented XLibris (Schilit et al. 1999) and Xerox Palo Alto Research Center’s (PARC) conversionoriented UpLib (Bier et al. 2004), made serious attempts to address this question of adding value to onscreen reading applications. XLibris investigated pen-based functionality that would tackle the difficulty of annotating and otherwise engaging with serious reading material such as technical articles, textbooks, legal documents, and intelligence sources. UpLib took on the problems associated with gathering and preparing the documents the user wanted to read. These projects were successful in their research communities but had scant influence on the eBook products that were constrained by hardware costs and limitations, publishers’ invariable concerns about piracy, and other market forces. Thus the second generation of electronic books rolled in with considerable fanfare and receded much more quietly, all but disappearing from the record. All the while, a revolution was taking place unnoticed on the sidelines as many people simply started reading newspapers, magazines, blogs, and other born-digital content on the screen, ignoring eBooks, and making do with whatever hardware they had in hand. During field interviews, I began to hear statements like this: “My dad and I like to read newspapers online. It’s faster and free.” Faster and free—the screen was winning the battle with little attendant anxiety or ado. Mobility, convenience, and portability were making a compelling argument for a turn away from paper. In a 2001 interview, a college student told me: if I’m going home to Colorado, I have to really be sure I’m going to read something if I’m going to bring it. Otherwise, why should I bring it? This thing [a handheld device with eBook software and all his course materials], I was like, “I’ll bring it, and if I read it, I read it; if I don’t, I don’t.” It doesn’t matter. It’s small; it’s handy (Marshall & Ruotolo 2002).
1.3
GENERATION 3: ePAPER AND THE QUIET REVOLUTION
The third generation of electronic book platforms (Figure 1.3) began to take advantage of a convergence of enabling technologies (low-power bistable displays, capacious storage, lighter hardware,
It is notable that Wikipedia’s entry for the RocketBook is very brief and the corresponding entry for SoftBook is altogether absent. Nor are the earlier products linked to Amazon’s Kindle as any hint of historical precedent. In fact, a search for SoftBook in Wikipedia asks the user, “Did you mean: songbook?”
introduction
FIGURE 1.3: The third generation of eBooks: low-power ePaper-based bistable displays. (a) Amazon’s original 6-inch diagonal Kindle. (b) Sony’s Reader with touch-screen display. (c) Kindle DX with 9.7-inch diagonal display.
the availability of digital content, and ubiquitous wireless) with warming consumer attitudes toward reading on the screen and purchasing unbundled content (using an iPod-like model of what a reading device might be like). Several different reading platforms—most notably, Amazon’s Kindle and Sony’s Reader—were introduced to a public that found them less alien and less of a novelty item than they had the previous generation of eBook devices. Interestingly, the third generation of portable eReaders—while considerably more appealing from a form factor and battery life standpoint—has yet to unleash the real power of electronic texts. The ability to carry a lifetime’s worth of reading in a mobile device is a compelling idea, and yet Clifford Lynch’s vision of a portable personal library has yet to be realized. Not so far, anyway, although the obstacles to such a thing have diminished substantially over the last decade-and-a-half of eBook history: Given the historic price–performance trajectories for storage, in a few years at least some high-end appliances will house hundreds, if not thousands, of books simultaneously, and certainly laptops with software book readers will house thousands or tens of thousands of books at once. Think of portable personal digital libraries, not portable electronic books, as the future role of these appliances (Lynch 2001, emphasis added). Then too, we have yet to unlock the potential collective effects of a world of electronic readers and writers. Functionality that involves computation over large collections of digital texts—along
reading and writing the electronic book
with wisdom-of-crowds approaches to analyzing our interactions with these texts—has yet to be fully investigated and implemented.
1.4
THIS BOOK
We are still hung up on the physical form of the book and its place in our culture. Perhaps this is rightfully so. Why, then, is it important to recount the short and somewhat disappointing history of reading and writing the electronic book? What is there to know? At first blush, the answer would seem to be, not much. An electronic book is like a print book, only with pixels instead of toner. Indeed, there have been many research projects and consumer products devoted to making electronic books seem familiar. Yet, if we look a little closer—if we get beyond cycles of hyperbole and disappointment—it would seem that we have learned quite a bit. We have gotten somewhere. There have been spectacular efforts documenting and analyzing the larger social forces at work on the book, on text, and on media. Series of post-McLuhan discussions have taken place, as well as countervailing efforts to discredit electronic reading and writing and to hold steady against the forces of change. Rather than joining the cacophony of voices from the literary and social science worlds, some of which we’ve already mentioned, this book will keep its nose down—in the book, so to speak—and examine instead a rather more pragmatic set of issues and developments that have arisen over the last few decades. This book is intended to reflect on its counterparts in a quotidian way, drawing on sources from information science, computer science, and human–computer interaction, but especially on the results of studies I have conducted with colleagues and by myself over the last decade-and-ahalf. In this book, we will steer clear of the turbid waters of critical theory and the more abstruse studies of human cognition; instead, we’ll stay focused on how the next generation will read and write the electronic book. We will start with a look at what an electronic book will need to do, beginning with a closer read of reading itself: the assumptions that underlie reading; the different ways that people read; the different reasons they have for picking up books or basking for hours in the screen’s warm glow. Because eBooks have an inescapable need to render words on the screen, this chapter first turns its sights to layout and legibility, developments that draw on the mature design disciplines of typography and text layout. Hand in hand with these techniques go studies of legibility. The second chapter closes with high-level coverage of eBook hardware requirements and recent advances in display technology. Once these basics are out of the way, Chapter 3 discusses reading-related functionality: annotation, navigation, linking, clipping, and bookmarking—in short, all of the things people can
introduction
do with an ordinary print book as they interact with their reading material. The emphasis in this chapter is on practice and on an analysis of what this practice means for the form and function of the electronic book. Like Chapter 2, Chapter 3 closes with a discussion of specialized hardware, this time to support interaction—support that goes beyond the traditional mice and menus. Reading is, above all, a social activity, so Chapter 4 takes a close look at the sociality of reading: how we read in a group, how we share our reading materials, and how we share the artifacts of our reading including annotations, bookmarks, clippings, and the books themselves. As is true of any discussion of sharing, we must pay attention to the associated issue of privacy; in the digital world, everything from page views to reading lists has become fodder to fuel our social networks. What we read (and what we make of it) is now potentially available to one’s online friends and to a broader community of readers. Because study results are integrated throughout the book, individual studies of reading and eBook use are not relegated to their own chapter. Instead, Chapter 5 talks about how a researcher might go about designing some types of studies that are particularly appropriate for exploring reading and for evaluating the results of eBook field deployments. Studies of reading and readingrelated activities are difficult for specific reasons; we will go through these reasons and discuss ways of compensating for the difficulties. No book about electronic books would be complete without an explicit discussion of content, i.e., how the electronic book is written. Hence, Chapter 6 delves into the underlying representation of content and efforts to create and apply markup standards to texts. This chapter also takes a practical look at text preparation, coupled with an examination of how print genres have made the journey to digital and how some emerging digital genres including blogs, wikis, and hypertext fiction may be realized as eBooks. As we discuss content, we also hit on the high (or some would say low) points of economic and legal underpinnings for the electronic book marketplace, most notably DRM and other mechanisms for protecting content from piracy and unrestricted copying. Finally, because we are on a computer, we can do things with digital texts that we were never able to do with print books. For example, computational power allows us to apply existing search and way-finding mechanisms inside the book; translation facilities and online references can be ready-to-hand; support for different kinds of reading that are part and parcel of different disciplines (such as law, education, and analysis) may be implemented within an eBook architecture that supports such extensions and adaptations. Chapter 7 discusses how we can do more with electronic books than just read them cover to cover. I devote the concluding sections of the chapter to collection-level functionality for digital texts, such as tools for gathering, triage, and analysis.
For example, just studying reading as it naturally occurs is difficult and is even considered a bit creepy: watching people read when and where they normally read is seldom comfortable for the observer or the observed.
10 reading and writing the electronic book
Earlier, I quoted Clifford Lynch as saying, “Think of portable personal digital libraries, not portable electronic books, as the future role of these appliances.” Thus, the chapter as a whole focuses on what it will take to make eBook platforms into more than simply books rendered electronically and what it will take to transform eBook platforms into portable personal digital libraries. In the end, some basic philosophical questions remain: Just how faithful should electronic books remain to print books? Should we say, the book is a known form, and document genres have coevolved with human practice; hence, the print book is the apotheosis of a particular mode of communication? Or should we take the radical position that our colleagues took during the 1990s when they heralded the end of books and the death of text? How should we even set about answering these questions about the future of the book and reading? My hope is that by the time the reader reaches the end of this book, the reader will feel equipped to venture opinions and write the next set of op-ed pieces, perform the next set of studies, build the next iteration of eBook prototypes, vet the next generation of eBook business plans, or engage in a heated argument with the stranger in seat 17C about the future of reading. • • • •
11
chapter 2
Reading When we talk about electronic books—eBooks—what we’re talking about is reading. Reading is the fundamental way that we engage with books, isn’t it? It seems self-evident and hardly worth talking about. And that’s exactly why it is worth talking about. EBooks have evolved amid an atmosphere of anxiety and skepticism. Anxiety has been provoked by the literati’s numerous predictions of the end of books, of changes in reading as we know it (Bolter 1991); this anxiety has provoked a tremendous backlash (Gass 1999; Birkerts 1994). Skepticism has followed in the wake of the collapse of the paperless office (Sellen & Harper 2001): why would anyone read on a computer screen when paper is so malleable and affords such natural interaction? As evidence of this anxiety, Gass (1999) wrote a predictably nostalgic article about the pleasures of reading print books. It did not go unnoticed. To introduce an editorial by humanities scholar and writer, Diane Greco, Eastgate System’s chief scientist Mark Bernstein wrote: Pundits nostalgically continue to disparage new media forms. William Gass issued the latest sortie in a recent issue of Harper’s Magazine, arguing that “words on the screen” (his phrase) cannot compete with the pleasures of paper-and-ink textuality, because digital media do not record the serendipitous events that can occur in individual readings. To support his claim, Gass points to his discovery of certain jam-stained pages in his copy of Treasure Island, which, to him, evinced praiseworthy engagement with the book. Although Gass seems to have outgrown the juvenile excitement to which the pages attest, nevertheless he values his copy, with its sticky leaves, as a relic, as tangible evidence of better days before everything became, in Gass’s words, “data day and night.” But surely Gass is talking about a certain kind of reading and a certain kind of interaction. And certainly we all have deeply held stereotypes about the nature of reading.
This sidebar was taken from http://www.eastgate.com/HypertextNow/archives/Gass.html. (Re-retrieved June 10, 2009.)
12 reading and writing the electronic book
What’s worse, we’re looking at a moving target: new genres emerge; readers’ purposes vary; new technologies are just over the horizon. Ten years ago, Web logs (i.e., blogs) were so new that they were worthy of comment in The New York Times. Fifteen years ago, John Seabrook, writing in The New Yorker, felt it necessary to explain what a home page was. Yet these genres are so much a part of our lives by now, we wouldn’t question them or why we read them: emerging genres are sneaky. Even the stability of ludic engagement, reading in its most fundamental form, is being questioned. Ludic engagement is that form of reading that we think of when we say someone is “lost in a book.” eBook design has been prone to focus on this sort of immersive reading—deep involvement with a single work—despite the thought that it is fast disappearing, given the availability of so many content choices and our social tendency toward fragmented attention. In his thoughtful book about documents and reading in the digital age, Scrolling Forward, David Levy discusses some of the factors that mediate against immersive reading: Changes in the technologies and the character of modern life may be putting an end to reading in depth. That’s the fear, at any rate, in some quarters . . . It isn’t that the book has gone away, but rather that the cultural conditions for [deep] reading . . . are fast disappearing. (Levy 2001, pp. 108–109) Levy differentiates among three types of engagement with documents: intensive reading, the traditional picture of reading that is the basis for much electronic book technology; extensive reading, which acknowledges that a reader may be using many different books at once; and hyperextensive reading, a distinction that brings the fragmented quality of human attention into sharp focus. Not only are readers using many different books at once, they are also reading parts of books and they may be reading these fragments out of order. Reading is aligning with our fragmented lives. Although at last count we’re printing more than ever, we may be printing not because we’re reading more, but rather because we’re reading less. In a recent internal survey, the most common reason that people printed documents was to read them later. In other words, we print so we don’t have to read. Although it’s tempting to think that this is a harsh conclusion and that we print to defer reading, interviews reveal that this deferred reading may never take place: once a document has been printed, it can be consulted as needed and is thus never read in the way that we conceive of as reading.
Katie Hafner, I Link, Therefore I Am: a Web Intellectual’s Diary, New York Times, July 22, 1999.
John Seabrook, Home on the Net, New Yorker, October 16, 1995.
reading 13
New reading technologies are also on the horizon: bistable displays give us the ability to furnish eBook platforms with low-power screens. But it is dangerous to fall victim to technological determinism; technology, by itself, says little definitive about where we are going. In 1945, President Roosevelt’s science advisor, Vannevar Bush, wrote an article for the Atlantic Monthly that many people regard as prescient. In As We May Think, Bush foresaw technologies like hypertext, realized in a device he called the “memex”: The owner of the memex, let us say, is interested in the origin and properties of the bow and arrow . . . He has dozens of possibly pertinent books and articles in his memex. First he runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items. Occasionally he inserts a comment of his own . . . Thus he builds a trail of his interest through the maze of materials available to him (Bush 1945). But he also quietly missed the mark on one important social change. A second quote, also from As We May Think, illustrates it the best: The advanced arithmetic machines of the future . . . will have enormous appetites. One of them will take instructions from a whole roomful of girls armed with simple keyboard punches and will deliver sheets of computed results every few minutes. There will always be plenty of things to compute in the detailed affairs of millions of people doing complicated things (Bush 1945). If we take this quote, and not the other, as the linchpin of change foreseen, we come away with a different picture. Computers were not going to be personal machines; they were going to stay put in their role as stodgy number crunchers. And women were not going to be writing blogs on computers; they were going to be relegated to the keypunch room. In Vannevar Bush’s time, who would have believed that men would learn to type? Certainly not Xerox executives, who, upon seeing the first personal computer at Xerox PARC, reminded the researchers that men would never do their own typing. Technological change can mix with social change in ways that no one really foresees.
I have heard this story from several different sources. Unfortunately, I don’t have the citation necessary to cement the story’s authenticity, so it will have to remain in the status of urban myth.
14 reading and writing the electronic book
Hence, in this chapter, we’ll start at the most basic level of eBooks: reading. We’ll examine assumptions about reading, explore some of the purposes of reading, and set forth a reading typology. Once we’ve sketched out these basic elements of how a reader engages with a book, we can examine the flip side of the question from the book’s perspective: why a print page looks like it does and why an onscreen page looks like it does. Then, because reading from computer screens has always stirred up a great deal of controversy, we’ll look into some studies of the effect of layout on readers’ performance and check our intuitions about why it’s been difficult to shift readers from the print page to the screen and why this difficulty is easing.
2.1
READING
The subject of this chapter’s first section is perhaps the most self-evident of all: what do we assume about reading, are these assumptions true, and what distinctions do we want to make among the kinds of reading that people do at work and at home? Looking at these assumptions and distinctions is important. They ultimately arise when we question the veracity of reading research (“Sure it works that way in the lab. But what happens when people are reading what they want to read?”) and when we question the reach of various products (“Sure, you might want to read a Star Trek novel on a Kindle, but will you like it when you’re reading Proust?”). Critiques of eBook technologies are deeply rooted in the types of reading that we picture: of course, you’ll need to annotate if you’re reading to learn; of course, you’ll need different modes of navigation if you’re reading for reference; of course, you’ll need a backlit device if you’re reading in the dark, in bed. And so we’ll start at the beginning.
2.1.1 Assumptions About Reading Let’s look at some of the assumptions we make about reading. First of all, we think of reading as a stationary activity, something people do seated at their desks in their offices, at their carrels in a university library, and in their easy chairs in their living rooms. We think of reading as passive, an activity in which at best we move our eyes across the screen, from word to word, from page to page, until we’re done. And we keep going in that unstoppable march forward from word to word until we’re done reading and go on to the next thing. And as far as being a solitary pursuit? Most certainly. Reading is an intimate act: we do it alone. We read in private. It’s just us and our cognitive processes. Together. Alone. And when we read for meaning (as opposed to reading purely for pleasure), we assume that reading is fundamentally about the information contained in the documents. This is not to say that we just absorb the material at face value—we might skim quickly, we might analyze the material
reading 15
deeply, or we might synthesize it with the other things we know—but reading is about the content of the book, not the container. But if we look at these assumptions, one by one, they are not straightforward at all. Mobile. The first assumption we can call into question is that reading is stationary. It is stationary only from the perspective that people can get into trouble if they walk and read at the same time: they can fall down a manhole or walk into a pole. Often, people carry their reading material with them to read when they have time or when they find a place conducive to reading. It doesn’t matter whether they’re reading on a mobile device or on paper. For example, a researcher said this of a technical paper she was reading: I took it [a technical paper] home a couple of times, but it never got anywhere there . . . You can see that it’s totally trashed. It’s been to the pool. It’s been just about everywhere with me (Marshall et al. 1999). If a mobile device is successful, it presents that same opportunity to change locations and situations until the right time and place arises for reading. Recalling the interview segment quoted in the first chapter, the Pocket PC’s success as a reading device hinged substantially on its portability: If I’m going home to Colorado, I have to really be sure I’m going to read something if I’m going to bring it. Otherwise, why should I bring it? This thing [a handheld with eBook software], I was like, “I’ll bring it, and if I read it, I read it; if I don’t, I don’t.” It doesn’t matter. It’s small, it’s handy (Marshall & Ruotolo 2002). Mobility doesn’t always mean that a dramatic change of venue must occur. Micromobility is a common phenomenon; a reader may move to a more comfortable chair or to a place where the light is better: I usually read in one of the chairs in [the living room]. That’s partly because I don’t have a desk lamp in here . . . [The chairs are] very comfortable (Marshall & Ruotolo 2002). The mobility of reading is underscored by taking a comparable look at writing. In interview studies, when I have asked “where do you work?” to set up places to conduct interviews with highly mobile students, researchers, and other professionals, most of them have answered more in terms of where they wrote than where they read; they see writing as their fixed, visible work, while reading is a highly mobile, highly fluid activity. Is this finding still true in an age of light laptops and increasingly capable small devices? Is reading still noticeably more mobile than writing? In a 2008 study of scholarly writing, I found that
16 reading and writing the electronic book
despite the quest for seamless replication of content (thus making both reading and writing possible everywhere), writing continued to be more place-constrained than reading (Marshall 2008b). Thus, all evidence continues to point to the fact that reading is mobile. And because it is mobile, when we contemplate electronic books, we have to think of them as something that will be toted along with everything else we carry around. Early generations of eBooks were heavy and bulky: it was natural to react to them as just another mobile technology that would need to be carried—and charged—along with all of the other special purpose gadgets we carry and charge. It is also necessary to consider how material will get on the device (and how the device will synchronize with other mobile devices). Will reading material accumulate and become a personal library? Or will ad hoc collections grow on each device according to the device’s use? We will look at these questions later in the book. Interactive. It is all too easy to stereotype reading as a relatively passive activity: videogames are interactive, reading is passive. Mortimer Adler famously suggested that critical engagement with a text is necessarily active (Adler and van Doren1972); hence he (and others of his ilk) use the term active reading to promote a higher degree of critical interaction with one’s reading and to characterize the thoughtful reader as being in perpetual analytic dialog with the writer. But interactivity need not be so high-minded as that. Grab a used textbook and you will find all kinds of annotations, worthwhile and otherwise. Magazines at the laundromat will have articles torn from them. Even the way books and papers are organized on our physical or digital desktops will remind us that reading begets a great deal of interaction. We discuss basic types of interaction in Chapter 3, how these interactions play into the social life of eBooks in Chapter 4, and more advanced interactivity in Chapter 7. Social. Is reading really a solitary pursuit? In Scrolling Forward, David Levy tells us that reading is inherently social: It is also worth noting that solitary reading always was, and still is, inherently social: how we read is ultimately determined by social conventions and community membership (Levy 2001). But beyond that abstract notion of sociality—that reading and writing are fundamentally communicative acts, taking place within a cultural context—there are ways in which we see evidence of reading together all around us. Students read together in a classroom; drivers read billboards together as they speed by the landscape; and people browse the Web together, possibly with one person looking over the other’s shoulder or in a multiway Skype call. If you think about it, you’ll be able to come up with many ways that people read together. People also share the artifacts of their reading, either inadvertently (they loan someone a book that they’ve annotated) or purposefully. They may send each other clippings or recommend books to one another.
reading 17
In the end, the sociality and sociability of reading is uncontroversial. Material. It is easy to treat the idea of information extravagantly: as something that explodes, as something there is too much of, or as something that has no material form. I have used this quotation from Geoff Nunberg many times in the last decade and a half, in many circumstances, but my overuse has rendered it no less evocative: Reading what people have had to say about the future of knowledge in an electronic world, you sometimes have the picture of somebody holding all the books in the library by their spines and shaking them until the sentences fall out loose in space . . . (Nunberg 1993). A book has an easy materiality, a materiality which can be purposefully examined, seen, and even smelled. In an article about Arion Press, an apprentice printer tells the journalist, “this work is so tactile—it’s a sensory experience,” noting the pleasure of working with ink, paper, cloth, and leather. Can you envision an intern, marking up an electronic text according to TEI dicta, saying that the keyboard has an enticing tactile quality? The materiality of books is appreciated implicitly. Online used book seller Alibris had a longrunning ad campaign in which they showed the tattered covers of memorable paperbacks—Coming of Age in Samoa, On the Road, and Catcher in the Rye—and offered the ability to reclaim the book that you’d lost “in a bus somewhere near Kathmandu.” The tattered cover is, in and of itself, a reminder of the book and all of the experiences surrounding it. EBooks inhabit an ambiguous place in the materiality of reading. The eBook has a form, but what does it mean? The heft of an eBook tells the reader nothing: The Elements of Style has the same heft as Fowler. The eBook page is malleable and may be reformatted on the fly. But reading remains essentially physical. One of the early objections to eBooks was the plaintive, “I could never curl up with an eBook.” A graduate student who participated in the Pocket PC study countered that objection, saying: And I heard things like, “Oh I could never curl up with a computer.” And I always test that for myself. And in fact, I would lie back on my couch with my feet up on this end, my head propped against the pillow there (Marshall & Ruotolo 2002).
In Heidi Benson’s “The Power of the Press” (San Francisco Chronicle, April 20, 2003).
Fowler’s Modern English Usage is a comprehensive style guide of substantial heft; The Elements of Style is a slender and idiosyncratic volume.
18 reading and writing the electronic book
And with that, she demonstrated her ability to sit comfortably on her sofa, reading textbooks on her laptop. Thus, despite our tendency to think of anything on the screen as virtual, reading itself is material, and many eBook designs make an effort to reclaim that material, physical quality.
2.1.2 Purposes of Reading One of the early studies that was performed with the explicit purpose of informing the design of what the paper referred to as “digital reading devices” was a diary study (Adler et al. 1998); what is important about this study is that it reminds us of the multiple purposes people have for reading, beyond simply reading for information. Thus, the categories include reading to support discussion, reading to edit or critically review text, reading to learn, and reading to answer a question. Schilit et al. (1999) break down some of these purposes for reading according to the level of interaction they require and whether they cross boundaries among multiple texts. They point out that reading a novel or browsing the newspaper requires relatively little interaction with the text and that studying a textbook, reviewing a proposal, or keeping up-to-date professionally requires substantially more interaction. Single text activities include reading a poem aloud and using a diagnostic manual; multiple text activities include surfing the Web or researching a topic. We can extend this contrast between active reading and immersive reading to form a space of reading types. This space is populated with examples in Figure 2.1. Marshall (2007) characterizes the reasons for reading a newspaper as including reading primarily for relaxation and as a diversion; reading to following the narrative of specific breaking stories; reading in response to particular recommendations; and reading broadly to stay informed or to keep up with the events of the day. What is clear from these varying discussions of reading’s purposes is that reading is seldom one thing, even given a single reader engaged with a single text. Furthermore, readers may not be able to fully articulate their reasons for reading. In the Times News Reader study (Marshall 2007), participants reported a variety of reasons for reading the publication (e.g., no stories about Paris Hilton on the entry screen or that the stories are well thought out and fact-checked); yet these reasons were somewhat dissonant with their actual reading practices (they are primarily seeking the latest news and may not look too far beyond the headlines). Reading to stay informed might include a check of the advertising inserts: What I did miss, funnily enough are the ads. It’s weird, but the Saturday and Sunday editions, especially the Sunday edition, there are a lot of ads. [You can find] specials on Broadway shows. I do use that a lot to book my tickets online (Marshall 2007).
reading 19
Active reading (purpose in mind) Requirements: interaction, manipulation
Executive skimming email to respond Professional looking through business magazines to see “what’s new”.
Coffee drinker paging through a newspaper
Lawyer reading key cases and viewing a video deposition Music student preparing for a final in music theory Analyst reading “hot” cables
Airplane passenger reading a novel
Immersive reading (focused attention) Requirements: transparency, legibility
FIGURE 2.1: Active reading versus immersive reading: a space of examples.
Reading a newspaper to stay informed may be virtually indistinguishable from reading a compelling novel or from tracking down information from multiple sources to answer a question: For example, there was this news about Wesley Snipes being arrested. Which was flashing on the banner stuff. And I wanted to know. . . . I came back home late, so I missed the evening news. It was that in-between couple of hours. It was 6:35, actually. So I missed the 5:00, the 5:30 news, and I wasn’t listening to the anchor. In any case, I went online—actually, the TV was on. And the TV was talking about this. And I went on the Times Reader to try and see if it was there (Marshall 2007). It is easy to see why categorizing a reader’s purpose is so problematic. Nonetheless, it is important to acknowledge that reading may be motivated by a multiplicity of purposes.
2.1.3 Types of Reading Just as there are multiple purposes for reading, there are distinctions among the different ways that people read. At the very least, we should distinguish among five rough categories that are easy to identify on video recordings. We use three of these categories for analytic purposes in Marshall and
20 reading and writing the electronic book
TABLE 2.1: Reading types. Type
Characterization
Reading
Canonical careful reading. The reader traverses the text linearly. The aim is comprehension.
Skimming
Faster than canonical reading. Traversal is still linear, but comprehension is sacrificed for speed. The aim is to get the gist of the text.
Scanning
Faster than skimming. Traversal becomes non-linear; the reader looks ahead and back in the story. The aim is often triage or to decide on further action.
Glancing
Pages are turned very quickly; the reader spends almost as much time turning pages as looking at them. The aim is to detect important page elements (e.g. beginnings and endings of articles, photos or figures, headings) until something holds sufficient interest to transition to another type of reading.
Seeking
Reader scans quickly for a particular page element (e.g. proper nouns) with an aim orthogonal to full comprehension. A study participant described looking for names in a magazine article: “I don’t know anything about pop music. I should because we’re always stumped in The New York Times crosswords by the pop music characters. I do know Beyoncé is an important character who appears in the crosswords. So I may get a few names out of it. But that’s about it” (Marshall and Bly 2005b)
Rereading
Rereading is a meta-type that is included in the table as a reminder that any type of reading may be occur multiple times.
Bly (2005a), but we also discuss the other two categories (skimming and seeking). These categories are evident in other discussions of reading but are not set out independently from reading purpose. Table 2.1 identifies five categories; a sixth orthogonal category, rereading, is also introduced. No discussion of reading types would be complete without mention of rereading. It is important to remember that some skimming and scanning is performed either before or after deep reading takes place. Thus, a text can be approached multiple times (sometimes with multiple purposes in mind). Rereading may occur in a different venue (i.e., on a different computer, or in print if the
reading 21
skimming was done on paper) than it did originally; it may serve as an introduction (if it is done before) or a reminder if it is done afterward. Unless a text is specified explicitly as new, we need never assume that it has not been seen before. The Pocket PC study (Marshall & Ruotolo 2002) reminds us that an eBook might be a com plementary reading venue, existing in an ecosystem of personal devices that also may be used to read. In that study, most of the students used the electronic texts on the handhelds to become familiar with shorter texts and excerpts, not to read deeply. They used words like skim and glance to describe how they read on the Pocket PCs. In general, the students characterized on-screen reading this way to contrast it with how they read print materials. Thus, in this situation, on-screen reading was not sustained deep reading but rather quick reads of course texts, possibly in situations in which the student’s attention was divided (such as in-class reading), with the intention of reading the text again. The students skipped around in the electronic texts, focusing on some parts more closely than others. The ability to search and focus on short segments of a longer text and to navigate through an extensive set of familiar materials seem to be universally cited strengths of reading on the screen. Thus, rereading across multiple venues should hold a prominent position in our taxonomy of reading types.
2.2
LAYOUT, TYPOGRAPHY, AND LEGIBILITY
What do we mean when we talk about layout? Usually, layout refers to various elements of page design: how a linear flow of text is represented on the print page or in a window on the computer screen. In the print world, page layout is largely the job of the publisher. That is, neither the reader nor the writer has much to say about how text appears on the page; the writer supplies the publisher with content—probably in digital form—and the reader buys finished book, and only through indirect feedback (the writer grumbles to her publisher; the reader buys an edition with a print style she believes to be more legible) does either have any control over how the book looks. On the other hand, a designer working for the print publisher may put a great deal of thought into choosing a type face and size; specifying the margins, column widths, and paragraph leading; placing figures and charts on the page; opting for text justification and hyphenation; and generally attending to the book’s aesthetics and legibility (Schriver 1997). Design matters become less straightforward when we start talking about reading on the screen. Digital publications with significant resources may draw on layout techniques from the print world, with publishers and designers controlling the page layout to a significant extent. At the
The online edition of The New York Times (http://nytimes.com) is a good example of this trend. Many publications have put extensive resources into their visual brands and are anxious to maintain these recognizable brands in their online sites.
22 reading and writing the electronic book
other extreme, on-screen layout may be left in the reader’s hands to a much greater extent, as it was originally in Web browsers, with the reader choosing the type face, the font size, and the general look of the text, and the writer/publisher specifying only a few functional elements of the document, such as heading levels and where paragraphs begin and end. Unlike print documents, digital documents offer the opportunity for the page layout to be computed at display time, guided by algorithms and heuristics (rules of thumb). Thus, the application—and therefore, the software developer—and the reader may play a larger role relative to the publisher in determining the final look of a document. Text size, style, position, and rendering techniques may be optimized to take advantage of the display hardware, in conjunction with the characteristics of the material being displayed and the reader’s own preferences. Hence, in electronic books, we often find that the final layout is produced through an interaction among three factors: (1) the content representation (i.e., how the content has been marked up, and how styles have been defined by the publisher); (2) the eBook software (which interprets the markup and applies any additional constraints imposed by a particular platform); and (3) readercontrolled settings (e.g., font size). In this chapter, basic elements of layout will be introduced and defined, along with a discussion of some common heuristics for computing a readable page display. We will expose only the tip of the iceberg in this chapter; the interested reader can find many published resources and ongoing discussions of designing typefaces and page layouts for the screen. Does layout matter, beyond the basic aesthetic concerns that are intrinsic to design? That is, most of us are aware that some documents and books just look better than others; we are also aware, perhaps acutely, that an ugly book is uninviting to read and that it is possible for text to be presented on the ragged edge of readability, with type that’s too small or too tightly spaced, or with lines of running text that are obviously too long for comfort. And certainly, we are aware of a look that is part of a publisher’s brand. But does layout have any bearing on actual reading performance? Does the way that the words are presented on the page have any effect on the speed with which a reader may make her way through the text or on her comprehension of its content? Furthermore, do these design principles change a reader’s immediate reaction to the content? In other words, is an argument more persuasive if the page is well laid out? Certainly, we might expect to have a different reaction to the same book read aloud by Paris Hilton and by James Earl Jones, but in the case of page design, are looks that important?
There are a great many long-running forums, blogs, and online resources on document design and typography; the enthusiastic reader is encouraged to go off and discover them.
For example, see O’Reilly’s technical book series.
reading 23
In general, the overarching importance of document layout is an area of some controversy; even empirical results may not be conclusive. Although we’d all agree that it’s more pleasant to read a well laid-out book, we might not say that a beautiful argument is more persuasive or that an aesthetic layout allows a reader to assimilate material more quickly and accurately. Thus, after discussing basic aspects of text layout, this chapter will cover some of the studies that have been performed in an effort to resolve these performance questions and quantify the differences they introduce. By now, almost all of us have had the experience of preparing a document in a text editor (or in a full-on desktop publishing application) on a personal computer, although we have varying degrees of experience with adjusting the word processing application’s default settings. Application developers make an effort to use default settings that work well for most people; thus, the layout functions of text editors can be fairly invisible to typical users. In fact, it’s not until we prepare a document that consists of more than running text—when we start fussing with tables, graphics, or embedded media—that we have to worry about these settings at all. Even then, we might be tempted to ignore the arcane terminology that appears to be a relic of the printing press (and the typewriter) and is today more germane to the work of graphic designers than it is to eBook readers and writers.10 The purpose of this chapter is to provide enough background so that the reader is conversant in all of the design elements that are discussed by studies of eBooks in use and in the more general studies of reading from the screen. This chapter will also provide us with a common vocabulary to refer to aspects of eBook layout; there is little sense in reinventing language that has evolved with the technologies of reading and writing. Legibility. Legibility (in this usage) is the extent to which running text presented on a display appears to be readable. Many different elements contribute to legibility. Figure 2.2 shows two contrasting eBook pages to give you a sense of what we mean when we use the word legibility: the same words may be on a page, but slight adjustments in the page design may give the reader a very different impression of its legibility. Use plays into legibility too. Our 2001 study of reading eBooks on a small form-factor device concluded that students were more likely to skim the material provided to them on the device; their reading was time-constrained (“I have 10 minutes before class”), interrupted and fragmented (“I read at the bus stop”), and nonlinear (i.e., students skipped from place to place in the text). In this situation, students are more likely to judge layout efficacy on how much they can see of the text
10
Document design is a complex topic; this book doesn’t have room to do it justice. The interested reader might refer to an authoritative design guide such as Shriver (1997) or to discussions of the use of graphics (Tufte 1990 and Bertin 1983 are classics in this area). Typography itself is practically a cult, and there are many worthwhile discussions of the art and science of type design.
24 reading and writing the electronic book
FIGURE 2.2: Two different layouts that illustrate legibility trade-offs. (a) Open layout that has been produced using legibility heuristics; the last line is selected for comparison purposes. (b) Tighter layout, optimizing the amount of content displayed on the page (for maximum context). Note the relative position of the selected line.
at once. That is, they’re willing to sacrifice reading comfort (e.g., ample leading, wide margins) for more context (literally, more words on the page). Genre also makes legibility a less straightforward matter. Certain kinds of material are very sensitive to layout; changing the layout changes the reader’s perception and understanding of the content. Poetry, particularly structured verse forms like rhyming couplets, provides us with com pelling examples of this phenomenon. Participants in the 2001 study cited above described verse with rhyming couplets where the last words of most (print) lines fell on new lines (on her Pocket PC display). When the rhyming words drew attention to themselves, the verse gave the reader an unanticipated sense that the poetry was doggerel. Re-reading the poetry properly laid out was necessary to correct this misperception and restore the verse’s intended meaning and sense. Figure 2.3 shows an example of this layout phenomenon. Thus, we can see that legibility is a complex interplay of factors. What kind of reading will the reader be doing? What constraints are introduced by the genre? Does the display impose any other constraints by virtue of its size or properties? Some type-rendering techniques
reading 25
FIGURE 2.3: Layout and genre: the perception of a poem is altered by its layout on the screen.
such as Cleartype take advantage of LCD striping; hence, if the display is not LCD-based, it is possible the technique will not work properly. Once we have teased out these mitigating factors, we can look at page layout in terms of more obvious physical elements such as type size, font family, font types, leading, spacing, and line lengths. There are other more subtle parts of type design—e.g., the letterforms (the actual shapes and characteristics of the letters) and counterforms (the enclosed spaces in letters like q, o, or g)— that are the concern of typographers; we will not worry about these design elements here. Type size. Unfortunately, type size is computed in less uniform terms than we might initially imagine. Most of us have encountered the standard type size metric of points. Points are used to measure type size both on the screen and in print in most English-speaking countries. There are approximately 72 points to an inch; in other words, we might expect a 12-point type to be 1/6 of an inch tall at its tallest point (i.e., from the bottom of its lowest descender, the part of the letter that extends beneath the line, e.g., the tail in a “y,” to the top of its highest ascender, the tallest part of the letter). But there are fascinating fudge factors built in to this measurement, due in part to its history and relationship to movable type (e.g., see the brief account given by Schriver). The same story that makes points such an interesting measure also makes them difficult to standardize. You have probably noticed that there is a great deal of size difference among comparable letters in different font families (which is why you often must redo document layout when you change fonts even if you change nothing else). These differences compound over the course of a book or lengthy document, and they play strongly into our discussion of legibility: some fonts may be easier to read on the screen than others are. Figure 2.4 illustrates the size difference between five uppercase Cs, all presented in 72-point type in Microsoft Word. Letter sizes were measured directly from the screen using a machinist’s
26 reading and writing the electronic book
(a)
(b)
(c)
(d)
(e)
Font
Times New Roman
Georgia
Arial Narrow
Lucida Sans Typewriter
Verdana
Size
72 points
72 points
72 points
72 points
72 points
Height
0.69 inch
0.72 inch
0.73 inch
0.71 inch
0.73 inch
Width
0.55 inch
0.55 inch
0.48 inch
0.49 inch
0.59 inch
C C C C C FIGURE 2.4: Comparing the measured width and height of a letter rendered on the screen in different fonts (in Microsoft Word). Letter spacing and kerning may also be affected by font choice.
caliper. The character sizes vary substantially in width (measured to be as much as 0.10 inch) and by a visually conspicuous amount in height (measured as 0.04 inch). Letter spacing and kerning. It is obvious from looking at most text displayed on the screen (or in print for that matter) that the letters are not simply placed an equal distance from one another; most fonts include embedded corrections that take into account the way the letters fit together into words. For example, an “o” next to an “x” (say, in “box”) looks very different from an “l” immediately adjacent to another “l” (as you would see in “llama”); because the outward bulge of the o coincides with the inward shape of the x, they can be placed closer together. Kerning is the term used to describe how the distance between specific letter pairs are adjusted for the sake of aesthetics and legibility. Kerning may be performed automatically, as part of the type design process, or both. Tracking or variable letter spacing also refers to text density that creates word shape. Beyond aesthetics or genre conventions, letter spacing that is either too tight or too loose can have an adverse effect on legibility.11 Not all fonts use such corrections, however. So-called proportional fonts—which include most of the fonts you see on the screen when you are reading—use kerning and variable letter spacing. Monospace or fixed-pitch fonts are used when it is necessary to predictably align letters; in a fixedpitch font, each letter occupies the same amount of horizontal space. For example, fixed pitch fonts are often used in software development tools, where indentation and alignment are used to reflect 11
Tight letter spacing can create an emotional reaction on par with listening to someone speak very fast, as in a salesman’s frantic high-pressure pitch.
reading 27
syntactic structures in the programming language. Courier and Lucida are examples of common monospace typefaces. Font hinting is another technique that is used for on-screen text rendering. Hints are created during the typographical design process and embedded in the font files so that different-sized fonts can be rendered clearly. In essence, hints specify which pixels are crucial definers of a letter’s shape. Font hints and their resulting shapes often can be calculated according to mathematical algorithms, but high-quality fonts demand designer intervention to make the text visually appealing and maximally legible. Families, fonts, and faces. Three decades of desktop publishing have had their effect on typographic terms, and many words that used to have distinct meanings are now used interchangeably. Instead of being a stickler, it seems better to be flexible and to sort out some of these things through common usage. Historically, a font meant a specific typeface (e.g., Arial) in a specific size (e.g., 12) rendered in a particular style (e.g., narrow). Today, font means something more general, e.g., Arial in any size or rendering style. A family is a set of fonts that use a particular typeface design; a font family includes all of the stylistic variations (narrow, thin, etc.), all of the size variations (8 point, 10 point, 11 point, 12 point, 14 point, etc.), and all of the rendering variations (bold, italic, etc.). Serif and Sans Serif. Although type designers have created an extensive palette of fonts that may be used to great effect in electronic book, there are two principal styles of typography: serif and sans serif. In other words, fonts either have serifs (think Times New Roman) or they don’t (think Helvetica). A serif is a short line that is used to decorate or emphasize the end of the basic strokes of a letterform. Serifs are felt to enhance legibility because they help the reader recognize the letterform and distinguish it from similar-looking letters; they also may provide the visual continuity that supports the perceptual grouping of letters into words. Thus, it is a common belief that a serif font is the most legible choice for rendering running text. Later in this chapter, we will investigate this question more fully: Does empirical data provide any reason, beyond aesthetic judgment, to use serif fonts for running text and sans serif text for headings? Are serif fonts really more legible than sans serif fonts? Practically speaking, sans serif fonts may strike the reader as cleaner and more modern. The bolded form of sans serif letterforms is easier to distinguish from the plain versions of the letterforms; hence, this makes them an appropriate choice for headings because it is easier for the reader to interpret heading hierarchies that use variation in type styles. Electronic publications often use serif fonts for the main body of the text and sans serif fonts for the headings, captions, and marginalia. Leading and line length. The word leading is derived from the strips of lead used in typesetting to form vertical separation between horizontal lines of type.12 Like font size, leading is measured in points and, for what we think of as single-spaced text, is typically between 1 and 6 points, 12
Hence leading is pronounced “ledding” not “leeding.”
28 reading and writing the electronic book
TABLE 2.2: Rough guidelines for leading by font size. Font Size
Points of Leading
9–11
1–3
12
2–4
14
3–6
16
4–6
18
5–6
depending on the type size; as you would guess, the bigger the font, the more generous the leading should be. Of course, text may be rendered with no leading between the lines without one line interfering with the next; we will discuss whether this affects legibility later in the chapter. Table 2.2, adapted from Fenton (1996), offers rough guidelines for setting leading for running text.13 Rendering techniques. Anti-aliasing is a computer graphics technique used to reduce the appearance of jaggedness when lines are drawn diagonally on the screen.14 Jagged lines are a natural side effect of drawing lines using a rectangular grid of pixels. In general, the higher resolution a computer display is (the more pixels per inch and the less distance between the individual pixels), the less this effect matters; however, anti-aliasing can compensate for the actual size and relative placement of pixels as they form diagonal and curved lines. Anti-aliasing does its magic by using subtle changes of color in the areas where the jagged edges are the most obvious. These slight changes in color cause a blending (almost a blurring) of the sharp edges to give the appearance of a straight line. This technique can be important for rendering fonts, because they generally involve many diagonal lines and curves, and the jaggedness can be quite distracting and may interfere with letterform recognition. Because this technique is at the pixel level, it is difficult to detect anti-aliasing unless the letterform is blown up. Figure 2.5 shows an example of an anti-aliasing technique used to smooth an “a” letterform in the Calibri typeface. The second generation of electronic book technologies saw the introduction of sub-pixel rendering techniques that build on the idea of anti-aliasing by using the properties of modern color 13 14
These guidelines are presented as just that; they are not intended to be hard-and-fast rules.
Anti-aliasing is not necessary for lines that run vertically or horizontally, since the pixels will line up with one another.
reading 29
FIGURE 2.5: Anti-aliasing technique applied to smooth the “a” letterform in the Calibri typeface. Calibri is a new face, designed specifically for on-screen reading. (a) An “a” letterform without anti-aliasing applied. (b) An “a” letterform with sub-pixel anti-aliasing.
displays, coupled with careful hinting of the fonts designed for on-screen reading. The best known of these techniques is Microsoft’s ClearType.15 Specifically, sub-pixel rendering techniques take advantage of the “stripes” in LCDs. That is, each pixel of an LCD consists of three narrow vertical stripes of color that the eye blends together; the pixel’s color is determined by the intensity with which each sub-pixel element is illuminated. By addressing each stripe of color separately—that is, lighting it only if the letterform actually passes through that sub-pixel, the screen resolution can be effectively tripled (at the expense of a minor perception of blurriness). Fonts designed for on-screen reading. Because the screen has different properties than the printed page—in particular, reduced resolution and different pixel shape—some recent typefaces have been designed to meet the needs of on-screen readers. Common examples of these typefaces include Verdana, Georgia, and Trebuchet; the first two were designed by Matthew Carter and the last by Vincent Connare. Typefaces designed for on-screen reading have slightly different properties than print typefaces. These characteristics include larger lowercase letters (so there is more open space in letters like “e”), more open letter spacing (so the individual letters don’t touch one another), and greater ubiquity (ensuring that most users will have them installed, thus making the designer’s job easier) (Quinn 2009).
2.3
STUDIES OF THE EFFECT OF LAYOUT ON READERS’ PERFORMANCE
The first studies of reading on the screen were motivated (whether explicitly or not) by the question of whether reading on the screen had a deleterious effect on reading performance when it was 15
Another color sub-pixel anti-aliasing technique is Adobe’s CoolType.
30 reading and writing the electronic book
compared to reading on paper. Although at the time of the initial studies, it was readily apparent that people did not like to read longer documents on the screen, and would instead print them out if given the choice, it was less clear what it would take to convince them to read on the screen. Would reading on the screen always be intrinsically inferior to reading on paper? Would people read less carefully and, consequently, with poorer comprehension when they shifted from print to display? Would they read more slowly and experience greater eyestrain and fatigue? Many of the earliest studies of reading on the screen were performed as a reaction to futurists’ claims that print forms would one day disappear. Certainly, given the state of display technology at the time of the studies, the claims were, at best, premature. In 1992, Dillon prefaced his comprehensive survey of studies of reading on paper versus reading on the screen by saying: Even so, paper is an information carrier par excellence and possesses an intimacy of interaction that can never be obtained in a medium that by definition imposes a microchip interface between the reader and the text. Furthermore, the millions of books that exist now will not all find their way into electronic form, thus ensuring the existence of paper documentation for many years yet (Dillon 1992). Needless to say, this prefatory remark—one that seemed like motherhood at the time—may now be subject to greater scrutiny. Millions of books are finding their way into electronic form and we are achieving a far greater intimacy of interaction with the screen, although we are still some distance away from where we need to be to make viable predictions about the end of books.16 It is still wise to wonder what kind of effects reading on the screen will have on performance as well as on reader satisfaction. The first conclusion of Dillon’s review of reading research to date (as of the early 1990s) was that it is dangerous to draw definitive conclusions from these studies because it is so difficult to control the numerous factors that affect both the outcomes of reading (using the familiar performance metrics of speed, accuracy, fatigue, comprehension, and preference) and the actual process of reading (eye movements, interaction with the medium, and navigation, which includes elements like link following, a type of navigation that was still fairly novel at the time the review was written, and page turning). In other words, the two delivery vehicles—the screen and the printed page— were so radically different that it was difficult for psychologists to compare them without stacking the deck right from the start.
16
As Robert Coover did in his so-named 1992 article that appeared in The New York Review of Books.
reading 31
Indeed, some of the experimental conditions ascribed to past research call ominous attention to themselves: in the on-screen condition, subjects read white text on a blue background or green on a black background; subjects reading from CRTs were hampered by possible glare from overhead lights; or subjects read on-screen text that was unnaturally large. It is easy to see why Dillon was reluctant to find these results conclusive, although he cautiously allowed that “reading speeds are reduced on typical VDUs17 and accuracy may be lessened for cognitively demanding tasks.” That readers preferred high-quality print documents over the display technology of the day—both for their legibility and for the interactions they afford—hardly seems surprising from the vantage point of 1992 when the article first appeared. The key to the early findings is that they are tied to the display technology of the day. Thus, the comparisons will continue to be meaningless until the quality of electronic text presentation reaches a paper-like level. At that point, we can revisit the legacy results evaluating the effect of image quality (i.e., typography and layout as they are realized on paper, e.g., Tinker 1963). It is also critical to consider the performance metrics: are speed and comprehension how we want to measure reading effectiveness? Finally, we must question the typical tasks for these performance measures: Is proofreading really a typical reading task? Is it the best gauge of accuracy? Dillon also points out that once the document in question becomes more than a single page/ single screen in length, other factors beyond image quality come into play. Indeed, the scrolling versus paging question is still a fundamental controversy that drives eBook design; we discuss it further from an empirical/design perspective in Chapter 3.2. Page turning, its advocates argue, allows the reader to continue to take advantage of the properties of the print book such as its intrinsic support of spatial memory. But scrolling has proponents too; scrolling advocates cite continuity and navigation speed as major advantages. In fact, studies as recent as Liesaputra and Witten (2008) continue to find no definitive resolution to the scrolling versus page turning controversy; the answer is contingent on so many different factors that it will continue to be difficult to resolve the question “once and for all.” One hypothesis that has been floated among members of the eBook community is that a weakness of digital publications is layout and that improving layout would greatly enhance peoples’ satisfaction with reading on the screen and might even improve their performance using conventional metrics. Indeed, it was postulated that one of the reasons that people prefer to read print books over eBooks hinges on print books’ superior layout. What would happen if we applied the known layout heuristics to on-screen displays of text and graphics? Chapparo et al. (2005) compared subjects’ reading performance for online documents prepared under four conditions: with adequate margins, without adequate margins, with optimal (open) 17
Video Display Units.
32 reading and writing the electronic book
leading, and without optimal (open) leading. In this study, the optimal use of white space affected both reading speed and text comprehension: subjects read the text with margins more slowly but comprehended it better; subjects also preferred the text that was prepared so that it had adequate margins. We might surmise that if a document is more pleasant to read, subjects will spend longer reading it and will score better on measures of comprehension. Optimal leading, on the other hand, had no effect on performance, but the subjects preferred it over text prepared with suboptimal (i.e., no) leading. A second unpublished study by Chaparro’s group18 demonstrated that applying on-screen layout heuristics to produce a better looking page (using typographically optimal headers, paragraph indentation, proper figure placement, visually appropriate quoting, etc.) does not translate into comparable improvements in reading performance. In an experiment that compared the two conditions (good and poor layout), no difference in reading speed or comprehension was found. Readers preferred the nice layout, but it did not change their ability to take in what was on the page. The application of good type design principles had even less effect: subjects did not notice them to appreciate them. But can rendering itself improve performance? In other words, if the letterforms are easier to recognize, can this translate into overall improvements in reading speed and comprehension? Gugerty et al. (2004) found that subjects’ word recognition improved slightly when the words were rendered with the Cleartype sub-pixel anti-aliasing technique. This improvement translated into small performance improvements in reading speed and comprehension for a sentence-level reading task; the improvements did not carry over into long-duration pleasure reading, however. Sheedy et al. (2005) performed another experiment on legibility—the accurate recognition of letterforms and words—to tease out the factors that make text legible on displays. Pixel height (which corresponds to font size), font type, and stroke width all influenced legibility, with optimal legibility attained at a 10-point font size. Four fonts were evaluated in this study: Times New Roman and Franklin were the least legible fonts and Verdana and Arial were the most legible. A recent study by Larson and Picard (2005) attempted to quantify the effect of the aesthetic differences that cause readers to prefer one page design over another. It has been found that mood influences task performance: a good mood will cause people to perform better on creative tasks than they would if they were in a bad mood. So in this study, armed with the knowledge that better rendering improves performance and that subjects prefer to read on-screen documents that look good, Larson and Picard showed that high-quality typography can put subjects in a good mood, and they demonstrated that you can feasibly ask “how good?” using established methods for measuring affect. 18
An account of this study may be found in the work of Larson and Picard (2005).
reading 33
In the end, when we contemplate the results of these studies, we find that lab subjects prefer nicely laid-out text, although the legibility enhancements provided by rendering techniques and layout do not result in corresponding increases in reading performance. In the field, what we see is more complex. There are trade- offs introduced by good layout: fewer words fit on the screen; hence, the reader may need to spend more time navigating and may put forth more effort retaining context (because there isn’t as much of the text available to quickly refer back to or to glance forward at). If we return to Figures 2.2 and 2.3, it is easy to get an intuitive sense for some of these trade-offs. Nothing is simple, but as the page on the screen becomes more comparable to the print page—and as the materiality of the eBook becomes more comparable to that of the print book—the objections to reading on the screen begin to disappear.
2.4
READING HARDWARE AND DISPLAY TECHNOLOGIES
The word eBook can refer to hardware, software, content prepared to be read on the screen, or a combination of all three. In much of this book, when we talk about eBooks, we’re by and large referring to the software—the reader—used to present the content. However, in this section, we’ll take a brief look at the actual hardware platform, the devices on which we read. While readers might choose to read on any computer—from a desktop with a large stationary display, to a laptop, to a smart phone—we will look at the elements of purpose-built hardware especially designed for reading. Let’s start with the distinction between active reading and immersive reading that we made earlier in the chapter. In reality, this distinction is artificial. Readers can change midstream from reading immersively to interacting with their reading or to skimming and scanning. However, these distinctions give us a place to start and a way to organize the discussion. Reading the newspaper on a screen suggests different hardware requirements than reading technical articles as you prepare to write your doctoral dissertation; likewise, reading office documents raises a different set of concerns than reading the latest best seller and making it part of your personal library. Thus, instead of discussing the existing electronic book products or research prototypes, we will discuss various elements of a capable hardware platform, no matter what the emphasis. If we revisit Figure 2.1, it gives us a structure for thinking about reading platforms and for eliciting requirements to specify different hardware components. Figure 2.6 shows this matrix. Immersive reading suggests a focus on legibility and ultimately on portability, since this might be the kind of reading material you would carry with you to a variety of places. Because we are suggesting this might be purpose-built hardware, it will have to be very light. Immersive reading—and to some extent, active reading as well—suggests that the user interface should disappear, and the modes of user interaction should be very straightforward and not require instructions to learn; after
Active reading (purpose in mind) Requirements: interaction, manipulation
34 reading and writing the electronic book
Laptop, notebook, or other mobile platform that is capable of running a range of standard applications
Tablet or device that supports direct interaction with material and is fully functional
Mobile reading platform with large display (for skimming/scanning)
Mobile reading platform, emphasis on storage and portability
Immersive reading (focused attention) Requirements: transparency, legibility
FIGURE 2.6: Hardware platform requirements as driven by different reading needs.
all, no one needs instructions on how to read a book, assuming they are literate. Active reading suggests a focus on interactivity; hence, we should be thinking about different modes of user input. Depending on the type of interaction we want to emphasize—say, annotation as opposed to rapid page flipping—we might come up with a variety of different mechanisms that support the style of interaction we have in mind. As we begin to think about the overarching activities associated with reading, storage enters the picture. Of course, we’ll always have to store the content somewhere, but perhaps it might be wise to get beyond the transience of the newspaper or novel-based immersive reading experience that dominates the lower quadrants of the matrix. Although active reading suggests that we’ll either need to rely on cloud-based applications that run in a browser or we’ll need to specify a general-purpose computer platform, the common element is high-bandwidth connectivity. Unlike the second generation of eBook devices that we discussed in the introductory chapter, we will need to think about higher bandwidth connectivity and—more than likely—synchronization among copies of our personal libraries as we use them on different devices.19 Finally, one significant overarching concern of mobile technologies enters the picture: battery life. People find little more 19
Products such as Microsoft’s LiveMesh address the problem of synchronizing files among personal devices. More complex replication problems are the focus of current research (Terry 2008).
reading 35
frustrating than being forced to stop reading because they need to plug in their eBooks; long-lived batteries, coupled with efficient use and management of available power, is essential to successful eBook platforms. One argument for the use of the bistable displays is that they consume less power than LCD displays of comparable resolution. Reading relies crucially on-screen legibility; at the same time, the mobility we associate with reading relies on reducing the weight of a conventional computing platform. While people can read on less-than-optimal displays without a significant performance sacrifice, they don’t like it (refer to the discussion earlier in the chapter). The third generation of eBooks aptly illustrates this: these new eBook products rely on the bistable display technology that has matured recently. It is well suited to text display, is light, and uses relatively little power (partly because once an image has been rendered, it is stable). Hence, we will focus on bistable displays, rather than the ubiquitous LCD displays. A bistable display is one that remains in a stable state without requiring power; this stability is in contrast to an LCD display, which requires continuous power to display an image. The most common type of bistable display is known variously as ePaper or E Ink (both of which refer to specific products). Electronic paper consists of tiny electrostatically charged spheres or charged particles suspended (in oil) in a planar plastic matrix; electrodes surround the somewhat flexible plane. Because of the way the display image is produced, the screen is reflective like paper, rather than backlit like a traditional computer screen. These properties—image stability, reflectivity, and the consequent wider viewing angle—make electronic paper more comfortable to read in most situa tions than other types of displays.20 An early version of this technology used charged spheres that were black on one side and white on the other. Electric power was applied to change the state of the individual balls in such a way that they rotated to form an image (Sheridon et al. 1997). In newer versions of the technology, electrophoretic displays, the images are created using charged pigment particles rather than the discrete bicolored balls. In other words, the individual pixels can be thought of as positively charged on one side and negatively charged on the other, but many particles form each pixel; to produce the bicolored effect, the particles all migrate to one side of the ball or the other. Figure 2.7 shows schematics of the early electronic paper technology, Gyricon (Sheridon et al. 1997), and the comparable electrophoretic display technology used by the third-generation eBook hardware like Amazon’s Kindle and the Sony Reader. Although its weight and reduced power consumption make electronic paper a good display technology to support immersive reading, ePaper is somewhat more problematic when it comes to the requirements introduced by active reading. It has a low refresh rate, so it is more difficult to implement the facilities necessary for skimming, for scrolling rapidly, or for flipping through the 20
It should be noted that bistable displays cannot be read in the dark because they are not backlit.
36 reading and writing the electronic book
FIGURE 2.7: Producing pixels using electronic paper display technologies. (a) Gyricon display. (b) Electrophoretic display.
pages. Currently, the technology is still limited to black-and-white images, although various experiments to extend the technique to color displays are underway. Predictions variously put low-power color displays on the scene as early as the end of the year and as late as the end of the next decade. Recent research has examined the feasibility of two-screen readers, which will give the reader a more natural view onto multiple pages. Chen et al. (2008) at the University of Maryland and the University of California have been building a dual-display prototype that supports interactions like folding, flipping, and fanning. The displays also detach from one another to provide more paper-like modes of interaction; if they are attached, the two displays are like two consecutive pages of a document, and if they are detached, they become windows onto pages of separate documents. We will discuss the interactive aspects of this prototype in the next section; the point here is that the price, weight, and power consumption of electronic paper has finally made a dual-display reader feasible. Past attempts have shown it is logistically possible, but not particularly practical (e.g., see the description of the Everybook in an article by Schilit et al. 1999).21 • • • •
21
It should be noted that the prototype gets around the display refresh issue we discussed earlier in the section by using screen real estate to provide a reduced representation of the document’s pages that is used for visual navigation. See Space Filling Thumbnails (Cockburn et al. 2006) for a detailed description of such as technique.
37
chapter 3
Interaction EBook products to date have focused on particular types of reading—reading for entertainment, as one would read a novel, or reading to become more informed, as one would read a business book— and have more or less ignored others (e.g., reading to learn, as one would read a textbook, or reading for reference, as one would consult an encyclopedia). These assumptions about types of reading have dictated general eBook functionality and the specific facilities for interacting with eBooks. For example, the platforms assume that navigation will primarily consist of page turning: the reader will approach a book in a linear fashion, reading straight from beginning to end. Similarly, much of the interaction is confined to simply marking one’s place in the text with a bookmark or highlighting a few choice selections in the text through a clunky menu-based option. Yet some types of reading demand more sophisticated functionality. Electronic textbooks have been less successful than electronic novels partially for this reason. Students need to be able to annotate, work problems in the margins, follow along in class, and take notes. Often, this kind of reading is referred to as active reading, and various pedants have offered prescriptive approaches to how readers should approach this activity. While active reading has been at the center of eBook research projects such as XLibris (Schilit, Golovchinsky, & Price 1998), it has yet to find its way into eBook products. Likewise, successful electronic reference books have taken advantage of the power of the computer to provide their users with the ability to, for example, translate words from an unfamiliar language or to navigate through the content in ways that are well aligned with the books’ expected use (Marchionini 1995; Egan et al. 1989). They may also include enhanced multimedia presentations and the ability to work across different computers. New modes of interaction and added functionality are not yet commonplace in eBook products, and when they are available, they seem like an afterthought rather than a fundamental part of the product design.
The complex economics of textbooks are another important factor in the marketplace response to electronic textbooks.
38 reading and writing the electronic book
In this chapter, we will examine some important types of functionality that are tightly interwoven with the act of reading, such as annotation, navigation, linking, clipping, and bookmarking, and the hardware needed to support this functionality. EBook-based collaboration is also a source of advanced functionality; Chapter 4 will extend the functionality covered in Chapter 3 into the social realm and will address aspects of reading and interaction such as sharing annotations, clippings, and entire books. Although fluid interaction with eBooks is necessary to give them the so-called affordances of paper, it is important to remember that reading may be part of a more complicated system of activities. In the final chapter of this book, we’ll build on Chapters 3 and 4 to examine additional functionality that supports the move from reading to writing. This type of functionality often involves working across electronic texts to gather, triage, and analyze different information sources. Taken together, all of these different mechanisms, functions, and related applications will give eBooks the ability to take on the challenges of active reading: reading to inform and instruct.
3.1
ANNOTATION
Annotation is a basic and often unselfconscious way in which readers interact with texts. A student struggling with a difficult philosophy text might trace his progress through the dense prose by highlighting line after line. An amateur cook might cross out an unused ingredient in a favorite recipe to reflect how she usually makes the dish. An engineer might pencil in the value of a constant in the margin next to a formula or might draw a rough graph to better visualize an equation. Each of these examples shows annotation used in a different way: to focus the student’s attention; to augment the cook’s memory; to actively engage with an engineering concept. Just as the uses vary, so does the long-term value of the annotation: the student studying for final exams might want his highlighting to disappear when he re-reads his textbook; the cook might want her annotation to become an indelible enhancement to her cookbook; and the engineer might want the notations to stay in place for the duration of a project, but not when she lends the book to a colleague. These examples show different kinds of people annotating paper texts for a variety of reasons. That’s because annotation on paper is a seamless, flexible, and well-developed practice (if sometimes taboo). Annotations on electronic texts have generally been more problematic. On some reading platforms, annotation is clunky, interrupting reading as the reader pulls up menus, makes selec
In this chapter, I take annotation to mean the kind of informal markings readers make on a page while they are reading rather than using the word to refer to published (and unpublished) scholarly annotations as one would find in a critical edition of an important literary work.
Interaction 39
tions, switches to a keyboard to type in text: the reader’s attention is refocused on the user interface rather than on the book’s content. Electronic annotation tools may also limit a reader’s expressive intent (e.g., forcing a highlight to be continuous when the reader wants to fragment it, or imposing neatness on a reader when she wants to scrawl). Sometimes the electronic annotations are stored in infelicitous ways so they are either gone when the reader returns to the eBook on a different computer or so they are recoverable when the reader believes them to be deleted and loans the book or document to a colleague. In fact, some early objections to electronic books are based on their immutability, because reading might not change them the way it changes print books. The French post-structuralist cultural theorist Baudrillard put it this way: The compact disc. It doesn’t wear out, even if you use it. Terrifying. It’s as though you’d never used it. So it’s as though you didn’t exist. If things don’t get old anymore, then that’s because it’s you who are dead (Baudrillard 1996, pp. 32–33). Indeed, the digital ideal is a book that doesn’t show wear or become shabby and dog-eared with use; functionality must be added so that the eBook’s pages reflect wear. On paper, annotation is one crucial record of a reader’s interaction, along with other telltale signs of previous engagement; there are compelling reasons to duplicate this effect in electronic books. If they had paid attention to the medium’s history, critics wouldn’t have harbored these fears about the sterility and immutability of electronic books; even in the earliest electronic books and hypertext systems, annotation was seen as a key form of interaction. In an influential keynote that Brown University professor Andreas van Dam delivered to the ACM Hypertext conference in 1987, he said: The reason I encouraged such annotations [in FRESS] was that I remembered that when I was in college with Ted [Nelson], I would always grab the dirtiest copy of a book from the library, rather than the cleanest one, because the dirtiest ones had the most marginalia, which I found helpful (van Dam 1988).
There have been legal cases based on the ability of an adversary to recover a document’s deleted annotations. Needless to say, this vulnerability has been the cause for some concern to developers implementing electronic annotations.
For example, the read wear user interface (Hill et al. 1992) uses various intuitive visualizations to show the reader how often he or she has accessed a section or passage.
40 reading and writing the electronic book
Indeed, there has long been considerable optimism about the power of annotations and the reader’s role in the intellectual ecology of the digital age. In this section, we’ll look at several different aspects of annotating electronic books with an eye toward how annotations are represented and stored and how they may form the basis of advanced functionality. In recent years, different communities have shown a great deal of research interest in annotations. Researchers have generally agreed that annotations can be a valuable artifact that reflects a reader’s engagement with and understanding of a text, an artifact that may persist beyond the immediate reading. There has been general recognition that reader annotations should not be tied to the particular eBook software that was used to produce them. Hence, it is important to develop uniform terminology for annotations and to represent them consistently.
3.1.1 Representing Annotations Readers don’t always annotate. If you browse novels in a used bookstore, you would be surprised to find many annotations on the books’ pages. Usually wear is shown in more subtle ways: damage to the book’s spine, old bookmarks, or dog-ears (folded-down page corners). On the other hand, a novel used as a textbook might have notes in its margins. In fact, these notes might be a strong indicator that the novel was used as a textbook. This example is a harbinger of a more specific claim about readers’ markings in their books: Readers annotate in a way that depends crucially not only on the book’s genre but also on their reason for reading it (Figure 3.1). Annotations are thus as diverse as readers’ motivations for reading. Let’s look at some annotations in print books (in this case, in textbooks since being a student offers ample reasons for marking in books). Figure 3.1a shows a link connecting two separate fragments of text; Figure 3.1b shows a page with extensive highlighting; Figure 3.1c shows asterisks in the margin (we might assume that the reader thought this passage was important); Figure 3.1d shows a longish bit of marginalia that refers to a sentence that the reader has underlined; and Figure 3.1e shows evidence of a coding system that a reader has developed to categorize his annotations. What is important here is the diversity of the marks. The annotators have found ways of writing on the print page that do not obscure the original text; they have managed to connect their notes with the words the notes refer to; and they have used personalized symbols if they thought it appropriate. In some less common cases, they have even found imaginative ways to code their markings. Let’s look further at the characteristics of these marks: what makes an annotation tick? Some of them make an explicit comment on the text. For example, Figure 3.1d shows a student’s interpretive note, “Eros is the force that binds things together and keeps the structure of things intact.”
Interaction 41
FIGURE 3.1: Examples of five kinds of annotations that demonstrate a broad range of reader intentions. (a) Linking. (b) Highlighting. (c) Margin marks. (d) Extended note anchored in the source text. (e) Coding system.
We don’t know whether the student wrote that note in class, echoing the professor’s lecture, or whether the note represents the student’s own reflection on the material. But that doesn’t matter for what we are trying to do here. What we do know is that there is an explicit comment (with an apparent meaning) and it pertains to a well-delimited region of the source text. We can also find annotations that are less explicit. Figure 3.1b shows a highlighted region. We don’t know what the student has to say about this text, only that the student took particular note of this passage while he or she was reading. The passage might have been important, or it might have been just difficult to understand. In other words, the body of the annotation is implicit or telegraphic; we have to assume that the meaning of the highlight will be apparent to the reader when the marks are re-encountered.
42 reading and writing the electronic book
TABLE 3.1: A space of annotation forms. Characteristics
Explicit Anchor
Implicit Anchor
Explicit body
Example: An interlinear translation of an underlined word.
Example: A note in the margin without any corre sponding marks in the text.
Implicit body
Example: An underlined sentence.
Example: An asterisk in the margin that does not explicitly specify extent.
It is easy to imagine an example that is the other way around: the reader has written a longish note, but we can only guess what portion of the text that the note refers to. In other words, the anchor of the annotation is implicit. Finally, there are marks like the ones shown in Figure 3.1c that are vague in both how much text they refer to and why they have been added in the first place. Table 3.1 summarizes these variations and provides an example of each. Table 3.1 does more than lay out a space of annotation forms: it also suggests an anatomy for annotations and a universal way to represent them. Work on integrating annotations into digital library systems (and other user interfaces that incorporate annotation functionality) has thus found it convenient to adopt a consistent terminology for parts of an annotation (e.g., see Agosti et al. 2004). A consistent terminology and representation of annotations makes it possible to develop interoperable systems and mechanisms for storing, reading, and rendering annotations in electronic books and digital libraries.
3.1.2 Anatomy of an Annotation From our table and the previous discussion, we can see that there are at least three elements to consider when we look at annotations: a body, an anchor, and a marker. The body of the annotation is any content that a reader has added to the work (e.g., a note in the margin or an asterisk). Many of the annotations that readers add to books are highlights or underlines with no explicit body. Any representation of an annotation must handle this common case of a null-content annotation. The second element of an annotation is an anchor, which delimits the scope of an annotation, even if the scope is implicit. Anchors are the heart of an annotation, because they tell us how any added content is related to the source material, the portion of the eBook that the annotation actually refers to. Anchors may not only be implicit; they may also be broad in scope—a review or
Interaction 43
rating might refer to the entire document rather than to a specific passage. Similarly, an annotation’s scope may be narrow: an interlinear translation may refer to a particular word and proofreading annotations may even be anchored to one or two letters (e.g., to indicate where a letter should be inserted). The third part of an annotation is a marker ; a marker tells us how the anchor should be rendered when it is displayed. In other words, we don’t just need to know what the annotation refers to in the source material; we also need to know how to display it. Let’s disentangle these abstractions using an example (Figure 3.1d) to see what these terms mean in practice. It looks like there are three distinct annotations on this paper page. The first one’s body is the marginalia “Eros . . . keeps the structure of things intact,” a comment that appears in a particular position in the margin. Its anchor (the portion of the anchor that is visible) spans two lines of source text, starting with the word “it” and ending with the word “the.” Finally, its marker tells us the anchor should be displayed as a black underline. The second annotation, the yellow highlight starting at “something” and ending at “change,” has a null body (i.e., there is no added content); its anchor spans the entire paragraph of source, and the marker is a yellow highlight. Finally, there’s a third annotation, an asterisk at a specified location, which forms the annotation’s body. It has no explicit anchor, but depending on what we’re going to do with the annotations later on, we might want to assume it is anchored to the same full paragraph as the second one. The null marker would tell us that the anchor is not displayed. In most implementations, annotations also have an author (i.e., it is important to record who created the annotation) and a time stamp that records when the annotation was created. The anatomy of an annotation may be arbitrarily complex, but these basic elements capture essential qualities of many different types of annotations created in a broad range of circumstances. While none of these interpretations of our on-paper examples are absolute, it is important to have a set of terms that will enable us to discuss annotations in a relatively consistent way, independent of how they are realized in software implementations on various platforms; a set of common terms makes it reasonably easy to compare different ways of anchoring annotations and displaying their markers. How common are the different types of annotations? The answer obviously depends on the material (its genre, level of complexity, and its physical form) coupled with the reader’s purpose (no matter how inviting a page is, a reader who is relaxing with a novel on the beach will make fewer marks—probably no marks beyond a drip of ice cream—than the same reader reading the same novel for a graduate course on Twentieth Century American Literature) and the particular
Specifying the position of an annotation’s body is tricky and highly variable; this portion of the representation is left as an exercise to the creative developer.
44 reading and writing the electronic book
circumstances (with any luck, a library book will have fewer annotations than a used paperback). But the data we collected for the annotations study reported by Marshall and Brush (2004) gives us a representative idea of the relative frequencies of different types of annotations. Table 3.2 summarizes the data. These data confirm qualitative observations I have made in other studies: by far, the most common kinds of annotations are anchor-only annotations such as underlines and highlights. Specifying anchors is a difficult and important aspect of handling annotations programmatically. Anchor specification relies crucially on the type of content being annotated and the ultimate functionality that a developer expects to offer. Is the content changing, as it would be in text editor? Is content moving in the electronic document relative to the page, but staying relatively static, as would be the case in a blog where new entries are being added, pushing the older content further down the page? Are we implementing annotation functionality for published eBooks? Will the page layout change (say, when larger fonts are used), or are page images fixed, as they would be in a PDF format document? Are there unique identifiers for book elements (such as chapters or paragraphs)? It is easy to see that specifying anchor positions will take some creativity and insight into the type of content and applications we intend to support. First, let’s look at most straightforward situation, a published electronic book stored as a PDF document. Because we’re talking about a published book, we know that the underlying content is not apt to change and the page layout has been pre-calculated when the PDF was produced. This stability allows us to anchor annotations geometrically, as an overlay to the electronic equivalent of a printed page. Depending on what we’re planning to do with the annotations, we might not even
TABLE 3.2: Typical relative frequencies of annotation types (data from Marshall & Brush 2004). Annotation Type
Total Number of Annotations
Relative Frequency (%)
1, 276
83.1
Body only (including marginalia and symbols)
105
6.8
Compound (anchor+body)
138
9.0
16
1.1
Anchor-only (including underlines, highlights, and combinations)
Other (e.g., doodles) Total
1,535
100
Interaction 45
need to know what the content is that we’re anchoring to, just where it is in the document. Thus, in our example, we might store the anchor as an x, y position on page z. But perhaps there are accessibility concerns or we’re rendering the eBook on a mobile device. We might want to increase the size of the text to accommodate visual impairment. Or we might want to reduce the text size so more can fit on a small screen. In this case, we might change the anchor specification to consist of a starting point (a particular character in the stream, beginning at a uniquely identified book element) and an extent or span (the number of characters beyond the initial position that the annotation refers to). For example, we might specify that an annotation is anchored in an eBook’s Introduction (where the Introduction is a structural element of the eBook file), starts 234 characters into the chapter, and extends 47 characters beyond that. Our two examples thus far are simple representations for the annotation’s anchor and may be sufficiently robust for our needs. But wait! We might want to specify annotations in such a way that the underlying text can change (as it would in a document under review and revision by a standards body), and the annotations on the text will still stay in sensible places. In this case, we may want to pick certain keywords from the underlying text and use them to represent the anchor (Brush et al. 2001). Thus, for our previous example (see Figure 2.1d), this might mean that the first annotation is anchored to the sentence containing absolute, beautiful, and things. Yet another method of specifying an anchor allows us to handle annotations on text that moves but does not change (say a blog entry that gradually moves down the page as new entries are added). In this case, we’ll want to uniquely identify text in a way that is position-independent, e.g., as a fingerprint, or some other method that allows us to calculate and maintain a compact representation of the underlying content so that that annotation can be anchored to specific content, rather than a specific document (Hong et al. 2008). For the sake of brevity, there are further complexities that we will not pursue here. For example, spatially proximate annotations may be combined to form a single unit, a compound annotation. In other cases, we might want to split an annotation into two or more parts because we want to handle each part differently. We may also want to handle standard metadata as a special type of annotation that refers to the entire document. It is easy to see why the representation of annotations is an important problem for us to tackle if we are going to develop interactive electronic books.
3.1.3 Linking What is a link? We can approach this question philosophically to reach a powerful and very general formal definition—in fact, more than 15 years ago, the hypertext community did just that (Halasz & Schwarz 1994)—but we will approach links in this section in a more operational way. From an eBook perspective, links connect nonadjacent spans of text, possibly in different books.
46 reading and writing the electronic book
Links actually have two functions. One is to specify a relationship: “Paragraph 13 is related to Paragraph 25 even though they are not adjacent.” Another is to support nonlinear navigation: “By clicking on Paragraph 13’s anchor (via its marker), you will traverse to Paragraph 25.” We will focus primarily on the first (representational) function in this section, since we are in the middle of discussing annotations, but the reader should keep the second function in mind when we go on to talk about navigation. If you are familiar with HTML, it should be clear that annotations share terminology with hypertext links. In HTML, a set of
tags specify a link’s anchor. Anchor tags enclose a span of text (or another object) that acts as the source end of a link. In fact, we can think of an annotation as being a particular case of a link, one in which the link destination is either a new entity (when there is a note or marginalia) or null (when the anchor and marker combine to create a simple highlight, underline, or some other anchor-only form). Most links in electronic books are span-to-page links, because that’s how they’ve been implemented. That is, link traversal takes you to the page that’s on the other end of the link; no span is usually indicated on the destination end of the link. From a representational point of view, it is easy to see how eBook links could just as well be span-to-span links; the user interface and underlying data structures would only need to provide the reader or author with the opportunity to specify this span when the link is created. Alas, this is an awkward dialog and it is difficult to indicate the destination span either during link inspection, traversal, or visualization. Thus, many developers have felt that it is simply better to specify links as span-to-page. This representation is entirely consistent with our notion of annotations. Should readers be able to create links beyond the implicit links they are creating when they annotate? Most eBook products have answered “no” to this question; they may feel that it is too complicated or confusing for readers and that it is unnecessary to offer such a capability. However, in the original pre-Web vision of hypertext, link-making was felt to be an important advance offered by electronic books, because it helped readers become writers. As it stands, all links are defined during writing and publication, often as part of the markup process. It remains to be seen whether reader linking will ever become an extension of annotations.
3.1.4 Functions of Annotation The function of annotations is tied very closely to the underlying motivation for reading in the first place. A high school student who writes “man v. nature” on the first page of Moby Dick because
Web links may be “weak” span-to-span links, since HTML allows the author to designate anchors that may act as link destinations by using the NAME property.
Interaction 47
his English teacher has explained the work’s motifs in class is doing something very different than an editor who puts an exclamation point in a manuscript’s margin to remind her to talk to the author about something she’d like changed before the work is published. A copy editor’s proofreading carat stands in further contrast to either of these annotation examples. In the end, it is clear that annotations fill many roles. Table 3.3 shows some examples that contrast an annotation’s form with its function. Why do annotations’ varied functions matter when we’re discussing electronic books? Isn’t it enough to say that the ability to annotate is a welcome capability for eBooks, regardless of their genre or the reader’s purpose? Even if lawyers make different kinds of markings than students (or even law students) and use these markings in different ways, doesn’t it just underscore the importance of a generalized annotation feature for any reading platform? The simple answer to these questions is that understanding the function of annotations may be enormously consequential when the time comes to design any advanced eBook capabilities. The annotations’ intended uses have far-reaching implications for design, from storage to data structures to advanced functionality to user interface. Let’s look at a simple example: how should we represent and store the annotations that a reader has added to an eBook? Our understanding of the annotations’ function assumes a crucial role in how we formulate an answer to this question. Let’s say we’ve been looking at a group of users who go wild with highlighting pens—“happy highlighters,” one of my colleagues has called them—and the pages of their books look something like the page shown in Figure 3.1b. Conversations with these users reveal that the only function of these highlights is immediate: the readers trace along the difficult text as they read to focus their attention. These annotations don’t have significant long-term value. Nonetheless, we’d want their creation to be seamless (i.e., we don’t want to interrupt the reader). But we’d also want the marks to be easily removed; the only reason they should stay with the book is if the reader picked up the book on another device to continue reading (or perhaps the reader wants a way of knowing that she’d read the book at some time in the past). In this case, perhaps we’d just store the annotations in the same file with the book and give the reader the ability to quickly remove them. They might be anchored and marked in a similarly straightforward way, say as the triple (offset, span length, marker type), where offset specifies where the highlight starts, span length specifies how many characters beyond the offset have been highlighted, and marker type specifies which pen the reader has used, say the yellow highlighter.
This example is perhaps over-simple, since it glosses over many of the nuances involved in properly designing storage; take it with a grain of salt.
48 reading and writing the electronic book
Table 3.3: Different functions of annotations. Form Example
Example in Field Materials
Apparent Function
Underlined structural elements (e.g., headings), asterisks in the margins
Signal for future attention
Highlighted phrases, circled words, other several-word anchors
Aid memory
Notation in margins (near figures or equations)
Help work a problem
Marginalia, longer notes on frontispiece or above chapter head
Interpretation
Extended highlighting or underlining
Focus attention through difficult narrative
Notes, doodles, drawings, other markings unrelated to the substance of the text
Reflection of the material circumstance of reading
Interaction 49
Now, let’s say we’re not talking about a published book, but rather about a standards document that’s under evaluation by an international standards body. Everyone on the standards committee has his or her own copy of the current version of the document, but the annotations that they make on these documents are public, are created on an ongoing basis, and need to be readily shared. A robust anchoring scheme would enable the annotations to survive across versions; storing the annotations in a database on a centralized server would allow them to be shared. This strategy is very similar to the one described in Cadiz, Gupta, and Grudin (2000). Such a design enables the eBook software to overlay the current set of annotations on the document when the committee member opens it; there may even be a polling scheme by which the software looks for new annotations at set intervals. Thus, in our second case, the annotations are not stored with the document; rather they are stored in a centralized database. Furthermore, they are not represented as a span and an offset, but instead in a more robust, content-based representation that allows them to survive minor changes in the text. It’s easy to come up with new annotation scenarios, each of which suggests slightly different requirements; the main message is simply that the annotations’ function matters, both in terms of how we design the immediate feature and in terms of the enhanced capabilities we add to the eBook system later on.
3.1.5 Status and Value of Annotations Just how important are annotations? If we reflect on reading in its many quotidian day-to-day manifestations—scanning the newspaper while we’re riding the bus, reading a novel on the beach, or skimming a research paper quickly for the sake of citing it appropriately—we quickly realize that the vast majority of reading situations do not involve annotation. Much of the time that we read, we neither do it with a pen in hand, nor do we need to. That said, there are many types of active reading in which the ability to annotate the material is important, if not vital. In these cases, if annotation capabilities aren’t offered by the eBook, then the reader usually prints the material on paper just for the ease with which paper documents can be marked on (O’Hara & Sellen 1997). Even the most environmentally conscientious reader turns to the printer when asked to review a journal submission, to proofread a document, or to refer to the document in situations that a laptop (or even a portable reading device) would be awkward. It’s easy to see what becomes of the marks on print books. Whether the reader likes it or not, if the marks have been written in pen, they will become part of her library. If the reader is a well-known public figure, these marks may even have intrinsic value as a window onto the reader’s thoughts. Even if the reader is not a cultural icon, if she thinks the marks are private, she’ll have
50 reading and writing the electronic book
to go to some effort to get rid of them: how many of us have erased lightly penciled notes from a margin before we let someone borrow one of our books or before we pass an article we have read to one of our colleagues? What then becomes of annotations on electronic books? Their disposition has become a choice, not an absolute. Are they indelible, as they may be on paper? Are they propagated to other copies of the book? Are they shared when the book is shared? Can they be made public as part of a rating scheme? Just how public is public? Suddenly, something as trivial and quotidian as a reader’s annotations can become a burden on the reader and a puzzle to the developer. First, let’s continue to be clear about what we mean by the word annotation. All through this chapter, we have taken annotations to mean “the unselfconscious marks people make when they are reading” and NOT the published scholarly commentary that might look the same as those un selfconscious marks. This distinction is important. Even when people casually share their personal annotations, if they want others to make sense of what they’ve produced, they will need to treat annotation as an explicit authoring activity; they will have to go out of their way to make their annotations intelligible to others (Marshall & Brush 2004). Chapter 4 plunges more deeply into how the artifacts of reading may be shared with others. What of anonymous marks that a reader might happen into inadvertently, say, by buying a used textbook? Any assessment of value in this case is likely to be tied to the reader’s perception of who the mystery annotator was: if the annotations are perceived to have been made by an expert, the reader may very well be influenced by them, but if they were made by a peer or someone who is new to an area, they are less well regarded and may even be annoying. As such, the reader looks for clues to indicate the annotator’s role. In a series of experiments to determine the influence annotations have on future readers, Wolfe (2000) discovered that annotations do affect the persuasive qualities of a text and that the students are most positively influenced if they think the annotator is a disciplinary expert. What about a person’s own annotations? It is important to consider whether they were made last week, last year, or another time in the reader’s life, and under what circumstances (i.e., why was the reader reading the book in the first place and why was she marking in it?). Unsurprisingly, students in the Pocket PC study ventured that the annotations they had made in class were more valuable and more authoritative than the annotations they had made when they were reading on their own (Marshall & Ruotolo 2002). Old annotations—even annotations that had seemed valuable at creation time—did not weather the passage of time well. One student, a graduate student in English Literature, talked about the annotations that she had recently encountered in the texts she had used as an undergraduate: Some of them [the annotations] are absolutely ridiculous and I can’t believe that I actually wrote this in pen in this book. Some of them are—I have no idea what I’m talking about.
Interaction 51
Some of them are really interesting, and it’s something I’d forgotten. It just depends on the notes. . . . When I did Milton, we were doing the epithets about Satan or something, so I underlined all of them. And when I was going back through it, I’m like “what on earth!” (Marshall & Ruotolo 2002) Note that she hasn’t roundly condemned all of her old notes—she admits “some of them are really interesting, and it’s something I’d forgotten”—but she does have misgivings about the value of her old annotations as a whole.
3.2
NAVIGATION
Navigation in electronic books has long been a topic of inquiry. Is full hypertext linking an efficient means of navigating through electronic books? Or are links disruptive? Is page turning better than scrolling or is it the other way around? Are book-like navigational structures such as the table of contents or an authored index sufficient to support reading on the screen? Because there is no way of avoiding navigation (short of publishing material that is sufficiently brief that it fits on a single screen so that readers never need to go anywhere), it is important to give navigation its due. Just how vital is navigation? Are readers mostly navigating while they are reading linearly (in which case, navigation is straightforward and occupies a relatively minor proportion of the readers’ overall attention) or are they jumping around in the publication—glancing at a paragraph here, skimming another one there, and reading deeply less frequently—a situation in which navigation plays a relatively more prominent role? Genre, activity, and context must all be considered to answer this question. Suppose the reader is holding a Kindle-like device in one hand, reading the newspaper while he is standing up on the subway; in this case, not only is navigation important, it’s also important to be able to carry it out one-handed. Or suppose our reader is lost in a novel; it would be best for navigation to simply disappear. In a study that focused specifically on navigation, we found that while our magazine-reading participants spent most of their time actually reading, they also glanced at and scanned far more pages than they read; when they were not reading carefully, it was important for them to be able to move along quickly (Marshall & Bly 2005b). In other words, navigation is intertwined with the act of reading; at best, it is both essential and invisible. Consider what Barthes said about the rhythm of reading in The Pleasure of the Text: We do not read everything with the same intensity . . . a rhythm is established, casual, unconcerned with the integrity of the text; our very avidity for knowledge impels us to skim or to skip certain passages (anticipated as “boring”) in order to get more quickly to the warmer parts of the anecdote . . . we boldly skip (no one is watching) descriptions, explanations,
52 reading and writing the electronic book
analyses, conversations; doing so, we resemble a spectator in a nightclub who climbs up onto the stage and speeds up the dancer’s striptease, tearing off her clothing, but in the same order, that is: on the one hand respecting and on the other hastening the episodes of the ritual (like a priest gulping down his Mass). . . And yet, it is the very rhythm of what is read and what is not read that creates the pleasure of great narratives: has anyone ever read Proust, Balzac, War and Peace, word for word? (Proust’s good fortune: from one reading to the next, we never skip the same passages.) (Barthes 1975) Many researchers’ questions about navigation have probed whether the behavior of digital media should be modeled after that of print forms. What is gained and what is lost by adopting the familiar tropes of pages and page turning? Are the standard navigational shortcuts like indices and tables of content, coupled with basic keyword search capabilities, enough to get the reader to her destination? Are the constraints and metaphors inherited from print media powerful or limiting? Research results answer these questions with a resounding “it depends!” As with other electronic book functionality, we need to ask ourselves what the reader is doing. Is the reader engrossed in a novel on an airplane, researching legal cases at her desk, or skimming a magazine in the laundromat, looking for the next story to read? If we assume that the reader is “lost in a book,” we wouldn’t need much convincing that page turning is the essential mode of navigation; indeed, there is a reason such books are referred to as “page turners.” But if a reader is researching a legal case, she will be looking first for a string to pull, a place to begin her research; once that key case is located, link-following—jumping from case to case, forward- and backward-chaining from a precedent to newer decisions—is a primary way that the reader goes through the materials. By contrast, skimming a magazine may be very different from either of the other situations; it may involve thumbing through many pages very quickly, glancing and moving on, progressing at varying speeds according to expected interest. Navigation is so fundamental to the success of electronic books that it is worthwhile to give it focused attention. It may be addressed by clever hardware solutions, by a variety of user interface techniques, or by using a combination of the two. Navigation hardware or software can easily become the sticking point of an eBook product. For example, one reviewer criticized Amazon’s Kindle saying, “The placement of the Kindle’s buttons on the side means that the first time you hand it to someone, they almost always turn a page accidentally.” Microsoft’s Reader (which is implemented as a software application that runs on a variety of mobile hardware) was criticized for the feedback it gave readers about where they were in the book. What must this navigational hardware or software support? At the very least, the reader must be able to move from one point of interest to the next, whether it is the next page, the next chapter,
Interaction 53
the next story, or some other narrative element. But that’s not all: the reader must also be able to answer questions that allow her to feel oriented. How long is this book? Where am I in it? What else is in this book? How much have I read so far? How much is left to read? Thus, we can think of navigation as having two different components: the ability to move around in the material, and the ability to stay oriented. Each of these components is more complex than it looks at first blush.
3.2.1 Three Navigation Scenarios As we explore different ways of moving through a book (or other sort of document), let’s keep three scenarios in mind. These are three ways participants in a recent study were observed (and described themselves) reading the New Yorker, a magazine that contains some long articles as well as some shorter features, cartoons, and advertising. The scenarios are taken from a paper that describes a study I performed with my colleague Sara Bly (Marshall and Bly (2005b). Our first reader, Jay, is a planner. He either looks through the table of contents for familiar authors or interesting topics or leafs quickly through the magazine before he starts reading in earnest. Because Jay is a movie buff, he often starts by reading the Current Cinema, a feature he locates by its position in the magazine (a few pages from the end). Subsequent navigation is guided by the table of contents; he dog-ears that page so he can get back to it easily after he finishes an article. Jay reads the cartoons serendipitously as he encounters them, but he doesn’t go looking for them. He doesn’t read the whole magazine; however, he does do one final flip through the pages to determine whether he’s “done” with an issue. Jay often reads himself to sleep, nodding out as he concentrates on the long articles. Our second reader, Gene, told us that he progresses straight through the magazine (in a manner similar to Barthes’ rhythms of reading), from cover to cover, reading, skimming, or glancing as interest and circumstance dictate. Although he takes in the table of contents as part of his linear perusal of the magazine, he referred to it as a preview, and said it doesn’t necessarily change his course. He interrupts himself as he reads a longer article to look at the cartoons—they are his first priority. Gene is conscious of the rhythms of his reading, describing the different types of skimming and glancing that he does (e.g., when he looks for names in an article that is otherwise not of interest) in addition to more focused reading. Some navigational nuances are apparent as Gene talked through how he selects what to read: As he reads the Table of Contents, Gene refers to a profile of sculptor Maurizio Cattelan: “I don’t care about ‘Profiles’ probably. . . . I’ll just look at the photographs by Richard Avedon, because they’re nice photographs. When I get there.”
The names in these scenarios have been changed to protect the participants’ privacy.
54 reading and writing the electronic book
When he reaches the start of the article, he re-affirms his earlier stance, “I’ve never heard of this artist. And the title is ‘The Prankster.’ And this face is completely distorted. This is not a subject that would attract me.” But when he finally reaches the Avedon photos in the article, he tells us: “This [the photos] got my attention. . . . So now, by virtue of [the photos], I might read the first paragraph. Otherwise I wouldn’t.” He backtracks and begins to read the long article. (Marshall & Bly 2005b) There are several things of note in this interview segment. The table of contents is used as an orientation mechanism, rather than being directly involved in navigational movement. Article metadata and elements like photos are another important aspect of Gene’s navigation—they confirm that he’s where he thinks he is. Serendipitous encounter with article content (in this case, a photo) causes Gene to recant his decision to skip the article; thus, he needs to find its logical beginning so he can read it. Note that a standard hypertextual jump would have had entirely different results: Gene never would have encountered the photo and thus would never have read the article. Skipping an article by turning its pages quickly has a different overall effect than skipping to the next article using a link. Finally, our third reader, Constance, flips randomly when she gets the magazine, looking for something to catch her eye. But her flipping is not entirely random; the magazine has a predictable structure, which Constance uses so she can be certain to check the poetry and other features she likes. She too uses the table of contents as a preview, not as a navigation mechanism. Constance actively seeks ads for new books and is easily distracted as she reads. Like the others, she relies on the magazine’s predictable structure for navigation and to maintain her sense of orientation. Naturally, these three scenarios not only reflect the type of reading and the genre of the publication; they also illustrate some important characteristics of navigation, not the least of which is the individual differences. Neither the material (a literary magazine) nor the activity (reading for entertainment or information) predicts how individuals will approach navigation. Close observation of these three readers (via videotaped sessions where and when the individuals normally read) revealed some other general patterns: the overwhelming majority of navigation acts were forward page turns; when people read print publications, they use a variety of lightweight navigation techniques to manage their attention; and readers rely on serendipitous encounter (via flips and incidental reading) to expose them to material they might otherwise skip if they relied on metadata (such as titles and authors) alone. What do we mean by lightweight navigation techniques? These are subtle interactions with the material to support the nonlinearity of reading: narrowing or broadening the focus by manipulating the physical magazine (folding the page or opening the magazine to look at both pages at once); letting one’s eyes stray to a page element out of the textual flow; looking ahead in the text to preview or anticipate; and looking back to re-read for context (e.g., to find the place in an ar-
Interaction 55
ticle where a person was introduced or a word was defined). It is not surprising that people have developed navigation techniques to manage their attention; if we look closely at annotation, many unselfconscious (and unremembered) underlines and highlights are for that same purpose, focusing one’s attention on specific passages. It is instructive to look carefully at a sequence of actions when someone reads a print article. Figure 3.2 breaks down Constance navigating through an apparently linear story frame by frame.
FIGURE 3.2: Detail of navigation while reading a print magazine. (a) Constance reads the initial page of a book review. (b) She finishes the first page and reaches for the binding in preparation to turn the page. Note that she can still see what she is reading. (c) She turns over the magazine and encounters a full-page graphic. She glances at the graphic. (d) Then she reaches to turn to the continuation of the review’s text. (e) But she lingers over the entertaining caricature before she completes the page turn. (f ) Constance opens to the next two pages of the review’s content, which gives her a glimpse of what’s to come and quickly informs her that there is much more to the review. (g) She folds the left page (the next page in the article) under. (h) She then flips the magazine to continue reading. But first, she glances at the advertising column, as is her habit, to see if there are ads for new fiction. (i) Then she successfully completes the flip of the magazine so the left page—next to read—is on top. (j) She changes the position of her hands so she’s holding the magazine comfortably, ready to continue reading the second page of the review. Readers frequently move their hands to their faces or hair while they read.
56 reading and writing the electronic book
How can this detailed look at reading on paper inform navigation facilities for an electronic book? First of all, we can see that the reader performed a series of useful lightweight navigational acts without extra thought or effort. It is also evident that the reader is always looking at something meaningful when she is reading (which is not always true of electronic book interfaces). But it is more provocative to realize that this typical series of maneuvers took time—6 seconds to be specific—and that there is a certain awkwardness to them: there’s a need to smooth and refold pages and a very real potential for accidentally losing one’s place when the magazine’s pages stick together. Having taken this close look at real navigation as part of reading, let’s return to the problem of moving around and orienting oneself in an electronic book.
3.2.2 Moving Navigation in electronic books can be book-like; it can also take advantage of the fact that we’re reading on a computer. Book-like navigation sometimes means that we are going to rely on the physicality of the book to suggest digital functionality. In other words, we may keep the page turning metaphor. Indeed, user interfaces based on page turning may regard the physical act of turning the page as so central to the experience of reading that they simulate the physics of the paper page (Chu et al. 2004). In such interfaces, the speed and fluidity of page replacement trumps other possible functionality. Since their inception, electronic books have raised the pagination controversy: is it better to turn pages than it is to scroll? In 1988, Frank Halasz dubbed this controversy as “the card sharks versus the holy scrollers” (Halasz, 1991). He was referring to the argument between proponents of Hypercard, an early Macintosh-based delivery vehicle for digital content that used a metaphor of screen-sized cards, and systems like Bell Lab’s SuperBook or Jef Raskin’s Canon Cat, which relied on searching, scrolling, and other means of navigation through long unpaginated documents. Although many eBook products have implicitly taken sides in this pitched battle, there is little evidence that one technique is reliably better than the other. Studies may uncover performance differences between these two ways of implementing linear navigation (e.g., Liesaputra & Witten 2008); however, they do not offer incontrovertible evidence, and user preference seems tied to many factors (what the user is accustomed to; how good the hardware or software scrolling mechanism is; the type of material that is used in the test; and other characteristics of how the system, content, and user interact with one another). Furthermore, pagination can imply several different things. Pagination is sometimes implemented as a fixed page size to emulate a print page; the PDF format has this characteristic. That
Interaction 57
way, both readers and writers can rely on the fixity of the page; the artifact is stable regardless of the rendering hardware. For instance, the instruction to “look at the first paragraph on page 8” will always mean the same thing, thus supporting simple collaborative use of the book or document without any further functionality. On the other hand, displays have varying properties, and commitment to a paginated presentation may not be the same thing as commitment to a fixed page. Microsoft’s Reader uses a paginated layout but recalculates the page size based on changing properties of the target display (e.g., the size of the display and the desired font size). Book-like navigation does not preclude using the structure of the book to navigate. Print books have developed various conventions for structural navigation such as Tables of Content and manually constructed indices. If these conventions are realized through functional markup (e.g., by naming the section divisions) and are coupled with links, electronic books can have easy-to-use navigation mechanisms based on structure. Some print books also provide a physical means for accessing the book’s structure. Think of a large dictionary with tabs on the side that allow you to open the book to the desired letter. Such physical tabs are sometimes emulated in electronic books by means of a visualization or by some other generalized jump function that does not rely on the exact structure of the document. Reading on a computer means search can be used to navigate within a document. Searching is especially effective if the reader is trying to refind a place she knows to be in the book. A simple search facility can be implemented using a standard text-indexing capability. That is, for collections like personal libraries, search is usually divided into two kinds of processing. First, an index is created beforehand (and extended when new content is added) by parsing the text in the items and building a list of terms, in which each term is associated with all locations in the text that the term occurs. At retrieval time, the index allows the search function to quickly locate and display passages that contain instances of the term (or, by union or intersection, terms); the reader may use this capability to navigate to the desired place (or places). Naturally, this is an oversimplified explanation of information retrieval (IR); concepts such as stop words (words that are too common in the collection to use as indexing terms) and stemming (reducing terms to a normalized or canonical form) are discussed in many basic IR references (e.g., see Marchionini 1995 for a discussion of information seeking from a user perspective; Salton & McGill 1986 for a basic primer that explains modern IR). This type of term search is familiar to most readers, since it is used in many applications and in most curated online collections.
Web search is more difficult because it requires the use of adversarial techniques. The simple content-based search we describe here is most useful when the items are well-represented by the terms they contain; on the Web, the pursuit of viewer attention has provoked content creators misrepresent what is in the documents. Web search more commonly uses inbound link-based metrics of popularity as first suggested by Kleinberg (1998).
58 reading and writing the electronic book
But searching based on words and phrases in the text may not get the reader where he or she wants to go. Generally, unless the reader has read the book before or knows a great deal about a subject area, terminology is going to be an issue. Either a given term is too general and won’t take the reader to a useful point in the book even if the reader uses repeated searches for the term or the author may use different language than the reader expects. For example, suppose a student is writing a paper about Shakespeare’s use of bird imagery. She remembers a passage in Merchant of Venice rich with birds. But she won’t be able to find the passage if she searches on “bird”; she’ll have to guess the name of a particular bird to navigate to the passage. In other words, if she thinks to search for wren, crow, lark, or nightingale, she’s in luck (unless she spells nightingale incorrectly!); if she searches on finch, she won’t find the passage. Further discussion of search functionality, and what it may be used for in eBooks, may be found in Chapter 7.
3.2.3 Orienting Orientation is the flip side of navigation. Navigation asks the question “how do I get there?,” while orientation asks the question, “where am I now?” This question may refer to the reader’s relative location in her personal library (“Which book am I reading? Which part of the collection is it from?”), in a particular book (“Where am I in the book?”), or in a subpart of the book (“How much of the chapter is left? What else is in this book?”). Thus, orientation need not refer to the entirety of a book or collection; it can also refer to attempts to discover one’s relative location. Print books offer the reader many clues about where she is. The reader can glance at the book’s pages and see the relative heft of pages that have been read relative to pages left to read. Page numbers are a simple orientation mechanism: if a book has 435 pages and the reader sees that she is reading page 200, she knows she is almost halfway through the work. A reader may also flip through a work quickly with the intent of orienting herself as opposed to navigating to the next place; in other words, the flipping will not result in a new focus of attention, but merely a renewed sense of where the reader is right now. There are a variety of ways of allowing the reader to orient herself; these techniques may involve some combination of abstract representations of place (e.g., page numbers), visualizations (e.g., maps of the book’s structure), and actions that will only temporarily disrupt the reader’s current focus of attention (e.g., flipping through a book or glancing ahead or back; several of these actions are illustrated earlier in this chapter in Figure 3.2). In practice, eBooks usually take advantage of some combination of these techniques to give the reader this vital sense of where she is. We will explore each in turn. First, one can use compact representations of relative location such as page numbers or headers and footers. An eBook reading application can number the pages so the reader can tell that she
Interaction 59
is on page 200/435. Because pagination in eBooks can vary according to display properties (a small screen as opposed to a large one) and reader preferences (a large font as opposed to a small one), readers sometimes find these traditional measures of progress and orientation to be frustratingly difficult to interpret. For example, if our reader is tackling Charles Dickens’ substantial Victorian novel Bleak House on an iPod, the work may seem to have an endless number of pages. We saw this effect in our studies as students (and other Pocket PC users) read long works on small screens; an English Literature graduate student described reading on her HP Jornada accordingly: You get this little screen, so you get no sense of even how long the work is . . . You have 600 pages, which means what? No-one knows. And so . . . I definitely don’t see it as a literary experience (Marshall & Ruotolo 2002). Because of the orientation difficulties introduced by this lack of material cues, most on-screen reading software couples representations like page numbers with visualizations of reader progress through the work; these visualizations may be interactive so they can be used for navigation as well as orientation. Figure 3.3 shows an example of a visualization that reclaims the lost material cues by simulating the physical properties of the book (e.g., thickness) to help the reader regain a sense of orientation. Others (e.g., Liesaputra & Witten 2008 and the Open Content Alliance) have constructed similar interfaces with varying degrees of attention to the literal physics of turning the paper page. A third means of supporting orientation builds on standard navigation functionality: orientation is a matter of hanging onto one’s current position while one looks backward or forward to gain a sense of what is to come or to remind oneself of where one has been. Figure 3.2f shows how judicious placement of a “finger” as a place marker allows one to flip forward rapidly and come right back and resume reading. Why is orientation vital? At first blush, orientation looks to be an insignificant subset of what might be implemented by way of navigation. Yet our studies have shown that if a reader lacks a good means of discovering where he or she is in the text, he or she becomes frustrated with the on-screen reading experience. Even in a linear reading trope (e.g., reading a novel with a conventional plot), readers interpret what they are reading by using their sense of orientation; a murder on page 25 of a mystery will be apprehended very differently than a murder on page 375. Returning to the navigation study (Marshall & Bly 2005b), let’s look at two examples of a reader orienting himself. The first involves a quick look forward, probably to see how much is left of a long article and to preview where the article is going, followed by an oscillating return to the spot the reader left off to regain purchase on the stream of text. Figure 3.4 shows the log of this orientation process; it is evident how important—yet how seamless—this combination of actions is to the reader’s progress through the long magazine article.
60 reading and writing the electronic book
FIGURE 3.3: The British Library’s page-turning visualization to aid the reader in navigation and orientation.
Let’s look more closely at the participant’s actions when he is returning to page 104 and page 105 to continue reading (the portion of the log below the double line) after his excursion to the end of the article on page 111. When asked to comment on what he was doing, the participant said: So something has made me re-read something that I just read. Now that happens to me. And I’ll think, “oh, oh, they’ve just referred back to something.” And so I’ll go back and re-read it. What often happens to me is an article will have a lot of names in it, and . . . it’ll keep mentioning somebody, and I think, “Oh, who the hell was that?” And so I’ll go back, scan for their name, until I don’t see their name any more, and then figure, “Okay, they’ve got to introduce this person” (Marshall & Bly 2005b). From this example, it is easy to see both the importance and the invisibility of navigation and orientation. If the reader is unable to accomplish this basic part of reading easily, he or she may reject eBook hardware and software out-of-hand. Although Figure 3.4 reminds us what a small part
Interaction 61
Time Code
Seconds
00:00
122
Picks up magazine (12/3) to continue reading David Kelly article.
02:02
178
Turns to p. 101; continues reading.
05:00
121
Turns to p. 102; continues reading.
07:01
13
Turns to p. 103, then to p. 104, where article continues. Glances at p. 105.
07:14
126
Resumes reading on p. 104.
08:20
4
Turns to p. 105.
08:24
6
Turns to pp. 106–107
08:30
5
Turns to pp. 108–109
08:35
5
Turns to pp. 110–111, where the article ends.
08:40
4
Turns back to p. 104.
08:44
14
Turns to p. 105; resumes reading.
08:58
62
Turns back to p. 104; resumes reading.
10:00
Action
Turns to p. 105; continues reading.
FIGURE 3.4: Navigation log for participant “Jay.”
of our time is spent actually navigating and orienting (out of over 10 minutes of reading time, the participant spent less than half a minute looking forward to the end of the article and looking back to reorient himself ), it also speaks to the significance of this time. Would the participant have finished the article if he couldn’t have found out how much was left, or would he have given up? Would it have made a difference to his interpretation of the article’s meaning if he discovered he was on the sixth page of a 100-page article, one that took up the entire issue of the magazine? Would he have been able to recover context gracefully had he been unable to look back and see who someone was? One danger introduced by precise navigation is the potential loss of serendipity (Marshall & Bly 2005a, 2005b; Marshall 2007). Because flipping through a print publication is, by its very nature, imprecise and nonlinear, it affords a great deal of serendipity. We should not underestimate
62 reading and writing the electronic book
the importance of this phenomenon. Consider the way a reader describes his interactions with the New Yorker: An article will go on and on . . . [and] at some point I’ll think, “okay. I’ve gotten this whole point. I’m tired of this. When is this going to end?” And if it’s like three more pages, then I may just either give up. Or just go into a scan mode, where I just flip, you know, see what grabs my attention (Marshall & Bly 2005b). Navigation has been an underappreciated aspect of reading. Successful eBook interaction will include inventive hardware and software mechanisms that will more fully support a reader’s range of navigation and orientation activities. Because on-screen navigation has the potential to be more comfortable and less awkward than reading print books (there are no pages to smooth and refold), and the capacity to be more imaginative (consider the many kinds of interactive visualizations that are possible and the types of haptic feedback hardware that have yet to be put in readers’ hands), we can expect considerable development in this area as the next generations of eBook platforms unfold.
3.3
CLIPPING
We have all had the experience of opening our email in the morning and finding a newspaper article sent by a friend, colleague, or family member. For the sake of consistency, we will refer to this mode of interaction with reading material as clipping. Clipping is a way people extract, save, and share portions of a longer work—a chapter of a book, for example, or an article out of a magazine, or even a memorable paragraph hidden deep within a long novel. Will people still want to clip if they can simply search and refind the item of interest again later? We can argue that they will, because people clip for many reasons (Marshall & Bly 2005a). They sometimes clip because they have encountered something of interest that’s off the immediate topic, something they don’t expect to remember or have reason to look for again. They may also clip an item to share it with someone (many of our most memorable experiences with clippings may well be in this category). In a study Sara Bly and I performed of how demographically diverse people clip material out of print and electronic periodicals (including magazines, newsletters, and newspapers) (Marshall & Bly 2005a), two findings were readily apparent: (1) each participant performed some variation of clipping, and more notably, each participant shared clippings and received them from others; and (2) at the outset of each interview, most participants denied that they did so themselves, preferring to ascribe this habit to friends, colleagues, and family members. Some emphatically denied saving clippings, although by the end of the interview, we had turned up many examples they had scat-
Interaction 63
tered through their physical files, stashed in their computer folders, slipped into print books, saved in their email, and stored in any number of other physical and virtual places. Thus, like annotation, clipping is an unselfconscious practice. We will examine clippings as we did annotations—by looking at as many examples as possible to understand their form, function, status, and value, and when they are used and how—with an eye toward how clippings can become part of the functional substrate of electronic books. We will start with clippings that people save for themselves. Because clipping is unselfconscious, it is to some extent invisible. It is also surprisingly ubiquitous; all the participants in our study showed some evidence that they had clipped material themselves or they had saved material someone else had clipped on their behalf. What were they doing with these clippings? Although our study focused on clippings taken from periodicals, because the periodicals represented so many genres, the results are likely to apply to material one would find in equally diverse genres of electronic books. When people clip material for themselves at home and at work, the most straightforward plan they have for the clipping is to use the information for future reference; in these cases, the information itself was regarded as valuable. For example, the owner of a commercial nursery showed us a large box that contained both whole magazines and clippings from other periodicals that are about interior decorating. She described the cache as “pictures of ideas” that she doesn’t intend to look at again until she has settled into her new house, which wasn’t yet built at the time of the study. Similarly, a financial consultant saved an article from The New York Times about a client in Hawaii. He planned to revisit the article before he took his next trip to Hawaii so he could “remind [himself ] of the situation that’s going on over there. There is some tension between some of these institutions and the locals there. So I wanted to make sure . . . I’d be aware of the situation.” Substantial collections of clippings may be used at the anticipated time, but our findings established that other clippings were forgotten and not brought to bear when they would have been the most useful. Time-sensitive clippings often expired before the clippings were re-encountered. Because clippings are hard to organize—they may not fit within any preexisting category and are thus prone to misfiling—they are difficult to find again. If the clippings are saved by reference [e.g., as universal resource locators (URLs)], the actual clipping may no longer exist by the time it is rediscovered. It isn’t unusual for people to realize that they are prone to forget their clippings. They thus devise strategies wherein they don’t even try to manage them. For example, a study participant who was a partner in a design firm told us: I rarely keep anything . . . Mainly because I’ve just proven to myself that they just go sit in a folder and I never look at them again. So I try to grab as much as I can out of it and continue on . . . I’m opportunistic. If I have the time, and the article comes that makes sense—whether it’s sent via link, I find it online, it comes in a magazine—then I’m done with it at that point . . . (Marshall & Bly 2005a).
64 reading and writing the electronic book
Nonetheless, this participant still had many clippings scattered throughout his paper files, on his desktop, and in his email. People also keep clippings as a reminder for intended future action. In other words, they’ll keep an article, an announcement, an advertisement, or some other clipped material as a visible cue that prods them to do something. The content may only indirectly suggest the action, but it is sufficiently evocative for the clipper to remember what it meant. For example, study participant who was a professional environmental activist had photocopied a picture from a newspaper story about salmon; he intended to use this fish picture to remind himself to follow up with a fellow activist in Chicago: “I have a fellow in Chicago, an activist with [an interest in this topic]. And I photocopied this because I wanted to mail it to him.” He told us that he might even follow–up by phone, using the article as an excuse. This finding—that clippings act as reminders—echoes the results of others in the personal information management (PIM) community who have studied the functions of other types of office documents (Boardman & Sasse 2004; Jones, Dumais, & Bruce 2002). Of course, there’s an element of uncertainty using clippings this way: the reminder might not be in the right place at the right time and there are limits to available space and attention. Furthermore, it is difficult to leave digital clippings out on one’s desktop; an unexpected reboot can remove a clipping from the person’s immediate view, taking with it the reminder function. Not all clippings that people save for themselves have such pragmatic functions (as immediate sources of information or as reminders of things to do); they also may be emotionally evocative, calling an event back to mind. We have found that people intend to keep this type of clipping “forever” and thus become interested in aspects of sustainability. For example, a study participant who was a woman in her late 20s had kept a copy of Highlife magazine and a program for a cannabis festival in Amsterdam (“where everything’s allowed”) to remind her of a European trip she took by herself in 1995. During an interview, she told us that even though “that’s a long time ago. I’ll probably keep [the magazine and a program for a festival she attended] because it was an experience I had and I want to remember it.” It is also not uncommon for people to save published material that evokes the person’s place in a global event (e.g., a local front page from September 11, 2001 or a magazine published at the turn of this century). EBook platforms (in their role as venues for personal digital libraries) can facilitate this type of interaction (clipping and keeping) because there is less intrinsic danger to keeping too much.10 However, it is difficult to anticipate the long-term value of such a clipping, and we have more than once observed that upon re-encountering items such as these, people find their desire to keep them inexplicable as the years roll on (Marshall 2008a).
10
Although we must remind ourselves that storage is only one part of the cost of keeping an ever-growing digital collection; human attention must be factored into the total cost associated with sustainability ( Jones 2004).
Interaction 65
The most obvious and ubiquitous purpose of clipping is to share the excerpted material with others; in our study, all of the participants both shared clippings and received shared clippings. Remember that these clippings might not be an article literally ripped from a print newspaper; rather they might be a URL sent in an email message that points to an article in an online magazine. When we think of clippings in this inclusive way, it’s easy to grasp their ubiquity. A full 40% of the examples we saw during the course of the interviews were shared clippings. This dominance suggests we should pay close attention to the function and value of the material that is shared. What is the social role of this sort of sharing? After all, varying interpretations of clippings imply different functionality. In Chapter 4, we delve into the social function of clippings, but we will continue to focus on clipping as a type of eBook interactivity in this chapter. Figure 3.5 shows the relative frequency of different reasons people give for clipping material out of periodicals. This chart is not intended as an authoritative reference for how often people do one thing instead of another with their reading, but to show the scope of motivations people have for doing this kind of thing. Most importantly, the figure underscores that readers aren’t just saving this material for its information content. Thus, like annotating, clipping is a fundamental way that we should expect people to interact with electronic publications including eBooks. It is important to translate clippings’ functions and roles into requirements for eBook applications. Some of these requirements (e.g., refinding a desired item) have been explored at some length in the human–computer interaction (HCI) and IR communities (Dumais et al. 2003); others place more subtle constraints on applications. It is clear that like annotation tools, clipping tools need to be very well integrated with reading so that this kind of unselfconscious secondary interaction— clipping—can be performed without seriously interrupting the primary one. How often do people
FIGURE 3.5: The relative prominence of various reasons for clipping material out of publications.
66 reading and writing the electronic book
refuse to use tools on a computer because they are simply too intrusive at the time they are needed? Sometimes, it’s just a matter of a few extra clicks on menu items, just enough to divert one’s attention from reading and interacting with the material to wholly interacting with the computer, and not reading at all. Or the tools are ignored because the user worries that her social intent may not be reflected appropriately. Or the physical characteristics of the clipping are missing (e.g., their ability to visually remind), thus rendering the clipping ineffective in its anticipated role. Clippings remind us that context and provenance are necessary to keep the products of our interactions with eBooks useful. If a clipping’s source has been forgotten and cannot be inferred from a property of its form (e.g., a recognizable font and writing style), it’s difficult to reestablish its authority: it’s different to save an article from The New York Times than from the South Bay Daily Breeze,11 yet there may be ample reasons to do either. Clippings also remind us of the need to re-encounter the excerpts saved from our reading. Sometimes, clipping is part of a larger activity that is driving reading (e.g., the reader is a student who is writing a term paper and is collecting material for subsequent analysis and synthesis). But as we learned earlier in this section, clipping is sometimes driven by serendipitous encounter; in this case, the clipping context tells us little about what provoked the reader to save some part of what he or she is reading. Hence, clipping underscores the need to somehow re-encounter or remember what has been saved at the appropriate place and time. Implicit query (IQ ) capabilities may address part of this problem; IQ uses the current context, the other documents in use, to run queries against the user’s personal collection (Teevan, Dumais, & Horvitz 2005). But just as there is a need for serendipitous encounter, there is also a need for serendipitous re-encounter: emotionally evocative material (the issue of Highlife we used as an example earlier in the section) is not necessarily a welcome interruption if the user is focused on an intellectually demanding task. That clippings are used collaboratively implies a need to extend collaborative tools to work in the preferred reading environment. If I see a quotation in an eBook that reminds me of my neighbor Evert, I don’t want to interrupt my reading, bring up a mailer, create a new message, find Evert’s email address (or his IM identity), copy–paste the quotation into the new message, send it, and get back to my reading. Rather, I want this secondary interaction to take place in a manner that doesn’t interrupt the primary one.12 Because collaborative reading is such an important topic, and has so much impact on eBooks, it merits its own separate discussion in Chapter 4. This discussion includes some thought to how sharing clippings and other forms of encountered information contributes to the notion of social capital (ala Putnam 1995; Wellman et al. 2001). 11 12
The local paper that served the southwestern part of Los Angeles when I was growing up.
Many designers have already learned this lesson; I can send the document I am working on right now to another device directly from Word’s main menu. Unfortunately, many eBook reader applications exist in splendid isolation and assume that the user will be doing nothing else besides reading.
Interaction 67
3.4
BOOKMARKING
What is bookmarking? Is it functionally necessary to distinguish bookmarks—specified places in an electronic publication13—from reader annotations? After all, highlights, underlines, and other marks in texts provide a way to return to a particular place. The most visible difference between annotations and bookmarks in print books is that bookmarks may be seen from outside the book rather than just being visible from within. Bookmarks may serve a number of functions and may either be a permanent or transient part of the reading landscape. At times, a bookmark may be entirely transient and simply hold the reader’s place across sessions: it is the place where the reader left off reading just before she was interrupted or just before she put the book down and went to sleep. Bookmarks of this sort may have no lasting value. A bookmark may also be useful in a limited scope, within the context of the task at hand, such as the bookmarks that signal the quotes to be used in composing a legal brief (Marshall et al. 2001). At the other end of the spectrum, a bookmark can be a persistent resource, a place the reader returns to repeatedly as a reference or shares with others. Let’s first look at transient bookmarks. These are often acknowledged as a just-in-time resource, one that is not supposed to have continued meaning after the fact. A student confronted with one of her own bookmarks during an interview said: Yeah, I have a dog-ear in here . . . I can’t remember what that signified. But at the time when I folded the page over and probably at the appropriate instance after that, I remembered what that meant, what I was supposed to go back to (Marshall & Ruotolo 2002). In cases like this, bookmarks and annotations serve very similar abstract functions over time: they form a personal geography over one’s reading material, reminders of where one has been, if not substantive comments on what one’s reading contains. Another student in the same study discussed her annotations as having the same geographic function as her bookmarks: So of this I’m starting to skim more. As it talks about things that are less relevant. I can’t remember—here’s another highlight, so I definitely read this far. Uh. I think I read to the end of the chapter. Yeah, there’s another highlight.
13
Here we are careful to distinguish eBook bookmarks from the Web browser function that sometimes uses the same term (but may be also referred to as a Favorite). Although these two types of bookmarks may have similar placeholding functions, they may also be quite different (finding one’s way back to a familiar place on the Web is quite different from opening a book to where you have left off reading). As such, they may be stored in a different manner (e.g., in a local directory) than the storage we might design for the equivalent eBook feature.
68 reading and writing the electronic book
Our observations of the roles of bookmarks echoes the findings reported in Abrams, Baecker, and Chignell (1998). They describe Web bookmarks as used to reduce cognitive load (i.e., placeholding across sessions), to facilitate access (i.e., a persistent landmark),14 and as a collaborative resource (i.e., to share material within a book, either as a place or as a quotation). We add to that taxonomy with the more general function of bookmarks as a part of the reader’s personal geography. Bookmarks need not be simply an anchoring point in the text. Sometimes, a reader purposefully marks her place in one document using a second document. Consider, for example, the case of a photo of gray clouds used as a bookmark in a meteorology textbook or a looseleaf sheet of class notes used to mark the chapter that the test is on. Although bookmarks like this (which are arguably very similar to annotations in that they have a body in addition to an anchor and marker) are less common than the other kinds of place markers (gum wrappers, playing cards, a ripped corner of a magazine cover, and the like), they pose an interesting point on the continuum. What is the status and value of these bookmarks? The type of bookmark that is most often shared is the kind that serves as a persistent resource; bookmarks used for other functions are rarely shareable because they are either transient (only valuable within a limited scope) or tacit (the reader gives no clue about why the bookmark is there) or both. I have observed bookmarks as important shared resources in places like law offices, where a group of attorneys will share a hand-tabbed copy of a book of statutes. Sometimes, this practice creates a profound problem when a new edition of the reference is published; the people who use the reference are reluctant to give up the added value of the bookmarks in favor of a book that’s more up–to–date. For the sake of representational consistency, it might be best to consider bookmarks as a kind of annotation (or vice versa) and to treat them in a similar fashion because they introduce the same design and implementation questions: are they stored with the book, in a separate local store, or in a central database? Because we can envision bookmarks that are themselves content (the cloud photo bookmarking a meteorology textbook), they may be seen as having the same general anatomy as an annotation: a body, which is usually null, but which may be bookmark content; an anchor, which is likely to refer to a physical boundary (a page) or to a major structural element (a chapter heading); and a marker, which refers to the visualization used to indicate the presence of a bookmark (say, a stripe along the edge of a page). In spite of the distinction we posed earlier in this section, our observations of the Web in use (by readers using the most common browsers of the day) bear out the generalizations we have made about how people bookmark print documents. While most people do use bookmarks (or favorites), they use them in very different ways. For some users, their bookmarks are almost wholly transient. 14
While we might readily tolerate a 404 error when our Favorites fail to turn up the proper Web page, we would feel quite differently if we reopened the book at a bookmark, and found a blank page.
Interaction 69
They mark a page they’d like to get back to in the near future, and—having bookmarked it—only use it a few times, possibly in the course of performing a specific task. Sometimes, these users have unruly lists of bookmarks, and only refer to the ones at the bottom of the list. Other times, they will organize these lists, but not use them (e.g., they are aware that few of their links work anymore). Still other users have bookmarks that are actually references—an online dictionary, a favorite porn site, a magazine they read regularly, an email portal, and so on. How do people organize the bookmarks that they keep? Abrams, Baecker, and Chignell (1998) describe a number of different methods for saving Web bookmarks (some of which seem more pertinent to electronic books than others). Users might not organize them at all (so they remain in the order they were saved); they might keep them in an ordered list; they might put them in folders; or they might export them to a separate application for the purpose. Thus, Web bookmarks are managed within a continuum of structure, from a time-based order which requires no user action, to managed structures. These results are similar to the findings presented by Shipman et al. (2004). Because this discussion of bookmarks is edging into the territory of personal libraries (i.e., managing multiple electronic publications) and PIM, we will relegate it to Chapter 7.
3.5 HARDWARE FOR INTERACTING WITH EBookS Active reading and other forms of non-immersive reading rely on the reader’s ability to interact with the material. We can divide readers’ interactions into three essential types: pen-based interactions (annotation and possibly clipping); navigational interactions (page turning, scrolling, flipping, and so on); and the interactions necessary to move from reading into related activities such as writing or communicating. Right away, we’ll relegate the third category to general purpose mobile devices: there’s no sense in reinventing the laptop, the smart phone, or any number of other multipurpose devices that enable readers to communicate, share material, or write. Hence, in this section, we’ll focus on two types of hardware: pen-like input devices and innovative techniques that support navigation.
3.5.1 Hardware That Supports Navigation As we learned earlier in this chapter, navigation is a fundamental interaction for reading. Although it occupies only a small portion of the reader’s time, it is an activity that represents a major fraction of the reader’s interactions. Thus, eBook hardware designers have endeavored to make basic navigation a prominent feature of their devices, realized as hardware buttons, sensors, or haptic feedback mechanisms. Buttons. Hardware buttons are probably the most ubiquitous means for turning pages in eBook hardware. Click a button; turn a page forward. Click a separate button; turn the page
70 reading and writing the electronic book
back.15 Page turning via a hardware button or rocker switch characterized most of the second generation of eBook devices. Figure 1.1a and 1.1b shows the page-turning buttons on the Rocket eBook and the SoftBook Reader. The first version of Amazon’s Kindle apparently got page turning wrong—no reviewer would let them hear the end of how easy it was to accidentally turn an eBook’s page by gripping the edge of the device to hold it. Even in the platform’s second incarnation, page turning receives considerable attention. Usability guru Jacob Nielsen wrote: Kindle shines in one area of interaction design: turning the page is extremely easy and convenient. This one command has two buttons (on either side of the device). Paging backwards is a less common action, but it’s also nicely supported with a separate, smaller button. (Nielsen 2009) Unfortunately, other types of navigation didn’t fare as well in Nielsen’s evaluation of the device; other types of navigation like moving around in the page and following links are relegated to a less successful joystick-like input mechanism. Sensors. Sensors are a means of detecting certain use conditions; they are mechanisms that measure an environmental characteristic and convert it into a form that may either be read from a display (as you would find in a thermometer), or (in our case) used as an input parameter to software that controls interaction. For reading software, a sensor might measure pressure or tilt, and convert it to input for a navigation technique. Ideally a sensor will not need calibration very frequently. Fishkin and his colleagues at Xerox PARC added sensors to a small reading device to detect user handedness to shift a page’s position in the appropriate direction so the margin would be large enough to accommodate annotations (Fishkin et al. 2000). They fitted another small device with tilt sensors (i.e., accelerometers); tilt may be used to represent desired scroll speed for skimming, so the lines literally fall off the page faster as the device is held at a steeper angle16. Finally, they used sensors on either side of the top of a display case to support page turning forward and back; the reader “flicks” the page in the desired direction by applying pressure to the pads. Chen et al. (2008) extended the idea of sensor-based navigation in their dual-display prototype. Flipping the display (the whole device, more accurately) advances or rewinds the page, depending on the orientation of the flip (clockwise versus counterclockwise). A hysteresis mechanism 15
Even when page turning is implemented in software, the focus is on one-button forward page turning. The Times News Reader’s basic scenario features a commuter reading one-handed, pushing a button repeatedly to page through the morning paper. 16
Apple’s iPhone has made this type of interaction commonplace via its on-board accelerometers; the iPod’s scroll wheel is a second example of how sensors can be incorporated to control scroll speed.
Interaction 71
saves the last page the reader has viewed on the second display. A similar gesture implements fanning through pages very quickly—a sensor means there is no need to interrupt the activity to look for a button or a software control.17 The Kindle uses an accelerometer to detect the orientation of the device. Tilt it one way and the page is displayed in the normal portrait orientation; tilt it the other and the display becomes landscape. It even detects when the device is upside down and orients the page accordingly. If the accelerometer is oversensitive, sometimes unwelcome accommodations are made.
3.5.2 Pen-Based Interaction Early eBook prototypes, such as FXPAL’s XLibris, used tablet hardware under the assumption that annotating with a pen required far less cognitive overhead than producing a comparable annotation using mice, menus, and a keyboard; freeform digital ink was seen as a way of providing relatively natural interaction with books and other documents displayed on the screen. (Schilit, Price, & Golovchinsky 1998b). Field studies of XLibris used in real situations (e.g., a reading group) demonstrated that this was indeed true; people read and marked in a very similar manner as they did using printed reading materials and a pen or highlighter (Marshall et al. 1999). In fact, the interaction was so natural that readers didn’t bother changing pen style and wrote instead with whatever pen they had in hand (as they would with a real pen). For example, a reader would write brief marginalia with the highlighter she was already using, rather than switching the stylus to write like a ballpoint-style pen (a change easily accomplished in the user interface). Although pen-based interaction is very natural for annotation, unfortunately it may require further thought about the rest of the user interface. What is easy to do with a keyboard, mouse, and standard menus may be difficult to accomplish with a pen input device. Adaptations like marking menus and gestural user interfaces have attempted to address some of the user interface problems that are introduced by pen input. Furthermore, studies have found that producing longer segments of text—notes and long annotations—is more comfortable using a keyboard. Even with a handheld device like the Pocket PC, several students wanted plug-in keyboards; students in the Pocket PC study said that they could type a great deal faster and more legibly than they could write, so when they were writing anything beyond the briefest marginalia, they wanted to type (Marshall & Ruotolo 2002).18 Much research has been performed to determine whether handwriting recognition is actually necessary 17
This functionality is duplicated by interacting with a trackball so the eBook can be flipped when it is set down on a table or other surface. 18
It should be noted that in this study, several students who had been exposed to Palm Pilots had found Palm’s Graffiti to be a more effective input method than trying to write normally on the screen.
72 reading and writing the electronic book
and whether other implicit structures (e.g., lists) can form the basis for interaction instead (Moran et al. 1995). Systems that do use pen interaction must be designed to work around the inaccuracies that are common in recognition.
3.6
ESSENTIAL BUT INSUFFICIENT
Thus far, I have strived to keep this discussion general and to describe the kinds of functionality and interaction that transcend specific intellectual work and eBook genres, and that are faithful to the interactivity offered by the physical form of the book (annotation, navigation, clipping, and bookmarking). Yet some of the most powerful functions developers can add to eBook platforms are specific in their intent: translation, inter- and intra-corpus linking, skimming tools, search capabilities, and analytic tools. This type of beyond-the-book functionality is so important that we will give it its own chapter, Chapter 7. The basic interactions and functions we explored in this chapter are the live-or-die aspects of the electronic book. Doing them right will not guarantee that readers will see the virtues of reading on the screen; but doing them wrong ensures that readers will reject eBooks out of hand. The ability to annotate, navigate, clip, and bookmark will not cause our hypothetical reader to say “Wow!,” but these are necessary components of interaction with electronic books. • • • •
73
chapter 4
Reading as a Social Activity I began Chapter 2, the chapter about reading, by calling into question the assumption that reading is a solitary activity. It isn’t: I invoked document scholar David Levy to assert that reading is inherently social. This is not to say that people don’t read alone; of course they do. They may even go to some lengths to separate themselves from other people in the name of quiet and to prevent unwanted interruptions. But reading is social in a way that crosses several dimensions beyond the immediate stereotype of a scholar deep in thought in a library carrel or a child curled up with A Wrinkle in Time in a picture window. To tease out these social dimensions, let’s call to mind the old but serviceable computersupported cooperative work (CSCW) two-by-two matrix that divides up the world (albeit somewhat artificially) according to place and time. These dimensions will give us four quadrants characterized by events happening in the same place versus events happening in distant places and events happening at the same time versus events happening at different times. Figure 4.1 sketches out this matrix, styled after Johansen (1988). This matrix gives us a simple framework to examine the social side of eBook use. Use that occurs in the same place at the same time implies that people are reading together. Reading together may mean that the situation has been deliberately organized as an opportunity to read together: reading groups or classroom discussions are both examples of such a situation. It may also mean that people have contrived informally to read together: two students push their chairs together to share a computer display or two friends talking on the phone browse the same Web pages together. The second variant, reading together over the phone, represents use that occurs at the same time, but in a different place. Activity in this quadrant of the collaboration matrix is usually intentional because it involves a communications technology; that is, the distance must be bridged by some kind of connection. Once people are reading at different times, some sort of persistent artifact of their reading activity must be involved: annotations, clippings, bookmarks, recommendations, page views, or the books themselves may be used as a record of the activity; otherwise, collaboration or some other kind of reading-related social interaction would be difficult. Hence, this aspect of sociality involves sharing records of reading. These records can be explicit, such as annotations, or implicit, such as page
74 reading and writing the electronic book
FIGURE 4.1: Collaboration place/time matrix.
views recorded in a log. Asynchronous sharing can likewise be something intentional and planned, such as a discussion of assigned reading conducted in an online annotation system (Brush et al. 2002) or the recommendations of books or authors that we might see in a social cataloging Web application like LibraryThing, or the sharing may be serendipitous and may take advantage of large-scale collective effects such as an online newspaper’s most emailed story. In the end, it seems that we have posed two basic distinctions that make very different demands on eBooks and their users. First, there are synchronous activities that involve reading together, i.e., either in a way that is co-located (e.g., reading together in a classroom) or remote (e.g., browsing the Web together over the phone). Then there are asynchronous activities that involve sharing records of reading. These asynchronous activities can be further divided into those in which the records of reading are intentionally shared (e.g., clipping a story from the local newspaper and mailing it to a friend) versus those in which the records of reading are aggregated and shared in a way that represents collective intelligence or the wisdom of crowds (e.g., assembling a bestseller list or a citation index). Taken together, these variations give us an interesting way to explore the diverse social side of eBooks.
4.1
READING TOGETHER
If we take the idea of reading as a social activity as literally as possible, then we will conjure up the setting of a classroom, meeting, or reading group—a place where people gather to read and to
LibraryThing (http://www.librarything.com) was developed by Tim Spalding; it is discussed later in this chapter in the “Sharing and recommending books” section.
reading as a social activity 75
FIGURE 4.2: Reading together in a structured situation. (a) Reading in a reading group. (b) Reading in the classroom.
discuss what they are reading. Figure 4.2 shows two examples of this kind of setting: Figure 4.2a is a video frame of the reading group that was the subject of the study reported by Marshall et al. (1999). Figure 4.2b is a video frame of an undergraduate class that participated in the study reported by Marshall and Ruotolo (2002). Although we can envision either group working from a projected display that everyone shares, in practice, it is more common (and for reasons we will discuss, in these cases more desirable) for people to participate in such a discussion referring to their own copies of the work in question. There are two relevant phenomena that we observed in these settings: the first concerns “getting everyone on the same page”; the second involves the transition from face-to-face discussions over reading materials to exploratory search (or to simple question-answering IR). These phenomena may also arise in less structured situations.
4.1.1 Shared Focus When people read together in a structured situation of the sort portrayed in Figure 4.2, they frequently refer to the materials they have read (and possibly annotated). This creates an immediate and pressing problem of shared reference: how does everyone in the discussion know that they’re
For the sake of argument, let us erase the colocated/remote distinction we made earlier and lump together distance learning with face-to-face classroom situations. What we really care about here is whether all of the attendees can see the same shared screen/page and whether they have one of their own.
76 reading and writing the electronic book
looking at the same thing? Shared reference is a persistent problem, even given stable print editions, although the problem is exacerbated when the copies differ for some reason—a different edition of the work has been read or an eBook has been reformatted according to display constraints. Thus, in the case of the reading group—a situation in which pagination is fixed and everyone is reading a copy of the same document—“sync” conversations were still required. In other words, reading group members still needed to know which part of the article the others were looking at, despite the apparent linear progression of the group through the document. Why didn’t they just all refer to the same display? It would have been easy enough to project the article so everyone could focus on the same text as the discussion’s emphasis shifted. Interviews with members of the reading group revealed that they wanted to see their own annotations in full context while they were paying attention to the shared topics. One reader remarked: They [his annotations] were all there in the physical document, and we’d gone from the beginning to the end anyway. And I wanted to get the context, so if somebody was saying something about something else, then I would have had it there. Readers rely on complete context rather than disaggregated parts (Bishop 1998). Although there are shortcuts to sharing a view of a single document, these shortcuts would have removed the reading group members’ ability to see their own annotations and notes in context. Observation of the group while they were discussing the article also revealed that one person or another might briefly and deliberately shift out of sync with the rest of the group to check something on his or her own. Figure 4.3 revisits the details of Figure 4.2. Notice how one member of the reading group is looking ahead in the paper; other members of the group are looking at the first page, while he is looking further. After such an excursion, the group member seamlessly regains purchase on the conversation without disrupting it to ask, “where is everybody?” In this case, he need only glance at where the other group members are in the article because all copies look essentially the same. In the classroom, matters were even more complicated. Rather than each student using his or her own copy of the same book, students (especially the English Literature graduate students) were each using editions of the text that they already owned (possibly from previous courses) or different editions that they had purchased secondhand (to save money). Sometimes, a shorter work used in the course was a small part of a thick anthology. Add to this mix several different electronic versions of the text, each reflowing according to screen or window size. Furthermore, some versions of the electronic text had been prepared in eBook format for Microsoft Reader and others in HTML for Web browser display. Obviously, page numbers—or in some cases, line numbers—were not a stable point of reference.
reading as a social activity 77
FIGURE 4.3: One member of the reading group skips ahead while still following the discussion.
Thus, if we want eBooks to support reading together while still allowing collaborators to look at their own annotations or on-screen context (e.g., notes in a separate window), they need to have easy ways of communicating a shared sense of place in the electronic work. People need to be able to co-navigate to the same place in the text without losing their own sense of context. It is telling that the undergraduate class in the study made extensive use of the materials on the Pocket PC in class, while the graduate class did not. This use (and nonuse) may have been influenced strongly by the way the course materials were structured: the undergraduate class navigated the course materials together in class using the preset hypertext links available in the table of contents; the materials the graduate class used were not structured this way. Instead, they would have needed to search for key terms to co-navigate to the right place in longer texts; search was reported as too slow and too clumsy to use in this way.
4.1.2 Collaborative Search and Reference Following One clear advantage of mobile devices like eBooks is that they offer the reader an opportunity to consult external references on the spot ( Jones et al. 2000). Furthermore, recent study results have revealed that exploratory search effectiveness is enhanced by collaboration (Pickens et al. 2008).
For example, if the class chose to navigate to a place in the text using a key term, it would have to be the first use of the key term in the work for the strategy to be effective.
78 reading and writing the electronic book
Taken together, these results might lead us to expect people who are reading together to launch into effective collaborative search sessions together, either for the sake of answering questions that have come up during the discussion or to explore new topics. Indeed, in our Pocket PC field study, the students (and faculty members) did just that. In this study, they were confined to using materials local to their devices for this type of search (the study predated ubiquitous wireless), but it is easy to envision broader on-the-spot research today. The undergraduate class used an extensive corpus of primary materials about the Salem witch trials, which had been preloaded onto each student’s Pocket PC at the beginning of the term; participants used the full course materials in classroom conversation to look up answers as questions arose. In a classroom session that covered Arthur Miller’s The Crucible, the discussion turned to a remark in the preface that referred to the hanging of two dogs at the Salem witch trials. The students immediately sought documentary evidence for the event; freed from the chronology mapped out in the syllabus, exploratory research evolved organically from classroom interest. Students were already anticipating the use of wireless connections in the classroom; they thought the kind of research that wireless access to the Internet would engender would make an interesting class even more compelling. As a related practice, we might expect eBook readers to follow references in this type of collaborative situation; reference following is discussed in Chapter 7.
4.1.3 Reading Together as an Informal Act When people read in public places, it may be the occasion for social interaction. Figure 4.4b shows two junior high school students reading together in a computer lab (the observation was part of the Walden’s Paths project reported in Furuta et al. 1997). Each student began the class session in the computer lab at his or her own PC (as shown in Figure 4.4a); by the end of the session, most of the students had clustered together in dyads (or more occasionally, in small groups). This is an extension of the phenomenon Michael Twidale reports in his discussion of over-the-shoulder learning (Twidale 2000): solitary reading turned into a productive kind of reading together as the students helped each other out and shifted their joint attention from the tools to the content. Over-the-shoulder learning may be essential to the adoption of electronic textbooks and other kinds of eBooks. In the Pocket PC study (Marshall & Ruotolo 2002), the students tended to teach each other strategies for using the new devices effectively. That is, rather than relying on the formal training resources offered to them by the Electronic Text Center, the students showed one another new ways to use the functionality that the Pocket PCs offered. Some of the learning contributed directly to the handheld’s use as a reading platform; other learning focused on the handhelds themselves (for instance, how to beam files to another handheld
Students, in general, viewed wireless access with great enthusiasm and anticipation, even when it was still relatively rare. For supporting evidence, see the work of Jones et al. (2000) on wireless tools in the stacks.
reading as a social activity 79
FIGURE 4.4: Examples of how people read together informally. (a) Students in classroom, one per computer. (b) Students began reading together. (c) A spontaneous gathering to read together.
or how to download games). The students might actively seek advice, but they were also opportunistic, capitalizing on opportunities to learn better ways to do common operations. One student said that he’d learned to navigate directly to a later page in an eBook by interacting with a classmate a few weeks after he’d received his Pocket PC: “Until then, I had to flip through.” Another student reported that she’d learned how to do things incidentally by socializing with other people who used the device. She said: It was things like closing applications. No-one knew how to do that, and my boyfriend was playing with it. And he said, “You have seven different things open. You might want to close them.” And so then I shared that with the class (Marshall & Ruotolo 2002). Likewise, Figure 4.4c shows an informal session in which two colleagues spontaneously began reading together in an open area in our workplace. This photo, taken in the mid-1990s, is of WebTV when the product was still a novelty. First, one researcher set it up and began browsing; not long afterward, another researcher joined him. It was readily apparent that the addition of a second person changed the dynamic of how the first person read. The two colleagues continued to read together, negotiating where to go next when both were ready to move on.
4.1.4 Peer-to-Peer Sharing Mobile devices seem to evoke a desire to reclaim the hand-to-hand physicality of sharing print publications. After all, it seems natural to share things by simply passing them from hand to hand, from one person to another. Then too, when documents are viewed as objects, meaning is conveyed by the way they are passed from person to person (Bowker & Star 1999). We also found this to be the case when the artifacts of reading are shared. In the Pocket PC study, students found the idea
80 reading and writing the electronic book
of beaming class materials to each other (via the infrared port) to be very compelling. Yet, in practice, peer-to-peer networking of this sort has been slow to materialize; instead, people have been using servers (i.e., “the cloud”) as an intermediary (Dearman & Pierce 2008). Yet, using peer-to-peer networking may be an interesting way to share material when people are physically proximate. In principle, it is easy to imagine co-workers syncing mobile devices so that they can share bibliographic updates to a local digital library. However, we should note that many DRM implementations preclude sharing eBooks at all; this vision of a shared local library will be difficult to realize under current restrictive policies. Peer-to-peer networking also comes into play as a personal digital library infrastructure; syncing copies of collections among a set of personal devices allows people to read and access the same material on different devices (without relying on a central server). We can think of sharing with oneself as differing from the type of peer-to-peer sharing described above. For example, you may want to share your own progress through an eBook across several reading devices so you can leave off reading on your Kindle and pick up the same eBook on your laptop.
4.2
SHARING THE ARTIFACTS OF READING
Once we move into the two lower quadrants of the matrix, we are talking about asynchronous reading, i.e., reading the same material at a different time; hence, something must be shared beyond the view of the same page or the discussion that focuses people on the same place in the text. To be inclusive, we will refer to that something (or those somethings) as artifacts of reading, tangible records that persist across time and space. These artifacts include intentional records, such as annotations, clippings, bookmarks, notes, and other purposeful things the reader has created while she was reading, and implicit records of reading that have been recorded by the eBook software that include logged events such as page turns, scrolls, opening and closing books, mouse clicks, and so on. Often, implicit records are referred to as telemetry because they allow someone to measure or observe the reading activity at a distance or at another time (or both). Let’s examine these shared artifacts more closely because sharing them is by no means straightforward. People often regard their personal annotations as just that, personal, and they may go to some lengths to remove them before they share their reading materials. Often, the records of reading are considerably more private than the materials themselves. In the United States, libraries have a long tradition of keeping the records of reading—for example, the list of books a patron has checked out—private; this privacy policy is so important that librarians have fought for it.
The American Library Association has developed guidelines for library privacy policies: http://web1.ala.org/ala/ aboutala/offices/oif/iftoolkits/toolkitsprivacy/libraryprivacy.cfm.
reading as a social activity 81
Yet—on the flip side—people sometimes want to know what other people are reading, especially if they hold the other reader in some regard (e.g., as an expert). Likewise, others’ annotations may be valued. What makes these artifacts private? Why would people want to share them? What makes these artifacts valuable? Are there ways of sharing artifacts of reading that may preserve privacy while giving others the virtue of their insight?
4.2.1 Reading to Know What Other People Know One common theme in many information use studies is that people want to know what other people know: in other words, they want to be in sync socially. In our study of news reading (Marshall & Bly 2005a), we saw that people are often motivated by a social purpose to read newspapers; this desire manifests itself in a desire to find out what other people are talking about: A lot of times what will happen is we’ll start talking about something in the office and everybody’ll be like: “oh. Well I saw it on blah-blah-blah a few days ago.” So it would be nice to be able to go back and find that article (Marshall & Bly 2005a). In the workplace, reaching social sync is particularly important in maintaining relationships with customers and clients. We have seen this effect in several studies. For example, a study participant who worked in a law office and characterized himself as a frequent reader of NYTimes.com said: What I really do with the NYTimes.com is I scan the headlines, because we have different practices, we have different clients. I like to know when someone is in the news so we can respond. So I can get my attorneys prepared—so we can be knowledgeable . . . I know where to look for Markets. I know where to look for Opinion, because I always read that. And then I scan down—I have really demanding health care guys—I’m always looking in those sections to see who’s there and who’s not. Or [looking for] related articles on the industry. It’s really about seeing it, finding it, and moving on. And I see it all at once (Marshall & Bly 2005b). Awareness is the key here: the participant is neither reading for comprehension nor for details, rather he is reading so that he knows “who’s there and who’s not.” In our study of the social use of clippings (Marshall & Bly 2004), we found that many clippings that are shared in the workplace (especially in customer-focused jobs) are passed around among co-workers for exactly this reason—to establish mutual awareness. Participants in the study frequently read periodicals with the idea that their customers read these same periodicals. The articles in these periodicals
82 reading and writing the electronic book
might, for example, help them understand the customer’s situation; a senior sales manager described putting an article on his colleagues’ desks if the article was about a significant event—layoffs or promotions—in the client’s company. He felt no need to explain a clipping like this because the practice is so common and ingrained. Awareness may extend to making sure that the form of the articles is reproduced: when study participants shared clippings for this purpose, they made an effort to duplicate the way that customers would be expected to encounter the information. So if the client would see the article in a print newspaper, the clipping would be distributed as it appeared in print even if the online form of the article was more readily available.
4.2.2 Sharing Annotations There has long been a tendency to idealize the annotations that people make while they are reading and to conceive of them in literary terms. In her “reading memoir” Ex Libris: Confessions of a Common Reader, Anne Fadiman begins with a rosy picture of shared annotations, “I have come to view margins as a literary commons with grazing room for everyone—the more, the merrier.” Some pages later, it is almost as if she has come to her senses and has peered inside one of these literary commons to see just who is grazing there and what they have left behind. Once inside, she succumbs to the harsh reality of the actual annotations: Not everyone likes used books. The smears, smudges, underlinings, and ossified toast scintillae left by their previous owners may strike daintier readers as a little icky, like secondhand underwear (Fadiman 1998). The shift from a literary commons to secondhand underwear may be warranted, if a trifle extreme. If we can share personal annotations, will we want to? What does it take to make a thicket of personal annotations to a fabled literary commons worth visiting? Annotations can be less lucid than their author imagines; unfortunately, it is not uncommon for readers to make far more annotations—underlines and highlights, especially—than they remember. EBooks (and electronic documents, in general) make it much easier from an implementation standpoint to share annotations and to move annotations onto different copies (or differ-
It should be noted that they weren’t just worried about the form; online and print sources may have the same headline, but one or the other goes into more detail. If they clipped the article from different sources, they ran the risk of the content differing in subtle, but possibly important, ways.
reading as a social activity 83
ent versions or editions) of a document (Brush et al. 2001; Golovchinsky & Denoue 2002; Hong et al. 2008). Scholarly annotations and commentary may be shared intentionally; in fact, this layer of scholarship that exists in the space between reading and writing is thought to be an important step forward in promoting the dialog between reader and writer (Bolter 1991). Digital documents also support the ready removal of annotations; as Chapter 3 discussed, some implementations maintain annotations separately from the base publication so they can be saved and not shared. Annotations may also be saved, stored, and not rendered; in other words, the annotations are both removed by virtue of being invisible and not removed by virtue of remaining in the file. This makes it very easy to share the annotations unintentionally. There was a legal case not too many years ago in which an attorney marked up a document for his colleagues. He turned off the rendering of the annotations to see what the final form of the document looked like, and without intending to, sent the document to the counsel for the other side with his firm’s annotations still intact. The sender was beyond embarrassment; he sought to assign blame and reprisal. This story is an apt illustration of the importance of the question of whether annotations should be shared: the answer is they may be extremely powerful and useful when they are shared and they may be extremely private too. A look at the typical annotation on a print book or document reveals that many individual annotations are hardly worth sharing or further thought: they are a persistent record of the reader’s own engagement with the material and not much else. Sometimes the reader finds them useful down the line, and more often not. Figure 4.5 shows a familiar type of annotation: a reader’s spontaneous reaction to the material. That the reader finds the content “Confusing to me” is probably of little use to other readers (although if every other reader annotated this paragraph similarly, we might think it was a difficult story. We address this case in Section 4.2.3). On the other hand, readers may use their personal annotations as a basis for what they share, modifying the annotation to make it intelligible to others (Marshall & Brush 2004). Figure 4.6 illustrates the relationship between the annotation a reader made for herself and the one she shared with her classmates. The annotation on the printed page (on the left) was very likely made in anticipation of how it would be used in the online discussion tool WebAnn (Brush et al. 2002), shown on the right. The example shown in Figure 4.6 makes annotation reuse seem straightforward. In practice, reuse and sharing is complicated by readers’ normal annotation habits. As Table 3.2 reminds us, most annotations are not immediately intelligible to other readers. Our study data reveals that only 9% of students’ personal annotations are complete and fully specified—i.e., they have a well-defined anchor (so the reader knows what the annotation pertains to) and an explicit body (Marshall &
84 reading and writing the electronic book
FIGURE 4.5: A reader’s personal annotation. Does it make sense to share an annotation that says, “Confusing to me”? Perhaps it does in a situation of high trust. Perhaps the teacher should see the annotation and the student’s peers shouldn’t. Perhaps the reverse should be true (the student’s peers see the annotation and the teacher doesn’t). One can imagine many scenarios under which this annotation should be shared and many others in which it shouldn’t.
Brush 2004). It is relatively rare to encounter an annotation that is unambiguously meaningful to someone else without further explanation. Let’s refer to this study data to see what else we can learn about the relationship between personal and shared annotations. Remember that the students’ personal annotations might well reflect their particular circumstances. In other words, they marked on the papers because they knew they’d be responsible for summarizing and discussing them online. If we examine the annotations that were in the proper form (anchor+body) to use in WebAnn, we see that only about one-third were used in the online discussions (fewer than 20% of the unanchored marginalia and fewer than 5% of the anchor-only annotations found their way into the online discussions). Reuse of the anchor-only annotations was higher (but not much higher) in the summary-writing task, still under 20%. What we see isn’t surprising: both the body of the annotations and their anchors need to be revised before the annotations are suitable for use in a public discussion. These results cast some doubt on the idea that readers will simply make their personal annotations public; rather, they will need to revise them before they share them. This is especially true of handwritten annotations one would make with a pen–tablet user interface. If we assume that annotation habits carry over from paper to freeform digital ink (Marshall et al. 1999), then we can also assume that more than a handwriting recognition capability and “publish this now” button will be required to make personal annotations publicly intelligible, even among immediate members of a community or work group.
reading as a social activity 85
FIGURE 4.6: A personal annotation is transformed into a sharable comment. (a) A student’s personal annotation on a published article. The annotation, which is anchored to a sentence in the article, reacts saying, “That may be true, but many of the early systems weren’t highly usable and usability is really taking hold.” (b) The corresponding annotation the student has shared with her classmates. It is anchored to the same sentence, and it says, “many of the early systems weren’t highly usable, yet we’re correcting that problem. VSD doesn’t examine how to fix past wrongs.”
That said, there may be other ways to share annotations. People may value experts’ annotations (Marshall 1997; Wolfe 2000), and there are systems that have been developed to support the sharing of annotations among the members of a workgroup or small team. Can we help people shop for annotations? Can we rifle through dozens and dozens of online annotations that represent the perspectives of many readers? Perhaps. But it may also be possible to aggregate annotations and use them that way.
4.2.3 Aggregating Annotations: The Wisdom of Crowds Imagine that a class uses a particular textbook. Each student marks it in his or her own way. One student uses a pink highlighter. Another student uses four different colors: pink is for term definition, yellow means important, green means “I don’t understand this,” and blue means “this is something I picked out when I re-read the book.” A third student underlines while he rides the bus to campus; his underlines are messy and approximate. Yet another student writes careful notes in the margin, some while she’s reading the textbook the first time, and others while she’s listening in class. A fifth student rarely annotates at all. A sixth marks in only a few places: where he stopped reading and at the beginning and end of the assignment. This variation is fairly typical (Shipman et al. 2003). What can we say about all these annotations? It seems safe to say that they represent a blip in the reader’s interest, or perhaps even that the reader thinks the text under his pen will be useful someday—on the test, in class—if not today.
86 reading and writing the electronic book
Readers seldom remember making most of these marks, so it seems sensible to say that if they’re important, they’re not very important, nor very reliable. Do readers’ annotations overlap more frequently than they would by chance? If one reader underlines 10% of the text, and another reader highlights 20% of the text, probability dictates that the readers would select the same text 2% of the time. Our initial calculations demonstrated that the text selections coincided a significantly greater proportion of the time than probability would predict (Marshall 1998). Furthermore, they overlap in interesting ways. We might surmise that the overlap occurs at topic sentences and pull-quotes, the text the writer, editor, or publisher thought was important. But this is not the case: the annotators converge on the text they think is important. The text that is marked the most frequently may be hidden in a long paragraph or toward the end of a section. These overlapping annotations may call attention to important points that would be otherwise buried within dense material. It seems as though there is a legitimate wisdom-of-crowds effect (Surowiecki 2004). Let’s look at a specific example, the annotations that the six reading group members made on a page of the technical article that they were discussing. Because we observed their meeting and interviewed them about their reading practices, we know that they were reading with the same purpose in mind. Their backgrounds are similar, although their specialties are different (that is, several specialized in signal-processing; several others specialized in HCI). Hence, the overlap in their annotations will probably be interesting. Figure 4.7 shows an example of a place on the page where their marks converged. Reader 1 did not annotate anything in this region. In fact, he seldom annotates when he reads; his entire reprint is devoid of marks. Reader 5 marks sparsely and only made one mark very early on in the paper. Hence, we wouldn’t expect either reading group member to have marked in the target region. Readers 2, 3, 4, and 6 made varying marks in the region. Without interpreting the specific marks, all we can say is that the phrase time-constrained clustering (and its immediate context) is of interest to the group. The set of equations themselves is of slightly less interest, perhaps because they are, to some extent, obvious to the readers. Observation of the group’s meeting demonstrates that this is the case. Different techniques may be used to identify overlapping anchors. Fingerprinting (Hong et al. 2008) and robust anchor specification (Brush et al. 2001) are two such techniques; other methods of identifying overlapping annotation anchors might rely on identical underlying texts (Schilit, Golovchinsky, & Price 1998a). There are many things we might do once we have identified these points of apparent communal interest: we can use them to bolster our characterization of what the paper is about (after
One member of the group led the discussion; this may mean that his or her marks reflected this extra responsibility.
reading as a social activity 87
FIGURE 4.7: A wisdom of crowds approach to aggregating reader annotations. (a) Reader 1’s annotations (none). (b) Reader 2’s annotations. (c) Reader 3’s annotations. (d) Reader 4’s annotations. (e) Reader 5’s annotations (none). (f ) Reader 6’s annotations.
all, we have identified terms that contribute substantially to the article’s relevance); we might use the identified text as a skimming aid (we can make this portion of the paper stand out to facilitate skimming); we can use the text we have identified in summarization or in visualizing the contents of the paper. In other words, that everyone singled out this text should be of consequence. Figure 4.8 shows how the results of this analysis might be used to support skimming (Marshall et al. 2001). It is easy to imagine how a researcher who joined the group late might look at the selected text to quickly catch up on past readings. This wisdom-of-crowds effect may be more pronounced as the number of annotators is scaled to larger communities. As a greater number of eBooks become available and as more people read and annotate on the screen, it becomes practical to explore whether this technique yields any
88 reading and writing the electronic book
FIGURE 4.8: Showing the consensus of many readers’ annotations to facilitate skimming. concrete benefits. It is presented here as an example of how the artifacts of reading may be used in new ways that were not possible in the print world.
4.2.4 Sharing Encountered Information Over 40% of the examples we saw in our clipping study had been clipped with the explicit purpose of sharing them with someone else (Marshall & Bly 2004); every participant in the study had on hand at least one shared clipping. For our purposes in this chapter, we can think of this practice as sharing encountered information. But sharing encountered information is something of a misnomer. In the study, it became apparent that sometimes the act of sharing was more important than the specific content that had been shared (although the content was always at least somewhat relevant to an interest or a shared value, e.g., a shared sense of humor). The content itself was more likely to be important if the clipping was saved for oneself rather than shared with someone else. Why, then, do people share encountered information if not for the information itself ? This kind of material was shared for many reasons: to keep in touch or develop rapport; for mutual awareness; to educate the recipient; to strengthen social ties (usually by demonstrating knowledge of the recipient’s interests); or for some combination of these reasons. Of course, it is important to remember that the recipient may give an entirely different account of why they received something than the sender would give for why they sent it. The recipient may say, “he sent this to
reading as a social activity 89
me to show me that he’s thinking about me,” while the sender is thinking, “this might change her mind about the war in Afghanistan.” Certainly the perceived utility of the shared material differs: frequently the sender thinks he or she is sending something of greater value than the recipient finds it. Sharing a clipping to keep in touch is not unlike sending a greeting card. For example, a father sent his high-school-aged daughter newspaper clippings that covered current events (sometimes as often as several times a day); of this practice, his daughter said, “Sending me the article is like a little note . . . And it’s nice to feel like someone’s thinking about you. It’s his way of saying ‘hello’ during the day.” Needless to say, she usually read these articles on the screen and then deleted them from her email. In general, the content of this type of material, while it necessarily fell within the scope of both parties’ interests, was not regarded as valuable and therefore was seldom intentionally kept. Sharing for mutual awareness is a common practice in the workplace. Sometimes a company’s management will distribute an article that mentions the company to all of its employees. A financial consultant in the clipping study reported that he reads the Wall Street Journal and The New York Times several times a week to see if his company is mentioned; he described how the articles are distributed among employees: “Those are two papers our company shows up in a lot, so usually we get an email that says, ‘hey. We’re listed today. Check out the article.’ ” Again, these articles may not be regarded as having any persistent value, unless they are further shared with clients (usually by sales people who have frequent contact with customers); in these cases, the clippings may also be shared to educate the recipient. Sharing clippings for this reason—to educate the recipient—is also common. One study participant had a preadolescent son who had been diagnosed with Asperger’s syndrome, a form of high-functioning autism. She had photocopied a “really good” print article from Time magazine and kept copies in a file for exactly this purpose. She told us, “I mailed this to so many people. Because it was very, very good.” She described attaching a note to one of the copies—“You need to read this”—and sending it to her son’s teacher. Material may be clipped and shared with the intent of strengthening social ties. Content of this sort usually demonstrates not only a familiarity with the recipient’s interests but also the fact that the recipient is on the sender’s mind. In such cases, the sender may be well aware that the actual information in the clipping won’t be that useful to the recipient; it just has to be sufficiently on the mark to accomplish its goal. Sharing clippings does not seem to be a practice limited to a distinct category of person or distinct social type—Pettigrew, Durrance, and Unruh’s (2002) information brokers—but rather
As we have noted elsewhere, it is often easier to keep these things than it is to methodically and consistently cull them.
90 reading and writing the electronic book
fulfills the somewhat overlapping functions discussed above. All of our participants not only shared information they’d encountered but also moved fluidly from acting as a giver to being a recipient. Findings suggest that the practice transcends age and specific context: it occurs both at home and in the workplace; it occurs when people read on the screen and when they read material in print. As such, it seems to suggest some specific eBook functionality. In a similar study, Rioux (2000) discovered that when people shared information they encountered on the Web, they had specific concerns. It is instructive to examine these concerns before we rush to develop new functionality for our eBook platforms. First, people often did not have the intended recipient’s contact information at hand. If the material included advertising, they wondered if the advertising would be sent along with the clipping. What would the clipping look like—would it mirror what they had seen on the screen themselves? Would there be long-term threats to anyone’s privacy as publishers tracked their interests and their friends’ interests? Would the clipping look like spam? Would it be perceived as impersonal? These findings underscore the importance of materiality: when people share published material, they want to know what it looks like and how it is excerpted. Consider, for example, clipping a photo from a newspaper article and sending it to a friend. Is there a difference between sending a photo and sending the entire article the photo is from? Have the photo credits been properly sent? Will the photo look as good on the recipient’s screen as it does on your own? Likewise, it seems to be important to have control over the mode of sharing. Consider the subtle differences among three modes of sharing a paragraph from an eBook: (1) embedding a clickable link to the paragraph in an email message; (2) excerpting the paragraph and putting it into the body of the message; or (3) sending the entire eBook as an attachment and using a bookmark to take the recipient to the desired page. In the first case, the recipient has to seek out the eBook, but has a message explaining the sender’s intent; in the second case, the recipient has the excerpt at hand, but no further context—the eBook’s authority is not conveyed; in the third case, the recipient has the entire eBook—and thus a substantial amount of context—but the extra material might be overwhelming. Naturally, it is up to the sender to determine her own intent and how to best convey that intent. In practice, people do consider these questions. A museum designer gave her own account of how she makes this sort of decision: My plan is to actually give it [a hardcopy of an article from Nature Online] to him [a project manager] and talk to him about it, rather than just put it in his in-basket because he’d kind
For the sake of convenience, let us say that the eBook is part of an online collection, for example, a shared library or an online resource like Google books.
reading as a social activity 91
of wonder where it came from or why he was getting it. And I’d rather say, “hey, I saw this online and it’s pretty interesting. Check it out” (Marshall & Bly 2004).
4.2.5 Information Brokering Studies of how people share information have led to the idea of information brokering. Information brokers are people who routinely mediate access to online information. Current research on information brokering in a community network setting shows that the practice “fosters social cohesion” as people search for materials on “behalf of another person (e.g., relative, friend) and not always at that person’s behest” (Pettigrew, Durrance, & Unruh 2002). Research on Internet use has demonstrated that people find ways to share their expertise on topics and help more junior users as a way of building social capital and strengthening social ties within a community (but that very heavy Internet use may also weaken community commitment; Wellman et al. 2001).10 Hence, the jury is still out on the overall effect of finding materials on the Internet relevant to the interests of others. Our own past work has found that it may play an important—if supplemental—part in building social capital and strengthening social ties within a community (Marshall & Bly 2004). It is important to note that in our study, we focused on encountered information which may not be driven by information needs; that is, the information that is shared does not meet any stated need on the part of the recipient. Information brokering has two possible implications for eBooks: (1) eBooks, or portions of eBooks, may be shared in response to a stated need; and (2) information that is encountered may be shared for any of the various purposes we discussed earlier in the chapter. In the latter case, an information needs framework is not the appropriate one for thinking about sharing; rather, we need to think about the effect the sharing has on social capital independent from the information the material contains.
4.2.6 Sharing and Recommending Books Recommendation is an important part of the social life of eBooks; it can take many forms and can rely on different sources of evidence. Books may be recommended explicitly, either from person to person, in much the same way as we have seen clippings recommended—that is, one person tells another, “Here’s a book you might like”—or to a wide audience, as is true on a blog or Web site. Books also may be recommended implicitly, via citation or quotation; that is, by citing a book or 10
This hypothesis that Internet-based interaction can strengthen, as well as weaken, a community’s social ties is one that developed as a response to Putnam’s famous “bowling alone” finding that public community participation has been on the decline in recent decades (Putnam 1995).
92 reading and writing the electronic book
quoting its author, we are saying, “go take a look at this work; it’s important.” Although not all citations are positive (i.e., you might cite an author’s work because you disagree with her), in some ways, just telling someone to look at a book for whatever reason amounts to a recommendation. It is much like the old show business adage that bad publicity is better than no publicity at all. Some recommendation is wholly content-based. That is, a recommender system might take some evidence of your interest—say, the text you’ve annotated or the last book you’ve read—and find you a book that has comparable content, either through text analysis or through a combination of text analysis and explicit citation (Schilit, Golovchinsky, & Price 1998a; Woodruff et al. 2000). Although this type of recommendation is not without a social component (both texts were written by authors interested in the same topic, and by interacting with one of the books, you were also showing interest in this topic), the recommendation is, by and large, implicit. Similarly, books might be connected with one another through shared quotations, because we can assume that sharing quotations means that works share key concepts (Schilit et al. 2008). Although the social component here is stronger (i.e., we are certain that the two authors looked at the same sources), the link between the two books is still inferred. Content-based recommendation has not been as successful as explicit recommendation, because social recommendation is often a more nuanced thing: a recommendation on a site like Amazon won’t just tell you that another reader liked a particular book; it will also tell you why the reader liked it and why you might like it too, beyond the topic simply being of interest. The reviewer might comment on less tangible aspects of the work such as mood, style, political leanings, and so on. And you might take the reviewer’s characteristics into account, including the reviewer’s reputation and how well the review is written. Recommendation may also be indirect—a topic is on the tip of everyone’s tongues for a day and a reader consults his favorite news sources to find out about breaking news. For example, a Times News Reader study participant told us: It was a few days ago. Michael J. Fox was accused of [faking]. So I saw that, I was on AOL for a moment. So I just got the quick headline and I got the photo. And so I thought this was a perfect use for [the TNR]. So I came here . . . (Marshall 2007). The reader knows what he wants to read about, how much he wants to read, and where he wants to go to read about it. It is no wonder that recommenders based solely on content have not been overwhelmingly popular.11 11
Note, however, that news aggregators have been very popular; it is telling that their popularity relies on their timeliness, not on the availability of specific topical content.
reading as a social activity 93
Recommenders have historically relied on assigned ratings, coupled with each user’s reading behavior (Goldberg et al. 1992; Konstan et al. 1997). When readers read an article, they assign a numerical rating; their own interests are defined by what they read. This partitioning of users into communities of interest allows the recommendations to be tailored, to some degree, to individual interests and tastes. Other recommenders are based on reader actions; for example, most newspapers track which article have been sent to other readers, which again takes a “wisdom of crowds” approach to recommending: surely the most popular articles are the ones you also want to read.12 The last few years have seen the rise of several important social networking sites for recommending books, eBooks, and other types of electronic publications. LibraryThing13 allows its users to catalog their own collections and, through these personal catalogs, meet other people with similar libraries or reading interests; the Web application bootstraps the cataloguing process by providing its users with records from multiple external sources such as the Library of Congress and Amazon. In essence, people are connected with other people through what they read. Zotero14 began by providing users with online bibliographic tools, not unlike EndNote and other reference management systems. Like LibraryThing, Zotero has grown in a collaborative direction, again pursuing the idea of connecting scholars through the references they use (but also allowing them to share reference materials through identifying common interests). Bookmarking services like delicious15 (formerly del.icio.us) are also oriented to tagging, sharing, and managing Web-based material. In addition to supporting recommendation, Delicious uses bookmark popularity to recommend sites associated with specific topics. Over the past decade, recommenders’ strategies have grown increasingly sophisticated leapfrogging beyond explicit the ratings and reviews that they originally relied on. Recommenders now use multiple sources of evidence—both implicit and explicit—to characterize the popularity and topicality of a material. Readers are not only connected with books; they are also connected with other readers through what they choose to read and what they cite as authoritative. • • • •
12
In the most general case of citation implying interest, we enter into the familiar territory of Web search, in which inbound links are taken as evidence of topical quality, a technique first suggested by Kleinberg (1998), but then popularized in the Google search engine. Obviously, taking on this broader topic is biting off more than we can chew. 13
http://www.librarything.com.
14
http://www.zotero.org/.
15
http://delicious.com/.
95
chapter 5
Studying Reading This chapter is about studying reading and studying electronic books in use. Why devote a whole chapter to studies? There are many excellent research methods resources that describe how to design and conduct studies, including texts on quantitative research design (Creswell 2009), qualitative methods (Marshall & Rossman 2006), qualitative studies in service of technology design (Sharp, Rogers, & Preece 2007), basic ethnography (Fetterman 1998), involving users in design (Brun-Cottan & Wall 1995)—in short, for every method imaginable for learning about human activity and human interaction with technology. Yet studying reading poses particular challenges. Reading is so essentially invisible and so commonplace that it is difficult to actually “see” it and to untangle it from the many assumptions that we make about it. Tzvetan Todorov, quoted by Nicholas Howe in Jonathan Boyarin’s compilation, the Ethnography of Reading, said of reading: Nothing is more commonplace than the reading experience, and yet nothing is more unknown. Reading is such a matter of course that at first glance it seems there is nothing to say about it (Howe 1993). Although there is much to be learned in the lab—how our eyes move across the page when we’re reading for comprehension, how typography affects our mood, how spatial memory works, and many other details of how we read—it is important to acknowledge all that we leave behind when we move into the lab. Thus, the main focus of this chapter will be on field studies, on finding out how people read in the wild. Watching people read isn’t easy and, in fact, has made for some of the most difficult field studies I’ve undertaken. Observing people read is by nature creepy. Once when I was involved in
I must hasten to add that this is the other Catherine Marshall, Catherine R. Marshall. Our identities are sometimes confused.
The reader should note that I have just provided examples of each kind of methods reference. There are many references in each area written in different styles with varying goals (practical how-to guides, more theoretical discussions of the rationale behind methods, and so on). Here’s where you will need to make some meta-decisions about what is important to you.
96 reading and writing the electronic book
the design of an electronic magazine product, I decided I’d watch people read on the airplane as a precursor to a more formal observational study. I just wanted to get the lay of the land, so to speak. So, during a short flight from San Francisco to Seattle, I watched a businessman across the aisle browse the golfing magazine he’d brought on board with him. I thought I was being subtle, allowing myself to cast sidelong glances at him across the aisle. He left his seat only once toward the end of the flight to use the lavatory, so I had over an hour to watch him and take notes. For some reason, he didn’t continue reading when he returned to his seat after his trip to the lavatory, so I focused my attention elsewhere. After the flight landed, when we were gathering up our belongings to get off the plane, he confronted me. “You stole my magazine,” he said. He had an accent. I didn’t know what to say. “No. I didn’t.” I finally said. “Open your briefcase and let me look,” he said. I was embarrassed and offended. How dare he! But after all, I’d been surreptitiously observing him for the duration of the flight. He must have sensed something was amiss. I sheepishly opened my briefcase so he could see for himself that I did not have his magazine. Even after he looked, I could tell he was unconvinced. Hence the hazards of watching someone read: there’s something creepy about it. Creepy and important. Simply asking people how they read and interact with their reading materials is not only hard—these are unselfconscious activities and people don’t always know what they do—but also misleading. People try their best to answer the interview questions and, in so doing, may lead the interviewer astray. When Sara Bly and I studied clipping, if we had asked people how they clipped at the outset of the study, we would’ve missed all of our participants, because they each denied that they did it at all. Furthermore, if we assume the rationale for these activities is self-evident, or that people have good insight into why they do some of the things that they do and we might miss valuable results. For example, if you ask a researcher why he writes in the margins of the articles he is reading, he might tell you, “Some people