Human-Computer Interaction. Interacting in Various Application Domains: 13th International Conference, HCI International 2009, San Diego, CA, USA, ... Applications, incl. Internet Web, and HCI)
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany
5613
Julie A. Jacko (Ed.)
Human-Computer Interaction Interacting in Various Application Domains 13th International Conference, HCI International 2009 San Diego, CA, USA, July 19-24, 2009 Proceedings, Part IV
13
Volume Editor Julie A. Jacko University of Minnesota Institute of Health Informatics MMC 912, 420 Delaware Street S.E., Minneapolis, MN 55455, USA E-mail: [email protected]
Library of Congress Control Number: 2009929048 CR Subject Classification (1998): H.5, I.3, I.7.5, I.5, I.2.10 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13
0302-9743 3-642-02582-X Springer Berlin Heidelberg New York 978-3-642-02582-2 Springer Berlin Heidelberg New York
The 13th International Conference on Human–Computer Interaction, HCI International 2009, was held in San Diego, California, USA, July 19–24, 2009, jointly with the Symposium on Human Interface (Japan) 2009, the 8th International Conference on Engineering Psychology and Cognitive Ergonomics, the 5th International Conference on Universal Access in Human–Computer Interaction, the Third International Conference on Virtual and Mixed Reality, the Third International Conference on Internationalization, Design and Global Development, the Third International Conference on Online Communities and Social Computing, the 5th International Conference on Augmented Cognition, the Second International Conference on Digital Human Modeling, and the First International Conference on Human Centered Design. A total of 4,348 individuals from academia, research institutes, industry and governmental agencies from 73 countries submitted contributions, and 1,397 papers that were judged to be of high scientific quality were included in the program. These papers address the latest research and development efforts and highlight the human aspects of the design and use of computing systems. The papers accepted for presentation thoroughly cover the entire field of human–computer interaction, addressing major advances in knowledge and effective use of computers in a variety of application areas. This volume, edited by Julie A. Jacko, contains papers in the thematic area of Human-Computer Interaction, addressing the following major topics: • • • •
eLearning and Education Games and Entertainment Work, Collaboration and Business Advanced Applications
The remaining volumes of the HCI International 2009 proceedings are: • • • • • •
Volume 1, LNCS 5610, Human–Computer Interaction––New Trends (Part I), edited by Julie A. Jacko Volume 2, LNCS 5611, Human–Computer Interaction––Novel Interaction Methods and Techniques (Part II), edited by Julie A. Jacko Volume 3, LNCS 5612, Human–Computer Interaction––Ambient, Ubiquitous and Intelligent Interaction (Part III), edited by Julie A. Jacko Volume 5, LNCS 5614, Universal Access in Human–Computer Interaction––Addressing Diversity (Part I), edited by Constantine Stephanidis Volume 6, LNCS 5615, Universal Access in Human–Computer Interaction––Intelligent and Ubiquitous Interaction Environments (Part II), edited by Constantine Stephanidis Volume 7, LNCS 5616, Universal Access in Human–Computer Interaction––Applications and Services (Part III), edited by Constantine Stephanidis
VI
Foreword
• • • • • • • • • •
Volume 8, LNCS 5617, Human Interface and the Management of Information––Designing Information Environments (Part I), edited by Michael J. Smith and Gavriel Salvendy Volume 9, LNCS 5618, Human Interface and the Management of Information––Information and Interaction (Part II), edited by Gavriel Salvendy and Michael J. Smith Volume 10, LNCS 5619, Human Centered Design, edited by Masaaki Kurosu Volume 11, LNCS 5620, Digital Human Modeling, edited by Vincent G. Duffy Volume 12, LNCS 5621, Online Communities and Social Computing, edited by A. Ant Ozok and Panayiotis Zaphiris Volume 13, LNCS 5622, Virtual and Mixed Reality, edited by Randall Shumaker Volume 14, LNCS 5623, Internationalization, Design and Global Development, edited by Nuray Aykin Volume 15, LNCS 5624, Ergonomics and Health Aspects of Work with Computers, edited by Ben-Tzion Karsh Volume 16, LNAI 5638, The Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, edited by Dylan Schmorrow, Ivy Estabrooke and Marc Grootjen Volume 17, LNAI 5639, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris
I would like to thank the Program Chairs and the members of the Program Boards of all thematic areas, listed below, for their contribution to the highest scientific quality and the overall success of HCI International 2009.
Ergonomics and Health Aspects of Work with Computers Program Chair: Ben-Tzion Karsh Arne Aarås, Norway Pascale Carayon, USA Barbara G.F. Cohen, USA Wolfgang Friesdorf, Germany John Gosbee, USA Martin Helander, Singapore Ed Israelski, USA Waldemar Karwowski, USA Peter Kern, Germany Danuta Koradecka, Poland Kari Lindström, Finland
Holger Luczak, Germany Aura C. Matias, Philippines Kyung (Ken) Park, Korea Michelle M. Robertson, USA Michelle L. Rogers, USA Steven L. Sauter, USA Dominique L. Scapin, France Naomi Swanson, USA Peter Vink, The Netherlands John Wilson, UK Teresa Zayas-Cabán, USA
Foreword
Human Interface and the Management of Information Program Chair: Michael J. Smith Gunilla Bradley, Sweden Hans-Jörg Bullinger, Germany Alan Chan, Hong Kong Klaus-Peter Fähnrich, Germany Michitaka Hirose, Japan Jhilmil Jain, USA Yasufumi Kume, Japan Mark Lehto, USA Fiona Fui-Hoon Nah, USA Shogo Nishida, Japan Robert Proctor, USA Youngho Rhee, Korea
Anxo Cereijo Roibás, UK Katsunori Shimohara, Japan Dieter Spath, Germany Tsutomu Tabe, Japan Alvaro D. Taveira, USA Kim-Phuong L. Vu, USA Tomio Watanabe, Japan Sakae Yamamoto, Japan Hidekazu Yoshikawa, Japan Li Zheng, P.R. China Bernhard Zimolong, Germany
Human–Computer Interaction Program Chair: Julie A. Jacko Sebastiano Bagnara, Italy Sherry Y. Chen, UK Marvin J. Dainoff, USA Jianming Dong, USA John Eklund, Australia Xiaowen Fang, USA Ayse Gurses, USA Vicki L. Hanson, UK Sheue-Ling Hwang, Taiwan Wonil Hwang, Korea Yong Gu Ji, Korea Steven Landry, USA
Gitte Lindgaard, Canada Chen Ling, USA Yan Liu, USA Chang S. Nam, USA Celestine A. Ntuen, USA Philippe Palanque, France P.L. Patrick Rau, P.R. China Ling Rothrock, USA Guangfeng Song, USA Steffen Staab, Germany Wan Chul Yoon, Korea Wenli Zhu, P.R. China
Engineering Psychology and Cognitive Ergonomics Program Chair: Don Harris Guy A. Boy, USA John Huddlestone, UK Kenji Itoh, Japan Hung-Sying Jing, Taiwan Ron Laughery, USA Wen-Chin Li, Taiwan James T. Luxhøj, USA
Nicolas Marmaras, Greece Sundaram Narayanan, USA Mark A. Neerincx, The Netherlands Jan M. Noyes, UK Kjell Ohlsson, Sweden Axel Schulte, Germany Sarah C. Sharples, UK
VII
VIII
Foreword
Neville A. Stanton, UK Xianghong Sun, P.R. China Andrew Thatcher, South Africa
Matthew J.W. Thomas, Australia Mark Young, UK
Universal Access in Human–Computer Interaction Program Chair: Constantine Stephanidis Julio Abascal, Spain Ray Adams, UK Elisabeth André, Germany Margherita Antona, Greece Chieko Asakawa, Japan Christian Bühler, Germany Noelle Carbonell, France Jerzy Charytonowicz, Poland Pier Luigi Emiliani, Italy Michael Fairhurst, UK Dimitris Grammenos, Greece Andreas Holzinger, Austria Arthur I. Karshmer, USA Simeon Keates, Denmark Georgios Kouroupetroglou, Greece Sri Kurniawan, USA
Patrick M. Langdon, UK Seongil Lee, Korea Zhengjie Liu, P.R. China Klaus Miesenberger, Austria Helen Petrie, UK Michael Pieper, Germany Anthony Savidis, Greece Andrew Sears, USA Christian Stary, Austria Hirotada Ueda, Japan Jean Vanderdonckt, Belgium Gregg C. Vanderheiden, USA Gerhard Weber, Germany Harald Weber, Germany Toshiki Yamaoka, Japan Panayiotis Zaphiris, UK
Virtual and Mixed Reality Program Chair: Randall Shumaker Pat Banerjee, USA Mark Billinghurst, New Zealand Charles E. Hughes, USA David Kaber, USA Hirokazu Kato, Japan Robert S. Kennedy, USA Young J. Kim, Korea Ben Lawson, USA
Gordon M. Mair, UK Miguel A. Otaduy, Switzerland David Pratt, UK Albert “Skip” Rizzo, USA Lawrence Rosenblum, USA Dieter Schmalstieg, Austria Dylan Schmorrow, USA Mark Wiederhold, USA
Internationalization, Design and Global Development Program Chair: Nuray Aykin Michael L. Best, USA Ram Bishu, USA Alan Chan, Hong Kong Andy M. Dearden, UK
Susan M. Dray, USA Vanessa Evers, The Netherlands Paul Fu, USA Emilie Gould, USA
Foreword
Sung H. Han, Korea Veikko Ikonen, Finland Esin Kiris, USA Masaaki Kurosu, Japan Apala Lahiri Chavan, USA James R. Lewis, USA Ann Light, UK James J.W. Lin, USA Rungtai Lin, Taiwan Zhengjie Liu, P.R. China Aaron Marcus, USA Allen E. Milewski, USA
Elizabeth D. Mynatt, USA Oguzhan Ozcan, Turkey Girish Prabhu, India Kerstin Röse, Germany Eunice Ratna Sari, Indonesia Supriya Singh, Australia Christian Sturm, Spain Adi Tedjasaputra, Singapore Kentaro Toyama, India Alvin W. Yeo, Malaysia Chen Zhao, P.R. China Wei Zhou, P.R. China
Online Communities and Social Computing Program Chairs: A. Ant Ozok, Panayiotis Zaphiris Chadia N. Abras, USA Chee Siang Ang, UK Amy Bruckman, USA Peter Day, UK Fiorella De Cindio, Italy Michael Gurstein, Canada Tom Horan, USA Anita Komlodi, USA Piet A.M. Kommers, The Netherlands Jonathan Lazar, USA Stefanie Lindstaedt, Austria
Gabriele Meiselwitz, USA Hideyuki Nakanishi, Japan Anthony F. Norcio, USA Jennifer Preece, USA Elaine M. Raybourn, USA Douglas Schuler, USA Gilson Schwartz, Brazil Sergei Stafeev, Russia Charalambos Vrasidas, Cyprus Cheng-Yen Wang, Taiwan
Augmented Cognition Program Chair: Dylan D. Schmorrow Andy Bellenkes, USA Andrew Belyavin, UK Joseph Cohn, USA Martha E. Crosby, USA Tjerk de Greef, The Netherlands Blair Dickson, UK Traci Downs, USA Julie Drexler, USA Ivy Estabrooke, USA Cali Fidopiastis, USA Chris Forsythe, USA Wai Tat Fu, USA Henry Girolamo, USA
Marc Grootjen, The Netherlands Taro Kanno, Japan Wilhelm E. Kincses, Germany David Kobus, USA Santosh Mathan, USA Rob Matthews, Australia Dennis McBride, USA Robert McCann, USA Jeff Morrison, USA Eric Muth, USA Mark A. Neerincx, The Netherlands Denise Nicholson, USA Glenn Osga, USA
IX
X
Foreword
Dennis Proffitt, USA Leah Reeves, USA Mike Russo, USA Kay Stanney, USA Roy Stripling, USA Mike Swetnam, USA Rob Taylor, UK
Maria L.Thomas, USA Peter-Paul van Maanen, The Netherlands Karl van Orden, USA Roman Vilimek, Germany Glenn Wilson, USA Thorsten Zander, Germany
Digital Human Modeling Program Chair: Vincent G. Duffy Karim Abdel-Malek, USA Thomas J. Armstrong, USA Norm Badler, USA Kathryn Cormican, Ireland Afzal Godil, USA Ravindra Goonetilleke, Hong Kong Anand Gramopadhye, USA Sung H. Han, Korea Lars Hanson, Sweden Pheng Ann Heng, Hong Kong Tianzi Jiang, P.R. China
Kang Li, USA Zhizhong Li, P.R. China Timo J. Määttä, Finland Woojin Park, USA Matthew Parkinson, USA Jim Potvin, Canada Rajesh Subramanian, USA Xuguang Wang, France John F. Wiechel, USA Jingzhou (James) Yang, USA Xiu-gan Yuan, P.R. China
Human Centered Design Program Chair: Masaaki Kurosu Gerhard Fischer, USA Tom Gross, Germany Naotake Hirasawa, Japan Yasuhiro Horibe, Japan Minna Isomursu, Finland Mitsuhiko Karashima, Japan Tadashi Kobayashi, Japan
Kun-Pyo Lee, Korea Loïc Martínez-Normand, Spain Dominique L. Scapin, France Haruhiko Urokohara, Japan Gerrit C. van der Veer, The Netherlands Kazuhiko Yamazaki, Japan
In addition to the members of the Program Boards above, I also wish to thank the following volunteer external reviewers: Gavin Lew from the USA, Daniel Su from the UK, and Ilia Adami, Ioannis Basdekis, Yannis Georgalis, Panagiotis Karampelas, Iosif Klironomos, Alexandros Mourouzis, and Stavroula Ntoa from Greece. This conference could not have been possible without the continuous support and advice of the Conference Scientific Advisor, Prof. Gavriel Salvendy, as well as the dedicated work and outstanding efforts of the Communications Chair and Editor of HCI International News, Abbas Moallem.
Foreword
XI
I would also like to thank for their contribution toward the organization of the HCI International 2009 conference the members of the Human–Computer Interaction Laboratory of ICS-FORTH, and in particular Margherita Antona, George Paparoulis, Maria Pitsoulaki, Stavroula Ntoa, and Maria Bouhli. Constantine Stephanidis
HCI International 2011
The 14th International Conference on Human–Computer Interaction, HCI International 2011, will be held jointly with the affiliated conferences in the summer of 2011. It will cover a broad spectrum of themes related to human–computer interaction, including theoretical issues, methods, tools, processes and case studies in HCI design, as well as novel interaction techniques, interfaces and applications. The proceedings will be published by Springer. More information about the topics, as well as the venue and dates of the conference, will be announced through the HCI International Conference series website: http://www.hci-international.org/
General Chair Professor Constantine Stephanidis University of Crete and ICS-FORTH Heraklion, Crete, Greece Email: [email protected]
Table of Contents
Part I: eLearning and Education Arab Children’s Reading Preference for Different Online Fonts . . . . . . . . . Asmaa Alsumait, Asma Al-Osaimi, and Hadlaa AlFedaghi Adaptation Decisions and Profiles Exchange among Open Learning Management Systems Based on Agent Negotiations and Machine Learning Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Silvia Baldiris, Ram´ on Fabregat, Carolina Mej´ıa, and Sergio G´ omez Accessing e-Learning Systems via Screen Reader: An Example . . . . . . . . . Maria Claudia Buzzi, Marina Buzzi, and Barbara Leporini
Optimal Affective Conditions for Subconscious Learning in a 3D Intelligent Tutoring System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pierre Chalfoun and Claude Frasson
39
Computer-Based Learning to Improve Breast Cancer Detection Skills . . . Yan Chen, Alastair Gale, Hazel Scott, Andrew Evans, and Jonathan James Virtual Classroom and Communicability: Empathy and Interaction for All . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Francisco V. Cipolla Ficarra
49
58
Communicability for Virtual Learning: Evaluation . . . . . . . . . . . . . . . . . . . . Francisco V. Cipolla-Ficarra, Miguel Cipolla-Ficarra, and Pablo M. Vera
68
Attention and Motivation in Hypermedia Systems . . . . . . . . . . . . . . . . . . . . Francisco V. Cipolla Ficarra and Miguel Cipolla-Ficarra
The Application of the Flexilevel Approach for the Assessment of Computer Science Undergraduates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mariana Lilley and Andrew Pyper
140
Development of Ubiquitous On-Demand Study Support Environment for Nursing Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yukie Majima, Yumiko Nakamura, Yasuko Maekawa, and Yoichiro So
149
The Effects of Prior Knowledge on the Use of Adaptive Hypermedia Learning Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Freddy Mampadi, Sherry Chen, and Gheorghita Ghinea
156
Supporting Learners in Adaptive Learning Environments through the Enhancement of the Student Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luca Mazzola and Riccardo Mazza
166
The Concept of IMPRESSION: An Interactive Instruction System and Its Practice for Real-Time Distance Lessons between U.S. and Japan . . . Takashi Mitsuishi, Fumiko Konno, Yuki Higuchi, and Kentaro Go
176
Improving Children’s Writing Ability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Joana Pereira, Lu´ıs Carri¸co, and Carlos Duarte From Paper to Module – An Integrated Environment for Generating SCORM Compliant Moodle Courses Out of Text and Multimedia Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hans-Martin Pohl, Benedikt Deicke, and Jan-Torsten Milde Development of a Simulator of Abacus: Ancient Analog Calculator on a Mobile Phone as a Teaching Material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kenta Saito, Yuki Makita, Vu Quang, and Hitoshi Sasaki
186
196
204
Table of Contents
A Proposal for a Framework for an e-Alumni Program Using SNS . . . . . . Hiroshi Sano Supporting End-User Development of Personalized Mobile Learning Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marco de S´ a and Lu´ıs Carri¸co
The Experience of Presence in 3D Web Environment: An Analysis of Korean Second Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SungBok Park, Ha Sung Hwang, and Myungil Choi
387
Influence of Real-World Ten-Pin Bowling Experience on Performance during First-Time Nintendo Wii Bowling Practice . . . . . . . . . . . . . . . . . . . . Kirsten A. Peters
396
Emotionally Adapted Games – An Example of a First Person Shooter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Timo Saari, Marko Turpeinen, Kai Kuikkaniemi, Ilkka Kosunen, and Niklas Ravaja DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tatsushi Takeuchi, Koichiro Watanabe, Tomoo Inoue, and Ken-ichi Okada
406
416
Part III: Work, Collaboration and Business New Health Information Systems (HIS) Quality-in-Use Model Based on the GQM Approach and HCI Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reem Al-Nanih, Hana Al-Nuaim, and Olga Ormandjieva
429
Table of Contents
An Information Visualization Approach to Hospital Shifts Scheduling . . . Carmelo Ardito, Paolo Buono, Maria F. Costabile, Rosa Lanzilotti, and Adalberto L. Simeone Designed to Fit: Challenges of Interaction Design for Clothes Fitting Room Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo Begole, Takashi Matsumoto, Wei Zhang, Nicholas Yee, Juan Liu, and Maurice Chu Usability for Poll Workers: A Voting System Usability Test Protocol . . . . Dana Chisnell, Karen Bachmann, Sharon Laskowski, and Svetlana Lowry
XIX
439
448
458
CAD and Communicability: A System That Improves the Human-Computer Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Francisco V. Cipolla Ficarra and Roc´ıo A. Rodr´ıguez
468
A Novel Visualization Tool for Evaluating Medication Side-Effects in Multi-drug Regimens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jon Duke, Anthony Faiola, and Hadi Kharrazi
478
Design of a Web Intervention to Change Youth Smoking Habits . . . . . . . . Kim Nee Goh, Yoke Yie Chen, Emy Elyanee Mustapha, Subarna Sivapalan, and Sharina Nordin
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work: The Case of IT Service Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eser Kandogan, Eben M. Haber, John H. Bailey, and Paul P. Maglio
504
Human Computer Interaction in Virtual Standardized Patient Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Patrick G. Kenny, Thomas D. Parsons, and Albert A. Rizzo
514
Towards Standardized Pen-Based Annotation of Breast Cancer Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Suzanne Kieffer, Annabelle Gouze, Ronald Moncarey, Christian Van Brussel, Jean-Fran¸cois De Wispelaere, Fran¸coise Kayser, and Benoˆıt Macq ImproV: A System for Improvisational Construction of Video Processing Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Atsutomo Kobayashi, Buntarou Shizuki, and Jiro Tanaka E-Assessment: A Suitable Alternative for Measuring Competences? . . . . . Martin Kr¨ oll
Arab Children’s Reading Preference for Different Online Fonts Asmaa Alsumait1, Asma Al-Osaimi2, and Hadlaa AlFedaghi2 1 Computer Engineering Dep., Kuwait University, Kuwait Regional Center For Development of Educational Software, Kuwait [email protected], {alosaimi,hadlaa}@redsoft.org 2
Abstract. E-learning education plays an important role in the educational process in the Arab region. There is more demand to provide Arab students with electronic resources for knowledge now than before. The readability of such electronic resources needs to be taken into consideration. Following design guidelines in the e-learning programs’ design process improves both the reading performance and satisfaction. However, English script design guidelines cannot be directly applied to Arabic script mainly because of difference in the letters occupation and writing direction. Thus, this paper aimed to build a set of design guidelines for Arabic e-learning programs designed for seven-to-nine years old children. An electronic story is designed to achieve this goal. It is used to gather children’s reading preferences, for example, font type/size combination, screen line length, and tutoring sound characters. Results indicated that Arab students preferred the use of Simplified Arabic with 14-point font size to ease and speed the reading process. Further, 2/3 screen line length helped children in reading faster. Finally, most of children preferred to listen to a female adult tutoring sound. Keywords: Child-Computer Interfaces, E-Learning, Font Type/Size, HumanComputer Interaction, Information Interfaces and Presentation, Line Length, Tutoring Sound.
studied adults’ preferences regarding font type and size from measuring adults’ reading efficiency, adults’ reading time, legibility, attractiveness, and adults’ font preference [1]. They found that Times and Arial font type with size 12-point were read faster than other fonts and sizes that were tested. Further, Arial and Courier font type were considered the most legible fonts. Again, Arial font type with 12-point size was the most preferable to read online. Another study carried out for adults compared the fonts used in previous editions of Windows with those new created for Windows Vista [3]. They found that the new fonts designed for Windows Vista such as Cambria and Constantia were more legible than traditional Times New Roman. While studies conducted on children indicated that the children prefer to read English text that is Arial font type with 14-point size and Comic font type with 12-point size [2]. Moreover, in English, adults prefer to read medium line length (approximately 65 to 75 CPL) and children prefer narrow line length (approximately 45 CPL) [4, 6]. On the other hand, few studies were conducted to find adult users’ preferences for Arabic script. For example, Hemayssi et. al. concluded that in order to increase legibility, it is preferred to use bold fonts, colors and clear icons [5]. Most existing research in this area is oriented to build guidelines for designing English e-learning programs. Therefore, in this paper, we examined the preferences of seven to nine years old Arab children's for five font types at sizes 12- and 14-point and their screen line length using an electronic story.
2 Usability Evaluation Process The usability evaluation of the Arabic electronic story for font type, size, and line length was conducted on July 2008. Participants were asked to spend half an hour with the e-story under the supervision of the ReDSOFT team member experimenter. At the beginning of the usability evaluation, participants were asked to complete a user profile questionnaire. The background questionnaire revealed that thirty five participants (16 males, 19 females) aged between seven and nine years old. Moreover, majority (86%) of the participants use computer at home, and almost all of them (97%) had previous experience with electronic reading. Each page of the e-story was written with a different combination of font type and size. In total, the e-story was presented in 10 pages representing the five font types (Arial Unicode MS, Courier New, Microsoft Sans Serif, Simplified Arabic, and Traditional Arabic) and two font sizes (12 and 14). Figure 1 shows sample of a page of the e-story.
Fig. 1. A sample of a page of the e-story
Arab Children’s Reading Preference for Different Online Fonts
5
After reading each page of the e-story, participants were asked to answer four questions to measure their satisfaction regarding font type and size used. 1. Is this font type and size easy to read? 2. Do you think you can read faster with this font type and size? 3. Is this font type and size attractive? 4. Would you like this font type and size to be used in your schoolbook? Text was displayed in the center of the screen. Fonts were black presented on a white background. A stopwatch was used to record the time participants took to read the paragraphs and the experimenter also noted the number of incorrectly pronounced words. To confirm findings, participants were asked to read a simple sentence with all font type and size combinations in a single page and select their preferable font type and size. Also participants were asked to choose the most comfortable screen line length while reading. The same font type and size (Simplified Arabic with size 14-pionts) script was presented to the participant in different line length conditions; 1/3 screen line length, 2/3 screen line length, and full screen line length. The participant was asked to rate his/her reading comfortance and satisfaction with the three different screen line lengths. We also explored a new direction for e-learning computer tutoring sound characters, which we believe will maximize students’ learning gains and enjoyment. The traditional scenario allows students to interact primarily with a single coach or tutor character sound on-screen. In this evaluation process, we allowed the child to select his own tutor sound to be a teacher, a boy, or a girl. The usability evaluation process measured seven main factors in the reading process: the ease of reading factor, reading faster factor, font attractiveness factor, the desire to use font combination in schoolbooks factor, the font preferences factor, the screen line length factor. And finally the tutor character sound factor. Table 1 shows different font types and sizes used in the experiment. Table 1. Font types and sizes studied Font type Arial Unicode MS Courier New Microsoft Sans Serif Simplified Arabic Traditional Arabic
3 Results 3.1 Ease of Reading Ease of reading measures whether a specific font type and size gave children an impression that the reading process was simple and trouble free. Analysis showed that
6
A. Alsumait, A. Al-Osaimi, and H. AlFedaghi
61% of the participants considered the Simplified Arabic font as an easiest to read font. The lowest ease of reading score was given to the Courier New and Traditional Arabic. Further analysis is done to find font sizes that help children in reading faster. The results showed that there is small difference between fonts that are sized 12-point and fonts that are sized 14-point. 57 % of the children thought that 14-point sized font are easy to read in oppose to 45% who thought that 12-point sized font is easy to read. The results revealed that using Simplified Arabic with 14-point font size was considered the easiest to read font for Arab children. Guideline 1 Use Simplified Arabic with 14-point font size to ease the reading for Arab children.
3.2 Reading Faster The second factor to be measured was reading speed. It is a variable that measures whether a specific font type and size gave children an impression that the reading process was done quicker. Analysis showed that font type has an affect on children's reading speed. Simplified Arabic and Microsoft Sans Serif had the highest participants’ score preference; 54% for reading faster. On the other hand, Courier New had the lowest preference score regarding the reading faster factor, only 34 % of the participants thought that they can read faster with it. When it comes to font size, results showed that children (47%) had preferred to read fonts that are sized 14-point. Fonts with size 12-point got a lower preference score. In addition, results showed that 60% of the participants believed that the Simplified Arabic with 14-point font size helped them read faster. In contrast, only 26% of the participants thought that Traditional Arabic with font size 12-point, will help them read faster. Guideline 2 Use Simplified Arabic with 14-point font size to speed the reading for Arab children.
3.3 Font Attractiveness Font Attractiveness measures whether children found a specific font type and size nice looking and eye catching. 66% of the participants considered the Arial Unicode MS as the most attractive font type among the types we tested. When it comes to font size, results showed that it has no difference in attractiveness scores. Font sizes 12-point and 14-point had similar attractiveness scores (around 60%). According to experimental results, Arial Unicode MS with 14-point font size got the best score when it comes to font type/size attractiveness.
Arab Children’s Reading Preference for Different Online Fonts
7
Guideline 3 Use Arial Unicode MS with 14-point font size when considering font attractiveness for children.
3.4 Desire to Use Font Type and Size Combination in Schoolbooks Desire to use font combination in schoolbooks factor measures whether children want to use a specific font type and size in their schoolbooks. The results showed that Simplified Arabic with font size 12-point and Arial Unicode MS with font size 14-point are considered the most desired font type and size combinations preferred to be used in schoolbooks. Guideline 4 Use Simplified Arabic with font size 12-point or Arial Unicode MS with font size 14-point when considering children's preferences for schoolbooks.
3.5 Confirm Font Preference 3.5.1 12-Point Font Size Figure 2 shows font type preferences percentage with size 12-point. Arial Unicode MS was selected as the most preferred (45%) font type with size 12-point.
50
Percentage (%)
40
30
20
10
0 Arial Unicode Courier New MS
Microsoft Sans Serif
Simplified Arabic
Traditional Arabic
Fig. 2. The font type preference percentage with size 12-point
3.5.2 14-Point Font Size Figure 3 shows font type preferences percentage with size 14-point. Arial Unicode MS was considered to be the most preferred (57%) font type with size 14-point.
8
A. Alsumait, A. Al-Osaimi, and H. AlFedaghi
60
50
Percentage (%)
40
30
20
10
0 Arial Unicode Courier New MS
Microsoft Sans Serif
Simplified Arabic
Traditional Arabic
Fig. 3. The font type preference percentage with size 14-point
Figure 4 shows which font size is preferred by participants. Result showed that participants preferred to read using 14-point font size. These results are in agreement with the results obtained through the usability evaluation process of reading the estory. 80 70
Percentage (%)
60 50 40 30 20 10 0 12-point
14-point
Fig. 4. Best font size selected by participants
3.6 Tutoring Sound At the end of the experiment, we asked the child to choose a character sound to tutor him/her through the final part of the survey. Figure 5 (a, b, c) shows the percentage of the selected tutoring sound character by the participants. The result showed that most children (48 %) regardless of their gender had selected the female teacher as a sound tutoring (Figure 5.a). This result is related to the fact that at this specific age (seven to nine years old) children are used to be taught by female teachers at the schools.
Arab Children’s Reading Preference for Different Online Fonts
60
50
Percentage
40
30
20
10
0 The teacher character "Hanan"
The boy character "Salem"
The girl character "Abeer"
Fig. 5a. The percentage of the selected tutoring sound character by all participants
60
50
Percentage (%)
40
30
20
10
0 The teacher "Hanan"
The boy character "Salem"
The girl character "Abeer"
Fig. 5b. The percentage of the selected tutoring sound character male participants
60
50
Percentage (%)
40
30
20
10
0 The teacher "Hanan"
The boy character "Salem"
The girl character "Abeer"
Fig. 5c. The percentage of the selected tutoring sound character by female participants
9
10
A. Alsumait, A. Al-Osaimi, and H. AlFedaghi
Interestingly, results also reveled that the male child participants have selected only the boy sound charter "Salem" (50%) along with the teacher sound character "Hanan" (50%) (Figure 5.b). Similarly, the female child participants have selected only the girl sound character "Abeer" (53%) along with the teacher character "Hanan" (47 %) (Figure 5.c). This indicates that children prefer only the teacher sound character along with the child tutor sound of the same gender. Guideline 5 To motivate both genders and gain their concentration, use neutral character (e.g. a teacher). To motive gender-specific participants' group and gain their concentration, use either a neutral character or an equivalent gender.
3.7 Line Length This research also investigated the effect of screen line length on reading performance and satisfaction. The same paragraph was presented in three different screen line lengths. Despite the fact that there were no significant differences in satisfaction scores (see Figure 6), a line length that supports faster reading could impact the overall experience of e-learning programs. Reading rates were found to be the fastest (4.59 sec/word) with the 2/3 screen line length, (5.44 sec/word) with the full screen line length and to be the slowest (6.56 sec/word) at the 1/3 screen line length. Participants reported either liking or disliking the extreme screen line lengths (1/3 and full line length). Those that liked the 1/3 screen line length indicated that the short line length helped faster reading and was easier because it required less eye movement and the paragraph seemed neat and clear. Those that liked the full screen line length stated that they liked having more information on one line and believe that it seemed to them that they are reading less information.
40
Percentage (%)
30
20
10
0 1/3 Screen Line Length 2/3 Screen Line Length Full Screen Line Length
Fig. 6. The percentage of the preferred screen line length
Arab Children’s Reading Preference for Different Online Fonts
11
Although some participants reported that they felt like they were reading faster at 1/3 screen line length, this condition actually resulted in the slowest reading speed. From this study, it may be beneficial to use 2/3 line lengths when possible. Guideline 6 Use 2/3 screen line length to speed the reading.
4 Conclusion Readable and satisfying e-learning interfaces are achieved through following e-learning design guidelines. These design guidelines need to be configured to suite some language characteristics. Therefore, this paper aimed to build a set of design guidelines for Arabic e-learning programs designed for seven-to-nine years old children. The results of evaluating the e-story indicated that children preferred to read text using Simplified Arabic or Arial Unicode MS with size 14-point. These font type and size combinations were considered, among the different font type and size combinations used in this study, to be the easiest and fastest font type and size combinations to read. Conversely, children, for font attractiveness, chose other preference. They preferred to read using Arial Unicode MS with size 14-point. Generally, font types with 14-point size were preferable over the other font types with 12-point size. In addition, participants were able to read faster with 2/3 screen line length. When it comes to the tutoring sound character, children preferred to listen to a female adult tutoring sound, which is closer to their actual life style. Tutoring sound characters can play a critical success factor in learner acceptance of e-learning programs. Larger combinations of font types and sized need to be assessed by Arab children. Further assessments are also needed to investigate the influence of other variables on the reading experience. Examples of those variables are: text color, effect of emphasizing pieces of text; bold, italic or underlined text, paragraph spacing, eye movements, and scrolling movements.
References 1. Bernard, M.L., Lida, B., Riley, S., Hackler, T., Janzen, K.: A comparison of popular online fonts: Which size and type is best?. Usability News. 4.4 (2002) 2. Bernard, M.L., Melissa, M., Talissa, F., Jan, M.: Which Fonts do Children Prefer to Read Online?. Usability News. 3.1 (2001) 3. Chaparro, B., Dawn Shaikh, A., Chaparro, A.: Examining the Legibility of Two New ClearType Fonts. Usability News. 8.1 (2006) 4. Dyson, M.C.: How physical text layout affects reading from screen. Behaviour & Information Technology 23(6), 377–393 (2004) 5. Hemayssi, H., Sanchez, E., Moll, R., Field, C.: Designing an Arabic user experience: methods and techniques to bridge cultures. In: Proceedings of the Conference on Designing for User Experiences DUX 2005 (2005) 6. McPherson, M.A., Nunes, J.M., Zafeiriou, G.: New Tutoring Skills for Online Learning: Are e-Tutors adequately prepared for e-learning delivery? In: Proceedings of EDEN 2003 The Quality Dialogue; Integrating Quality Cultures in Flexible, Distance and e-learning, Rhodes, Greece, June 15-18, 2003, pp. 347–350 (2003)
Adaptation Decisions and Profiles Exchange among Open Learning Management Systems Based on Agent Negotiations and Machine Learning Techniques Silvia Baldiris, Ramón Fabregat, Carolina Mejía, and Sergio Gómez Institute of Informatics and Aplications (IIiA), Universitat de Girona, Spain [email protected], [email protected], {carolina,sergiog}@eia.udg.edu
Abstract. We have developed some projects [1,2] for addressing the heterogeneity problem in open learning management systems (LMS). In [3], an independent adaptation platform to support competences development through personalization is presented. Three user characteristics (competences profile, learning style, and accessing context) are modeled by means of analyzing user interaction data in a LMS. This process is supported by the assigment of independent adaptation tasks to different JADE intelligent agents. In this paper we introduce some negotiation strategies among those intelligent agents in order to: 1) select the best types of adaptation through collaborative tasks, and 2) generate standards and exchangeable user profiles based on the inferred user characteristics, describing the mechanisms to mobilize these profiles between different LMSs. These profiles support the generation of specifics learning designs for each particular user. Keywords: competence development, adaptation, intelligent agents, adaptive hypermedia, machine learning.
Adaptation Decisions and Profiles Exchange among Open LMS
13
main issues to be addressed in the context in an open learning environment because of the impact that it could have in the learning design process. In this paper, we have focused on how to establish collaborative work among the agents in the ADA+ adaptation platform and we present a general schema for the IMS specification exchange among e-learning platforms. For the first problem we propose the definition of an optimization problem to relate the tasks developed by each agent in the platform and the cost associated to the development of these tasks. The second problem will be addressed by combining semantics and structural techniques for similarity analysis among XML documents. The paper is structured as follows. First, in the section 1 an approach for the state of the art is presented, and its relationships with the problems related in the paper. Next, the two problems mentioned above are described. In the section 4, a detailed description about the proposal is presented and finally some conclusions and future works are shown.
2 Related Works In this section we introduce some related work and its relationship with our research works. The first research area related with our proposal is the use of intelligent agents technologies in education. Many educational systems are known to reduce navigation effort, time to achieve the learning goal, and learner retention, and to increase quality of learning. In particular, Shaboo [5], Mas-PlanG [1], MAS-SHAAD [2] and aLFanet [6] are projects developed by Universidad Industrial de Santander, University of Girona and UNED, respectively, which show how user modelling processes and adaptation take into account different user characteristics independently to improve learning. Some of these projects use intelligent agents to develop adaptation tasks but frequently this set of agents do not collaborate among them to achieve a common objective. Our research efforts are oriented to include some characteristics from adaptive hypermedia systems in the context of LMS, taken into account the autonomy and mobility of the agents. An adaptation platform [3] was developed with this purpose but it is necessary to improve our work including collaborative task among the agents in the platform. This is the first objective in this paper. Conceptual modeling to facility the interoperability among LMS is the second research line of our interest. IMS specifications are one of the main elements in the user modeling and adaptation process. They are exchangeable XML documents with an associated schema. The analysis of the structural and semantic similarities between XML and with the problem of detecting change among these kinds of documents is another large research area. Tendencies for the analysis of the structural similarity in a XML document can be summarized into three groups. The first one considers a XML document as an ordered tree [7]. The second one is based on the tag similarity [8], and the third one consist on the interpretation of the XML document as a time series which applies the Discrete Fourier Transform [9], analyzes the frequency of this signal and infers the similarity measure.
14
S. Baldiris et al.
Some algorithms such as MH-Diff [10] and LaDiff [11] have been developed for detecting change between structured documents algorithms. For the specific XML documents, XMLTreeDiff [12], XyDiff [13], X-Diff [14] has been developed. Our intention is not propose a new algorithm but to use the best one with the purpose of generating a full user model exchange. The user modelling process defines and maintains up-to-date user models, where user models are a representation of information about an individual user that is essential for an adaptive system to provide the adaptation effect [15]. The result of this process is a user model that consists in different user characteristics and can be expressed in different formats according to the system that stores and exchanges it. User model servers or shells are systems in charge of maintaining the user models and they generally have the follows characteristics: 1) it use a particular language to represent the user characteristics, 2) require specific collaborative filtering techniques in order to create stereotypes of user groups, 3) count with a user history recorder, and also 4) apply techniques such as user models retrieval, update and validation. Some of the developments in this area are GUC [16], UM [17], BGP-MS [18], DOPPELGÄNGER [19], TAGUS [20], GUMS [21], PROTUM [22], UMT [23]. A main issue regarding the exchange of information between user model servers is the analysis of the information that it contains. This problem is related for us with the XML similarity analysis.
3 Problem Description Our research works are based on the following two hypotheses: The first one is to support the learning process with the aim to improve the learning. In this case, the knowledge about the users (user modeling) such as previous competences, individual preferences and human values has to be considered [1] and the learning process has to be adaptive according to this knowledge. The second hypothesis is that in an Table 1. Characterization of the framework elements to support competence development
Specifications Exchange Service Agents Technologies • Technological Standard Based Learning Adaptation Process Agent Technologies • Platform Independence • Technological Standard Based Decision Generation Process Machine Learning Techniques Collaborative Filtering • Platform Independence • Technological Standard Based User Modelling Process Machine Learning Learning style, competences, etc. Independent Agents Platform LEARNING PROCESS MODELLING (Competences, Course Structure, Learning Resources and Activities, Evaluation Another Necessary Models)
Adaptation Decisions and Profiles Exchange among Open LMS
15
open learning environment the possibility to interchange user profiles based on competences can facilitate the management of the learning process. Table 1 presents a general vision of different process to verify these hypotheses. In [4] we describe a user modeling and e-learning adaptation process based in the use of different IMS Specification. The user modeling process defines the method for inferring some user characteristics. In particular, the specific competence level, the collaborative competences level, the user learning style and the context. User modeling process supports a specific learning adaptation process [3]. The result of this process can be an IMS Learning Design (LD) level B, which is a course that takes into account some user features. This adapted learning design permits to deliver learning objects (LO) and activities according to the values of user characteristics presented above. The ADA+ adaptation platform was developed with this proposes and consists in different intelligent agents, which have particular adaptation tasks (see table 2). Table 2. Specific Agent Task in the adaptation platform
Specific Agent Task Planner Agent Specific Competence Agent Collaborative Competence Agent Learning Style Agent
Context Agent
Task description − Generates an adapted IMS-LD (level B) for users. − Generates the planning problem using competence definition and the LO metadata. − Defines the collaborative level of the user using a grouping behaviour. − Defines the delivering order of the learning resource type using automatic classification techniques. − Monitors the change in the user learning style according to the user interaction. − Selects equivalent LO according to the user access device and accessibility user preferences.
Actually, agents in ADA+ adaptation platform carry out some tasks independently. The reason behind this is that we are interested in creating a flexible collaborative environment among intelligent agents. This environment is meant to consider additional users’ characteristics when needed. However, we have analysed that the problem of context limitations in the virtual environment can produce undesired adaptation results. For these reason, one of the purposes of this paper is to introduce an approach based on the definition of an adaptation decision process in order to identify the types of adaptation that can be offered in a specific moment according to the learning context limitations. On another hand, one of the main purpose for applying specifications and standards in education technologies is to facilitate the information exchange among LMS.
16
S. Baldiris et al.
IMS consortium has worked in the generation of different kind of specifications with the objective of creating a standard e-learning language and increase the systems interoperability. These languages permit to specify products and sub-process of the learning process such as competence, learning designs, and assessments among others. We present the Specification Exchange Service (SES) as a mechanism for exchange information among e-learning platforms, in particular, IMS specifications. It proposes a general framework supported in an adjusted version of X-Diff algorithm [24]. SES could support the decision process by increasing the quantity of resource than can be used by the agents in ADA+.
4 Proposal Description In this section, we present our proposal to establish a collaborative work among the agents in the adaptation platform, and present a general schema for the IMS specification exchange among e-learning platforms. 4.1 Adaptation Decision through Detection of Context Constrain and Negotiation When the adaptation idea is introduced in the context of a virtual learning process, the starting point is still the modelling of the teaching-learning process as in the traditional (face to face) process. However, the initial work for the teacher increases because she needs to perform new activities using a LMS and adjust the environment according to the requirements. If the teacher does not provide the necessary information to the systems, multiples context limitations can be detected in the environment. These limitations impact negatively the automatic generation of the adaptive learning design. Table 3 summarizes the different limitations that can be found in the learning environment. Table 3. Agent tasks vs.context limitations
Specific Agent Task Specific Competence Agent Collaborative Agent Learning Style Agent
Context Agent
Context Limitations − Existence and quality of the Competence definition. − Existence and quality of the LO metadata according to the needed model. − Amount and quality of users interaction in collaborative tools. − Amount of learning objects according to the different LO metadata resource types. − Amount and quality of users interaction with the different LOM resources types. − Existence of the equivalent LO for the topics.
Adaptation Decisions and Profiles Exchange among Open LMS
Description 1 if competence definition exist % 1 if collaborative interaction exist % 1 if learning objects exist % 1 if equivalent learning objects exist % 1 if objects metadata exist % %
These constraints can be expressed in terms of the specific variables. Table 4 shows some of them. In ADAPTAPlan project each particular agent can define the value of specific variables and they use them as the input of the decision process. The natures of the variables permit us model the decision problem as an optimization problem, where the objective is to reduce the cost of accessing distributed resource repositories in order to obtain the necessary information to generate the adaptation task. In this way, the decision process can be supported in a cost matrix where different source and its cost are related in terms of different characteristics as network traffic needed, purchase cost, availability, and importance, among other. Taken into account the cost associated to the necessary resources for each particular adaptation task and also the cost associated to combinations of adaptation tasks, the decision is defined. As a result of this collaboration process based on the cost of the different resources, an adapted IMS-LD is generated. 4.2 Specification Exchange Service (SES) Consider two different e-learning platforms where the corresponding students developed different learning activities. Each particular platform is associated to a specific user modeling system and user model shell system. As was described above, the user modeling system obtains user model for the users through the analysis of user interaction and the user model shell system maintain the model (see figure 1). The question is: how can the information in these systems be shared? In last years, XML have become the most common language to represent information to be shared about the learning process and the people. Indeed, IMS specifications use XML as a representation language and schemas for the structure definition and validation.
18
S. Baldiris et al.
Fig. 1. Exchange Problem
For this reason we have tested some algorithms for the analysis of the similarities between XML documents, algorithms were shown in the section 2. We have selected diffX algorithm [24] in order to detect the structural changes in the XML documents and for supporting the generation of a new user model which is a mix of previous two documents. diffX algorithm was selected for including move operation that another tested algorithm does not implement. The process of the new user model generation is shown in the figure 2.
Fig. 2. Complete User Model Generation
The first part of the process, matching, considers the XML documents as an ordered tree, identifying isolated tree fragment mapping (i.e. an iterative top-down mapping technique). The result of this mapping is the definition of the set of sub trees that are the same in both trees. This identification permits to determinate the set of necessary operations to transform one document into another. The set of operations are applied to the document of the LMS that has generated the request. Then the document is sent to the requester LMS. Semantic analysis of the XML document in the matching process it is supported in the schemas associated to different IMS specifications.
5 Conclusion and Future Works In this paper we deal with two problems associated to the use of adaptive e-learning platforms and we propose an approach for each of them. The first one is about how the incomplete learning process modeling can produce undesired adaptation results in the LMS. We propose a strategy based on the collaboration among the intelligent agents in order to generate an adequate adaptation decision. The decision strategy is supported by a cost matrix where different source and its cost are related in terms of
Adaptation Decisions and Profiles Exchange among Open LMS
19
different characteristics as network traffic needed, purchase cost, availability, and importance. The second problem is related to the need of information exchange among elearning platforms, user profiles exchange, competence definition or any other type of information, in order to address this problem one approach supported in diffX algorithm was introduced. Proposals presented in the paper are under construction in the context of A2UN@ project (TIN2008-06862-C04-02/TSI). In particular, different types of repositories for supporting the agent collaboration are under construction. Testing process for comparing different algorithms have been developed and the integration of different syntactic and semantic algorithms is been implemented.
References 1. Peña, C.I.: PhD Thesis: Intelligent agents to improve adaptivity in a web-based learning environment. Universidad de Girona (2004) 2. Mérida, D., Cannataro, M., Fabregat, R., Arteaga, C.: MAS-SHAAD a Multiagent System Proposal for an Adaptive Hypermedia System. In: Proceedings of IJCEELL journal Special issue: Adaptivity in Web and Mobile Learning Services (2004) 3. Baldiris, S., Santos, O., Huerva, D., Fabregat, R., Boticario, J.G.: Multidimensional Adaptations for Open Learning Management Systems. Accepted at TUMASA Workshop of 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Sydney (Australia), December 9-12 (2008) 4. Baldiris, S., Santos, O.C., Barrera, C., Boticario, J.G., Velez, J., Fabregat, R.: Integration of educational specifications and standards to support adaptive learning scenarios in ADAPTAPlan. International Journal of Computer Science and Applications (IJCSA). Special Issue on New Trends on AI techniques for Educational Technologies 5, 1 (2008) 5. Moreno, G.D., Baldiris, S.M.: Degree project memories: Adaptive Hypermedia System for Teaching Object Oriented Programming. Universidad Industrial de Santander (2003) 6. Santos, O.C., Boticario, J.G.: Meaningful pedagogy via covering the entire life cycle of adaptive eLearning in terms of a pervasive use of educational standards: the aLFanet experience. In: Nejdl, W., Tochtermann, K. (eds.) EC-TEL 2006. LNCS, vol. 4227, pp. 691–696. Springer, Heidelberg (2006) 7. Shasha, D., Zhang, K.: Fast algorithms for the unit cost editing distance between trees. Journal of Algorithms 11, 581–621 (1990) 8. Buttler, D.: A Short Survey of Document Structure Similarity Algorithms. In: International Conference on Internet Computing 2004, pp. 3–9 (2004) 9. Flesca, S., Manco, G., Masciari, E., Pontieri, L., Pugliese, A.: Fast Detection of XML Structural Similarity. IEEE Transaction on Knowledge and Data Engineering 17(2), 160–175 (2005) 10. Chawathe, S., Garcia-Molina, H.: Meaningful change detection in structured data. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, Tucson, Arizona, USA, May 13-15 (1997) 11. Chawathe, S., Rajaraman, A., Garcia-Molina, H., Widom, J.: Change detection in hierarchically structured information. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, Montreal, Quebec, Canada, June 4-6 (1996) 12. Cubera, F., Epstein, D.: Fast Difference and Update of XML Documents, March 1999. Xtech, San Jose (1999)
20
S. Baldiris et al.
13. Cobena, G., Abiteboul, S., Marian, A.: Detecting changes in XML documents. In: Proceedings of the 18th International Conference on Data Engineering, San Jose, California, USA, February 26 - March 1 (2002) 14. Wang, Y., DeWitt, D., Cai, J.: X-Diff: An effective change detection algorithm for XML documents. In: Proceedings of the 19th International Conference on Data Engineering, Bangalore, India, March 5-8 (2003) 15. Brusilovsky, P., Millán, E.: User Models for Adaptive Hypermedia and Adaptive Educational Systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) Adaptive Web 2007. LNCS, vol. 4321, pp. 3–53. Springer, Heidelberg (2007) 16. Van Der Sluijs, K., Houben, G.: A generic component for exchanging user models between web-based systems. International Journal of Continuing Engineering Education and Life-Long Learning 16(1/2), 64–76 (2006) 17. Kay, J.: The um Toolkit for reusable, long term user models. User Modeling and UserAdapted Interaction 4(3), 149–196 (1995) 18. Kobsa, A.: Modeling the user’s conceptual knowledge in BGP-MS, a user modeling shell system. Comput. Intelligence 6, 193–208 (1990) 19. Orwant, J.: Heterogenous learning in the Doppelgänger user modeling system. User Model. User-Adapted Interact. J. Personal. Res. 4(2), 107–130 (1995) 20. Paiva, A., Self, J.: TAGUS: A user and learner modeling workbench. User Model. UserAdapted Interact. J. Personal. Res. 4(3), 197–226 (1995) 21. Finin, T.W., Drager, D.: GUMS1: A general user modeling system. In: Sixth Canadian Conference on Artificial Intelligence, Montreal, Canada, pp. 24–29 (1986) 22. Vergara, H.: PROTUM: a prolog based tool for user modeling. WIS-Report 10, WG Knowledge-Based Information Systems, Department of Information Science, University of Konstanz, Germany (1994) 23. Brajnik, G., Tasso, C.: A shell for developing non-monotonic user modeling systems. Int. J. Human-Computer Studies 40, 31–62 (1994) 24. Al-Ekram, R., Adma, A., Baysal, O.: diffX: An Algorithm to Detect Changes in MultiVersion XML Documents. School of Computer Science, University of Waterloo (2005)
Accessing e-Learning Systems via Screen Reader: An Example Maria Claudia Buzzi1, Marina Buzzi1, and Barbara Leporini2 1
Abstract. The evolution of the Information and Communication Technology (ICT) and the rapid growth of the Internet have impelled the pervasive diffusion of e-Learning systems. This is a great opportunity for visually-disabled people provided that both the interactive environment, created by the Learning Management Systems, and the Learning Objects, created by teachers, are properly designed and delivered. In this paper we investigate interaction of the blind user with an open source Virtual Learning Environment (Moodle) and discuss how the use of the W3C Accessible Rich Internet Applications (ARIA) suite may improve the experience of navigation via screen reader. Keywords: e-Learning, accessibility, usability, blind, ARIA.
When designing for blind users, it is necessary to consider the three main interacting subsystems of the Human Processor Model: the perceptual, motor and cognitive systems [5]. Sightless persons perceive page content aurally and navigate via keyboard. This makes the “reading process” time-consuming and sometimes difficult and frustrating, if the contents are not designed with special attention to their needs. The cognition part of the interaction is important, since many learning techniques are only relevant to people with good vision and may not apply to someone with a visual impairment. Thus, alternative ways to deliver the same content should be provided. Furthermore, a blind person may develop a different mental model of both the interaction and the learning processes, so it is crucial to provide an easy overview of the system and contents. Non-visual perception can lead to: 1. Content serialization. The screen reader reads the contents sequentially, as they appear in the HTML code. This process is time-consuming and annoying when part of the interface (such as menu and navigation bar) is repeated in every page. As a consequence, blind users often have to stop the screen reading at the beginning, and they prefer to navigate by Tab Keys, from link to link, or explore the content row by row, via arrow keys. 2. Content and structure mixing. The screen reader announces the most important interface elements such as links, images, and window objects as they appear in the code. For the blind user, these elements are important for figuring out the page structure, but require additional cognitive effort. 3. Table. If the table’s content is organized by columns the screen reader (which reads by rows) announces the content of the page out-of-order, and consequently the information might be confusing or misleading for the user. 4. Lack of context. When navigating by screen reader the user can access only small portions of text and may lose the overall context of the page; thus it may be necessary to reiterate the reading process. 5. Lack of interface overview. Blind persons do not perceive the overall structure of the interface, so they can navigate for a long time without finding the most relevant contents. 6. Difficulty understanding UI elements. Links, content, and button labels should be context-independent and self-explanatory. 7. Difficulty of working with form control elements. The new JAWS version (v. 10) simplifies the interaction with forms since it can automatically activate the editing modality (for text input) when the virtual focus arrives at the text box (for instance when the user presses the tab key). However, with previous screen reader versions the user may have great difficulties since switching between exploration and editing modalities is required (i.e. form mode on/off). 8. A blind person is unable to access multimedia content such as video streaming, video conferencing, and captioning. If an alternative description is not present the user may lose content. In this paper we analyze accessibility and usability for the blind of two demo courses offered by Moodle (http://www.moodle.org/), a very popular open source eLMS. Then we illustrate how ARIA, the suite developed by the Web Accessibility Initiative (WAI) group of W3C, may facilitate interaction for the blind. Lastly, the paper presents our conclusions.
Accessing e-Learning Systems via Screen Reader: An Example
23
2 Related Works E-learning systems pose new challenges with respect to classic user-centered product design, where the target is a set of homogeneous users. Learner-centered Design must answer to the needs of multiple learner categories due to differences in learning strategies, know-how, experiences, motivation to learn and, not least, user age and ability. If appropriately designed and implemented, e-Learning systems are more effective and useful than classroom learning [4]. Various studies focus on the usability of e-Learning systems and some also include a general discussion on accessibility, but to our knowledge only a few focus on totally blind persons. Fifteen years ago Nielsen proposed an informal method for evaluating systems and design usability based on verifying the conformance to a set of principles of usable software design (heuristics) performed by experts [8]. This approach, which detects a high percentage of problems, is cost-effective and easy to implement compared to usability testing. However, to be effectively applied to the e-Learning domain these general principles needed to be further refined. A few years later, Squires and Preece embedded usability heuristics in the socio-constructive theory and specified criteria ad hoc for e-Learning [12]. However, even today various researchers criticize the lack of accurate studies in this field. In [1] the authors take the first steps in defining a methodology for the rigorous evaluation of e-Learning applications, but accessibility for special needs students is not analyzed. Furthermore, Zaharias critically examined the usability of e-Learning applications and proposed a new usability measure: the student’s intrinsic motivation to learn [15]. Developing a usability evaluation method based on a questionnaire, he carried out two large empirical studies showing the reliability of this approach. For Sloan et al. the goal of universal accessibility on the Web is inappropriate and instead it is necessary to explore multiple routes to provide equivalent experiences [11]. As Kelly et al. argued, rather than demanding that an individual learning resource be universally accessible, it is the learning outcome that needs to be accessible [7]. Based on user profiles, metadata and dynamic connection to resources, the user’s experience can be customized to match his/her abilities. Then an appropriate design is crucial for improving the accessibility and usability of eLearning Systems. De Marsico et al. [5] defined methodological guidelines involving users with disabilities as well as pedagogical experts in the development process, believing that input of different know-how may enrich the quality of e-Learning applications, and provide a more satisfying learning experience. They also include two examples of building and providing learning objects accessible respectively to visually- and hearing- impaired students. Rodriguez et al. describe a project for improving e-Learning experience for the visually impaired, based on ethnomethodology and taking into account psychosocial issues, the user context and experience [9]. Next they created different learning object formats suitable for the blind, including DAISY (Digital Accessible Information SYstem). However, although authors describe the methodology used to improve learning materials no general guidelines are offered to the reader. Within the framework of a project aimed at providing an accessible e-Learning platform for disabled and adult learners, Santos et al. [10] illustrate a methodology for developing standard based accessible courses which use two-step evaluations. However for the totally blind, more specific UI features are necessary than those provided
24
M.C. Buzzi, M. Buzzi, and B. Leporini
in this study, such as providing a page overview, full control of interface elements and easy and rapid navigation via keyboard. E-Learning is a great opportunity for visually-disabled people, provided that both the interactive environment (created by the LMS) and the learning objects are properly designed and delivered. In this paper we mainly focus on accessibility and usability of the virtual environment for the blind, taking as an example the environment offered by Moodle. Concerning Moodle, Debevc et al. [4] compared usability of two LMSs, i.e. Moodle vs a proprietary system, when delivering an identical package of educational materials. This usability study was carried out with a SUMI (Software Usability Measurement Inventory) evaluation that is not specific for e-Learning environments. However indications derive from this study, that point out Moodle’s weaknesses (vs the proprietary system) in efficiency as well as learnability.
3 The Moodle Virtual Learning Environment To evaluate accessibility and usability of a Virtual Learning Environment (VLE) via screen reader, we chose Moodle, an Open Source LMS which offers a rich VLE. Thanks to its large spectrum of available tools and options it has become very popular worldwide. Specifically it offers (http://www.moodle.org): Assignment modules, Blogs, Chat, Choices with multiple response options, Course resources (Moodle pages, uploaded files or web links), Databases, Forums, Glossaries, Interaction, Lessons, SCORM packages, Surveys, Quiz module, Wikis and Workshop modules. The richness of the VLE is Moodle’s strength but the environment‘s complexity can also create difficulties in interaction via screen readers. 3.1 Evaluation Methodology In the first step of our research, we analyzed two demo courses available on the website (http://demo.moodle.org) by using the screen reader JAWS for Windows (http://www.freedomscientific.com) v. 9.0 and 10. We used both the MS IE version 7.0 and the Mozilla Firefox version 3.0.5 browsers. The test was carried out by all the authors independently; afterwards, outcomes were compared and integrated. One author has been totally blind since childhood and uses the JAWS screen reader every day; thus she knows this tool’s functions very well and is able to use advanced commands. By analyzing the test results we noticed that in spite of her great expertise using JAWS, she was unable to perceive the exact structure of the layout. This paper’s sighted authors carried out the test using only JAWS basic commands, but viewing the Moodle UI allowed them to rapidly understand the origin of obstacles encountered when interacting via screen reader. Therefore, integrating both these outcomes led to a more accurate analysis. The different experiences of the authors when using JAWS allowed us to cover a variety of interaction modalities: i.e. both basic commands, which simulate the use of beginner users, and advanced screen reader functions. 3.2 Exploring the Moodle Demo To illustrate how a blind user interacts with the Moodle environment, we explore several pages related to two selected courses, according to specific features to be
Accessing e-Learning Systems via Screen Reader: An Example
25
tested: main page functions, topic outline of a selected course, interacting environment, and so on. When opening the demo page, before doing the log-in, JAWS announces four headings. Through the special command “Insert+F6” JAWS shows the headings list. Although different levels are assigned to these headings, the levels order used is unsuitable. In fact, level 1 is associated with the course title; thus, by pressing the key “1” the virtual focus moves to the beginning of the page. In this case, it would be more appropriate to assign the level 1 to the “Log in” section, which logically is the first part of the interface the user must navigate to enter the system. After the login, the list of available courses is shown. Once the chosen course is clicked, its page is loaded and opened. The screen reader informs the user about headings as well as the number of links. Basically, this page seems to be accessible, because several accessibility features are detected (for instance, a hidden link pointing to the “main content”) by the screen reader. Also, headings are immediately identified and announced by JAWS. But, are all these features suitable for comfortable navigation by blind users? Let us consider the “Higher Education Film Studies Module” (in the following referred to as Course A), available as a demo course.
Fig. 1. Moodle demo - Course A
After selecting this course the following elements are detected by JAWS: • Headings: 32 headings are announced. By exploring the heading list (Insert+F6 command) we note that most of them refer to the course, but the others are related to the e-learning tools (e.g. chat, blog, and so on). Those headings are too numerous to be navigated comfortably. Although headings are used, the page is too long to be read by keyboard via screen reader. Furthermore, it is not very clear which headings refer to the course modules and which ones to the interacting platform functions (see below).
26
M.C. Buzzi, M. Buzzi, and B. Leporini
• Layout table: to show the Topic outline a table has been used for the layout. Specifically two tables are used, the second nested in the first cell of the first table, but JAWS announces only one table. Although headings are used to split the long content, a table should not be used for the rendering. Moreover, the “summary” attribute of the table is “table layout”, totally useless. A more appropriate summary value could be “table of contents”; in this way, by just pressing the letter “t” JAWS will announce immediately “Table of contents”. If a layout table is used, at least a meaningful summary should be applied. • Number of links. The page contains too many links (in our case 115 links). • Images. Many of the images in the pages of the course are not perceived by the screen reader since no alternative text is provided or descriptions are negligible. After the log-in as teacher, we tried to insert an image and found that the alternative text is now mandatory. However there is no control on the number of characters inserted, so it is possible to skip this edit field with only a space press. In order to understand whether the issues encountered in the selected course could be generalized, we attempted to navigate another demo course. Specifically, we picked up the course “Moodle Features Demo” (in the following referred to as Course B). Also the main page of the second course includes a great number of links, two tables used for visual layout, and 25 headings. However, the level assigned to each heading as well as their arrangement differ from Course A. In this case, first the headings for interacting with the Moodle environment are available, and then the Topic outline. Consequently, it seems the UI is not consistent among the courses even if they are very similar (probably due to changes in the Moodle versions used to create these courses). However, headings arranged in this way might make the page structure somewhat unclear. In fact, it is difficult to know how many modules compose the course and their titles and contents, since the module number has not been included in the tag and thus does not appear in the headings lists (Fig. 3); to obtain this information the user must read the page in a sequential way, as shown in the Fig. 2 – left section. All headings are mixed and not well grouped by clearly indicating which are modules (See Fig. 3), and which are associated with the virtual environment (on the right side in Fig. 1). In other words, the main regions: “menu”, “topic outline” and “interacting actions” should be better indicated. For example, by opening the page related to “eXe SCORM package”, the screen reader recognizes some unclear elements: • various graphical icons; the image tags have the ALT attribute, but the alternative descriptions used are not clear for the type of the icon. For instance, several links are preceded by an icon labelled “Completed” (See Fig. 2 – right section). It is not clear what this means. • Frame: in the page a frame called “scoframe1 frame” is identified by JAWS, which is not particularly clear. Even if frame content is explored in a sequential way, is unclear what the student. According to the last version of accessibility guidelines [WCAG 2.0] frames should be avoided (deprecated). The following figure reports the page content as it is interpreted by the screen reader JAWS.
Accessing e-Learning Systems via Screen Reader: An Example
COURSE B Features Demo: eXe SCORM package Heading level 1 Moodle Features Demo You are logged in as Link Sam Student (Link Logout) Heading level 2 You are here list of 4 items Link Moodle Demo/ ... ? eXe SCORM package list end … eXe Scorm Package Link Expand/Collide Graphic Completed Link eXe Scorm Package list of 5 items nesting level 1 Graphic Completed Adding an image Graphic Completed ... list end nesting level 1 list end Review Mode scoframe1 frame Adding an image Graphic img6_2 The eXe logo In eXe it is easy to include images with your web content. ... scoframe1 frame end
Fig. 2. Page segments sequentially read by JAWS (up/down arrows): on the left Course A main page, on the right a SCORM module function of Course B
Fig. 3. List of headings of the demo course A (“Insert+F6” JAWS command)
28
M.C. Buzzi, M. Buzzi, and B. Leporini
In short, although the pages seem to be accessible, our initial interaction using the screen reader JAWS shows some usability issues when interacting via keyboard and in a sequential manner. The few pages considered in our preliminary evaluation highlighted some difficulties orienting oneself among different information (i.e. headings and links), as well as in handling conceptual information. For instance, in some cases pictures or graphical representations are used to provide information on modules or on intersecting concepts. In a learning system it is also important to make those concepts accessible and usable by alternative modalities.
4 Applying ARIA When designing a web interface it is essential to keep in mind both usability and accessibility principles. Accessibility is a basic pre-requisite for allowing access to page content, while usability guarantees an easy, simple, efficient, rapid and satisfactory navigation and interaction. Navigation is vital for special-needs persons, and in particular for the blind, because it is crucial for them to be aware of their current location on the webpage and how to return to the beginning, or how to reach a certain point in the materials [4]. The use of ARIA would enhance Moodle’s usability in many ways, as explained and exhaustively illustrated by the ARIA best practises document [13]: • Reduce the amount of unnecessary text announced. Specifically the use of Table as layout may be silently ignored by the screen reader if the table is tagged with the presentation role :
…
• Definition of regions to allow the user get a page overview. Moodle utilizes heading levels to structure the page and to allow the user to perceive the whole structure. Using only standard (X)HTML code, the usability of keyboard navigation is reduced so blind users are forced to use tabbing for accessing active elements (form elements and links). To simplify interaction and allow easy jumping to main interface regions, developer usually relay on creating a link to the main content or use heading levels to structure the page (since the screen reader gives a table of headings). However the use of headings to mark sections is not consistent across web sites [13]. ARIA allows marking sections with standard specifying XHTML landmarks or defining customized regions:
…
• Simplifying keyboard navigation. The use of landmarks/regions also allows to simplifying navigation via keyboard since the user may jump from one region to the next by pressing a key (in JAWS v.10 the “;”). Furthermore, the developer using the attribute “flowto” defines the order in which regions should be visited:
These are basic considerations to improve the interaction via screen reader with the VLE offered by the LMS, that should be integrated on the basis of all observations reported below. Further improvements are possible in the specific learning object
Accessing e-Learning Systems via Screen Reader: An Example
29
prepared by the teacher; thus it is very important to provide automatic tools to support the easy creation of accessible and usable learning objects.
5 Conclusion In this paper we analyze the usability of the Moodle Learning Environment for the blind. Specifically we analyzed two demo courses provided as examples by the system, highlighting features that could be improved. We mainly focus on the navigability of the virtual environment provided by Moodle, which by offering several educational tools integrated into one system, may create complex interfaces. It is important to notice that LMSs may greatly favours the student learning process since the same educational material may be transmitted anywhere, anytime, at any learning rhythm, in a format suited to each individual's ability. On the other hand, since LMSs automatically add a virtual environment to the educational material, if the virtual environment layout is not appropriately designed with a thorough knowledge of accessibility and usability issues, it may induce problems that could be spread to the learning objects themselves. This highlights the importance of considering usability issues from the beginning of the development of every LMS. Making a VLE suitable for the abilities and skills of all users offers many challenges. When defining the graphical UI it is fundamental to consider the needs of sighted users but the needs of the blind should also be kept in mind when writing the UI code. Specifically, the same information should be provided through both visual and auditory channels, the design should be optimized for reading via screen reader, the UIs should be easy to use via keyboard and no additional cognitive effort should be required of the blind user. In conclusion, we believe that our findings could have general applications and that applying ARIA would enhance usability via screen reader in any Virtual Learning Environment.
References 1. Ardito, C., Costabile, M., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., Rossano, V.: An Approach to Usability Evaluation of e-Learning Applications. Universal Access In the Information Society 4(3), 270–283 (2005) 2. Card, S.K., Moran, A., Newell, T.P.: The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates Inc., New Jersey (1983) 3. Debevc, M., Bele, J.L.: Usability testing of e-learning content as used in two learning management systems. European Journal of Open, Distance and E-Learning (2008), http://www.eurodl.org/materials/contrib/2008/Debevc_Bele.htm 4. Debevc, M., Verlic, M., Kosec, P., Stjepanovic, Z.: How Can HCI Factors Improve Accessibility of m-Learning for Persons with Special Needs? In: Stephanidis, C. (ed.) HCI 2007. LNCS, vol. 4556, pp. 539–548. Springer, Heidelberg (2007) 5. De Marsico, M., Kimani, S., Mirabella, V., Norman, K.L., Catarci, T.: A proposal toward the development of accessible e-Learning content by human involvement. UAIS Journal 5(2), 150–169 (2006)
30
M.C. Buzzi, M. Buzzi, and B. Leporini
6. Eijl, P., Pilot, A., Voogd, P.: Effects of Collaborative and Individual Learning in a Blended Learning Environment. Education and Information Technologies 10(1-2), 51–65 (2005) 7. Kelly, B., Phipps, L., Howell, C.: Implementing a holistic approach to e-Learning accessibility (Retrieved), http://www.ukoln.ac.uk/web-focus/papers/alt-c2005/accessibility-elearning-paper.doc 8. Nielsen, J.: Usability inspection methods. In: Heuristic evaluation, pp. 25–62. John Wiley & Sons, Inc., New York (1994) 9. Rodriguez, E.P.G., Domingo, M.G., Ribera, J.P., Hill, M.A., Jardi, L.S.: Usability for All: Towards Improving the E-Learning Experience for Visually Impaired Users. LNCS, pp. 1313–1317. Springer, Heidelberg (2006) 10. Santos, O.C.B., del Viso, J.G., de la Cámara, A.F., Sánchez, S.P., Gutiérrez, C.R., Restrepo, E.: HPCN-Europe 1994. LNCS, pp. 796–805. Springer, Heidelberg (2007) 11. Sloan, D., Heath, A., Hamilton, F., Kelly, B., Petrie, H., Phipp, L.: Contextual web accessibility - maximizing the benefit of accessibility guidelines. In: Proceedings of the 2006 international cross-disciplinary workshop on Web accessibility (2006) 12. Squires, D., Preece, J.: Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers 11(5), 467–483 (1999) 13. W3C. WAI-ARIA Best Practices. W3C Working Draft 4 February (2008), http://www.w3.org/TR/wai-aria-practices/ 14. Wilson, R., Landoni, M., Gibb, F.: A user-centered approach to e-book design. The Electronic Library 20(4) (2002) 15. Zaharias, P.: A usability evaluation method for e-learning: focus on motivation to learn. In: Proceedings of CHI 2006 extended abstracts on Human factors in computing systems (2006)
Using Tablet PCs and Pen-Based Technologies to Support Engineering Education Ignacio Casas1, Sergio F. Ochoa2, and Jaime Puente3 1
Department of Computer Science, Pontificia Universidad Catolica de Chile, Chile 2 Department of Computer Science, Universidad de Chile, Chile 3 Microsoft Research, Redmond, WA, USA [email protected], [email protected], [email protected]
Abstract. Several experiences and results of the Tablet PC adoption have been reported, mainly in American universities. Although the benefits seem to be highly interesting, it is not clear if they are replicable in developing countries. In order to try to understand the impact of Tablet PCs on engineering education in Chile, the authors conducted several experiments at the two traditional Chilean universities. This paper reports the experiences and the obtained results, comparing them with those obtained in American universities. Keywords: Tablet PCs, Pen-based Technologies, Engineering Education, Mobile Computing.
by the authors in order to determine the impact that pen-based computing have in engineering education in the two main Chilean universities. Section 5 discusses the obtained results. Finally, section 6 shows the conclusions and the future work.
2 Tablet PCs in High Education During the last years several researchers have been doing experiments in order to determine the impact of using Tablet PCs in education. Some of them believe that a “Tablet PC has the potential to dramatically alter the educational process. This new technology significantly changes the way students and teachers interact. It adds completely new dimensions to classroom interaction by providing digital ink and drawing tools for writing, sketching, and drawing; and for real-time collaboration” [1]. At the moment we have seen reports of particular experiences where Tablet PCs have became interesting tools in certain scenarios. Now the challenge is to identify in which scenario the Tablet PCs can effectively add value. For example, Anderson et al. show how Tablet PCs can be used to improve the knowledge delivery and facilitate the interaction between students and the instructor during a distributed lecture [2]. Berque et al. report the results of using Tablet PC and DyKnow tools to support collaborative problem solving [3]. Davis et al. and Anderson et al. report experiences of using Tablet PC to take notes during a class [2, 4]. In addition, the authors have shown, in previous works, how these mobile devices can be used to improve the evaluation process in computer science courses [9] and also to support the activities of communities of practice in science education [5]. There are also several articles describing educational experiences on physics, mathematics and agronomy, in which Tablet PCs have been used as the supporting platform for the teaching-learning process. Considering the experiences reported in recent literature, and focusing just on the main features of the Tablet PCs, we can say that the main activities to be supported with these devices are the following ones: 1. Taking notes. Students can take notes on the slides the professor shares with them, or just on a digital blank sheet. These digital notes can be organized, shared, searched, emailed and linked to other resources [6]. In that sense, this functionality seems to be highly useful, and better than the paper notebook or even a laptop. 2. Enrich the lecture presentation. Instructors can use Tablet PCs to enrich the information they are presenting during a lecture. It could be used, for example, to do marks or make annotations on the slides, or use the Tablet PC as a replacement for (or supplement to) the black/whiteboard [2]. Since digital annotations made on these devices can be shared, the cognitive load of the attendees is reduced. 3. Support group work. During work meetings the group members (i.e. students and/or instructors) usually carry out brainstorming, discuss proposals/alternatives to carry out the job, and validate ideas or the work done. Usually a black/whiteboard is used to support these activities; however Tablet PCs could also be used for this purpose [3]. A benefit of using these devices instead of whiteboards is that notes written on the space shared by the attendees are easier to record and share. In addition, it is not longer needed to erase the whiteboard because its physical space is completely used.
Using Tablet PCs and Pen-Based Technologies to Support Engineering Education
33
In this list we have excluded the activities that can be performed with a laptop. In other words, we included just the activities requiring the special features of a Tablet PC. Please note there are activities that are variants of the listed ones, which were not included in the list. Summarizing, it seems the use of these computing devices could replace the use of the paper notebook and also the black/whiteboard.
3 Impact on the Students and Instructors Experiences reported in the literature show the students are usually comfortable using Tablet PCs to take notes and perform problem-solving activities [13, 14, 15]. Something similar happens with the instructors’ opinion [5, 15]. In addition, most of the researchers that explore the use of these devices in educational scenarios state that Tablet PCs facilitate the active learning [12, 16][Roschelle et al., 2007]. Considering these previous works, some of the main contributions reported about the use of Tablet PCs, are the following ones: • Benefits for students. Students recognize the possibility to take notes is an important contribution of Tablet PCs usage. The free-style handwriting possibility makes the students more comfortable to express their ideas in sketches or annotations [14]. The possibility to exchange digital resources among the students (or with the instructors) was recognized as an important feature [16]. Finally, students find valuable the possibility to store and manage their courses information in digital format [16]. • Benefits for instructors. Many instructors have found that Tablet PCs not only can replace the black/whiteboard, but also extend the resources pool to be used during a lecture [6]. It makes the lectures more dynamic and interactive [Roschelle et al., 2007]. In addition, the annotations the instructor performs during a lecture can be easily stored and shared with the students. Several researchers report an increment of the students’ interest during the classes and also an improvement in the way used by the instructors to deliver knowledge [13, 16]. Based on the authors’ experience, many of the envisioned benefits are inherent to the use of laptops during lectures. Other benefits, for example the change of students’ and instructors’ attitude during the lectures, could be the result of the redesign of the teaching-learning process performed to take advantage of the Tablet PCs features. Of course, there are several benefits that really are a consequence of these devices usage, for example the possibility to make handwriting annotations on the lecture slides. Next section describes a preliminary study carried out in two traditional Chilean universities, which tries to determine the impact of Tablet PCs in computer science and engineering courses. The study also discusses the obtained results and compares them with the findings reported by other researchers in USA.
4 Empirical Study In order to try to understand the impact that the use of Tablet PCs could have in engineering education in Chile, the authors conducted an empirical study in two traditional
34
I. Casas, S.F. Ochoa, and J. Puente
Chilean universities: Universidad de Chile and Pontificia Universidad Catolica de Chile. This study was performed in computer science courses at the engineering school and it involved several types of Tablet PCs and also software products. 4.1 Experience in the Universidad de Chile This experience involved students of a computer science course (CC51A: Software Engineering) during two semesters: fall 2008 (29 students) and spring 2008 (20 students). Ten Tablet PCs were delivered among the students in order to support two main activities: course examinations using MOCET [9] (Figure 1), and group problem-solving using MS OneNote [10]. In addition, the instructor used Classroom Presenter [2] during half of the course lectures, and students using Tablet PCs took notes during lectures using this software. When the course was finishing, the instructor and the students filled a survey that evaluated the experience of using these devices. Some of the issues identified by the participants were the following:
Fig. 1. Use of MOCET at University of Chile
• The hardware matters. Students using particular models of Tablet PCs (DELL Latitude XT and HP Pavilion TX1000) consistently reported problems to write/erase annotations, regardless of the software they were using. In addition, these students also reported precision problems when these devices recognize the stylus location on the screen. • The working conditions matters. Students and the instructor identified the importance of avoiding situations where only some of the students had Tablet PCs. Some educational activities designed to take advantage of the Tablet PC features, for example the group design, were difficult to perform for students that did not have available these devices or a whiteboard. Therefore, if you are going to perform an activity that takes advantage of these devices’ features, be sure that all the students count with Tablet PCs. • Training is required. Students and the instructor reported an important improvement in the usefulness of the devices and the effectiveness of their work, once they were trained in the use of Tablet PCs. • The instructor mobility becomes reduced. Because the Tablet PC must be connected to the projector through a physical cable during a lecture, the mobility of the
Using Tablet PCs and Pen-Based Technologies to Support Engineering Education
•
•
•
• •
•
35
instructor becomes reduced. After some sessions the instructor started to use MaxiVista [7] as intermediary to recover the mobility. Although the solution was good, it required to have two computers in the classroom to deliver the lecture. Tablet PCs do not replace the blackboard. Although the Tablet PC’s functionalities are comparable to those of a whiteboard (and even better in some aspects), the information shown to the attendees through these devices is replaced when the instructor change the slides. In the case of the whiteboard, it acts as an extended screen of the instructor presentation; therefore it keeps visible the information written on it, even when the instructor changes the slides. In other words, students and the instructor think the whiteboard and the Tablet PC are complementary. Useful to make free-style annotations. Students and the instructor highlighted the capability of these devices to make free-style handwriting annotations. Several software products were used to carry out this activity; all of them were useful and comfortable for the users. They are similar to laptops in several aspects. Students and the instructor identified as an important contribution the capability of mobile computing devices to record annotations and manage information in digital format. They also highlighted the capabilities of these devices to share information and communicate among them in a simple way, even when the users are on the move. However, these functionalities are not only available in Tablet PCs, but also in laptops. Something similar occurs with the possibility to perform simulations, or execute or compile programs during the lectures. In other words, several of the advantages reported as result of the Tablet PCs usage are also present when using laptops. The instructor and the students feel comfortable using Tablet PCs. Users felt comfortable using these devices during the experiences. Although some training was required, the learning effort was worthy. Computing devices could be distracters during lectures. Students and the instructor agreed that mobile computing devices can be a distracting element during the lectures. If they do not have a clear role to play during the lecture, their use compete with the instructor’s speech. The redesign of the lecture style makes them more active. The participants in these experiences recognized the lectures became more active and participative. However, it seems to be a consequence of the lectures style redesign more than a result of the use of Tablet PCs.
4.2 Experience in Pontificia Universidad Catolica de Chile This section describes the experiences of introducing Tablet PCs in two courses at the Pontificia Universidad Catolica de Chile during 2007: “Computing Software Tools for Engineering” which is a workshop course (iic2100) and “Introduction to Programming” (iic1102). The main objective of the experiment was to motivate engineering students to learn and enjoy the art of problem solving and project team work supported by computers. In order to reach such a goal, we started to use Tablet PCs in the classroom and we modified the dynamic of the teaching-learning process in these courses. Tablet PCs were used in the classroom for modeling, problem solving and programming.
36
I. Casas, S.F. Ochoa, and J. Puente
Fig. 2. Use of OneNote at Pontificia Universidad Catolica de Chile
The course iic2100 was the subject of this experimentation during two terms in 2007. Forty students participated in each semester. Twenty one Tablet PCs (HP Compact TC4200) were delivered among the students. These devices were shared by the students and they were used to support problem solving during lectures (Figure 2). The impact of the Tablet PCs usage was measured based on opinion surveys from students, teaching assistants and professors. A similar experimentation process was conducted in the course iic1102: Introduction to Programming, during the Fall and Spring terms in 2007. One hundred students participated each semester. Twenty one Tablet PCs (HP Compact TC4200) were delivered among the students in order to support group problem solving during lectures. In spite of all the positive aspects of the use of technologies in the classroom, a few problems were observed; for instance, annoying delays at the beginning of some activities due to instability of the wireless network and software server (a technical assistant was present in the classroom to solve this kind of incidents). Problems also occurred with software tools and their different versions. It was also observed that at the beginning of the class some students had difficulties to turn on the Tablet PC or manage the stylus-pen, due to simple lack of knowledge (this problem was easily solved). Importantly, some students became distracted with the technology and engaged in Internet searching, chatting, playing or e-mailing, but after several sessions this distraction diminished. The students highlighted, as a positive factor, the possibility to solve problems during classes. They considered the use of Tablet PCs helped to make the lectures more active and interesting. However, they also identified limitations in the problemsolving process because not all of the students counted with one of these devices. Most of the students agreed that the use of technology motivated them. It is also important to note that in both courses there were no students considering Tablet PCs as an inhibitor factor; however they recognized that some training is required to take advantage of these devices features. They considered that Tablet PCs facilitated the discussions and the group work during lectures.
5 Discussion At the moment, the use of Tablet PCs in computer science courses seems to have positive effects. However, there is not a comprehensive and scientific study that
Using Tablet PCs and Pen-Based Technologies to Support Engineering Education
37
shows which are the real benefits of including this kind of technology in our courses. Although using mobile computing devices during lectures brings several benefits, it does not seem to be consequence of using just Tablet PCs. Our experiments have also identified that computing devices can become a distracter factor during lectures, if there is not a clear role assigned to them. These experiments also have shown that the use of Tablet PCs motivates to students and professors, however it is not clear if the use of laptops generates the same effect. Provided the courses involved in the experiments tried to take advantage of the Tablet PCs features, and following the recommendations of other researchers in the area, we redesigned their teaching-learning process. Now these courses are more active and there is more interactive work during the lectures, which produced a positive impact among the participants. However this result seems to be a consequence of the redesign process more than the inclusion of Tablet PCs. The experiences help us to identify that not all of the Tablet PCs provide an adequate support for these activities, and the mix of technology and paper-based work is not a good combination; particularly, if students have to use paper notebooks because there are not enough computers for everybody. Although the students recognized the importance of Tablet PCs for taking notes and facilitating group work, the use of these devices requires a training process. Instructors were happy when using Tablet PCs. They think these devices help to improve the quality of the lectures; however they are not able to replace the black/whiteboard. In one of the experiences, a lack of the instructor mobility was identified because of the use of Tablet PC to deliver the lecture. Such problem was then solved using additional software technology.
6 Conclusions and Future Work The use of Tablet PCs in engineering education seems to be growing more and more every day. Several researchers have reported experiences of the Tablet PCs adoption, mainly in USA. They have identified a list of benefits derived from the use of these devices. In order to try to understand the impact of this type of computers t on engineering education in Chile, two experiences were conducted at two traditional Chilean universities. The obtained results show some similarity with those obtained in American Universities. However there are others that are a bit different. The lack of resources for every student opens new challenges for the adoption of these technologies in developing countries. The experiments have also shown that several benefits associated to the Tablet PCs adoption are more a consequence of others activities around this situation. At the moment it is not clear which benefits are the direct consequences of the Tablet PCs adoption, however the balance seems to be positive. More experimentation and scientific work is required to understand its impact of this emerging technology.
Acknowledgements This work was partially supported by Fondecyt (Chile), grant Nº 11060467 and also by Hewlett Packard through the Technology for Teaching Higher Education Grant Initiative.
38
I. Casas, S.F. Ochoa, and J. Puente
References 1. Alvarado, C., Anderson, R., Prey, J., Simon, B., Tront, J., Wolfman, S., Tablet, P.C.: Computing Curriculum. In: Summary of the Tablet PC and Computing Curriculum Workshop, Seattle, USA, August 4 (2004) 2. Anderson, R., Davis, P., Linnell, N., Prince, C., Razmov, V., Videon, F.: Classroom Presenter: Enhancing Interactive Education with Digital Ink. IEEE Computer 9(40), 56–61 (2007) 3. Berque, D., Bonebright, T., Dart, J., Koch, Z., O’Banion, S.: Using DyKnow Software to Support Group Work: A Mixed-Method Evaluation. In: Prey, Reed, Berque (eds.) The Impact of Tablet PCs and Pen-based Technology on Education, pp. 11–20. Purdue University Press (2007) 4. Davis, K.M., Kelly, M., Malani, R., Griswold, W.G., Simon, B.: Preliminary Evaluation of NoteBlogger: Public Note Taking in the Classroom. In: Prey, Reed, Berque (eds.) The Impact of Tablet PCs and Pen-based Technology on Education, pp. 33–42. Purdue University Press (2007) 5. Herrera, O., Ochoa, S., Neyem, A., Betti, M., Fuller, D., Aldunate, R.: Mobile Portfolio to Support Communities of Practice in Science Education. In: Proceedings of HCI International 2007, Beijing, China (July 2007) 6. Huettel, L.G., Forbes, J., Franzoni, L., Malkin, R., Nadeau, J., Nightingale, K., Ybarra, G.A.: Transcending the traditional: Using tablet PCs to enhance engineering and computer science instruction. In: Frontiers in education conference (FIE 2007) (2007) 7. Bartels Media GmbH. MaxiVista (Last visit, December 2008), http://www.maxivista.com/ 8. Nguyen, H., Bilen, S., Devon, R., Wise, J.: Adopting Tablet PCs in Design Education: Student Use of Tablet PCs and Lessons Learned. In: Richards, G. (ed.) Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007, pp. 1172–1177. AACE, Chesapeake (2007) 9. Ochoa, S., Neyem, A., Bravo, G., Ormeño, E.: : MOCET: a MObile Collaborative Examination Tool. In: Proc. of HCII 2007, Beijing, China (July 2007) 10. Microsoft Corp. OneNote (2007), http://office.microsoft.com/es-es/ onenote/FX100487703082.aspx (Last visit December 15, 2008) 11. Prey, C., Reed, R.H., Berque, D.A. (eds.): The impact of Tablet PCs and Pen-based Technology on Education. Purdue University Press (2007) 12. Prey, C., Weaver, A.: Tablet PC Technology: The Next Generation. IEEE Computer 9(40), 32–33 (2007) 13. Price, E., Simon, B.: A Survey to Assess the Impact of Tablet PC-based Active Learning: Preliminary Report and Lessons Learned. In: Prey, Reed, Berque (eds.) The Impact of Tablet PCs and Pen-based Technology on Education, pp. 97–105. Purdue University Press (2007) 14. Sommerich, C., Collura, K.: Learning with Mobile Technology in high School: A HumanFactors Perspective. In: Prey, Reed, Berque (eds.) The Impact of Tablet PCs and Penbased Technology on Education, pp. 127–136. Purdue University Press (2007) 15. Toto, R., Lim, K., Wise, J.: Supporting Innovation: The Diffusion and Adoption of Tablet PCs in College of Engineering. In: Prey, Reed, Berque (eds.) The Impact of Tablet PCs and Pen-based Technology on Education, pp. 147–155. Purdue University Press (2007) 16. Tront, J.G.: Facilitating Pedagogical Practices through a Large-Scale Tablet PC Development. IEEE Computer 9(40), 62–68 (2007)
Optimal Affective Conditions for Subconscious Learning in a 3D Intelligent Tutoring System Pierre Chalfoun and Claude Frasson Département d’informatique et de recherche opérationnel, Université de Montréal, Montréal, Canada {chalfoun,frasson}@iro.umontreal.ca
Abstract. In this paper we take a closer and in-depth look at initial results obtained from a previous novel experiment conducted with a 3D subliminal teaching Intelligent Tutoring System. Subliminal priming is a technique used to project information to a learner outside of his perceptual field. Initial results showed great promise by illustrating the positive impact of the subliminal module on the overall emotional state of the learners as well as their learning performances. Indeed, since emotion monitoring is critical in any learning context, we monitored the physiological reactions of the user while they learned and while they answered questions. We present a detailed and precise look at the optimal affective conditions that set the best learners apart. We will also explain a most surprising finding: the positive long term impact of subliminal priming on the entire learning process. Keywords: optimal affective conditions, HCI, subconscious learning, 3D ITS.
perception or complete awareness to the task at hand [5-8]. In fact, the existence of perceptual learning without perception has been neurologically proven and accepted [9]. Furthermore, recent work has put forth the performance increase in performance when using a subliminally teaching Intelligent Tutoring System [10]. Yet, subconscious learning systems are still widely absent in the HCI community. We intend to investigate in this paper the optimal emotional state of learners when using a subliminal teaching ITS by stating two research questions. First, in learning to solve a problem in a 3D virtual system, is there a significant emotional state in which the best learners are that sets them apart from the rest? Second, in answering questions following a learning session, what significant relationship can we establish between learners’ emotional state and subliminal projections? The organization of this paper is as follow: In the next section, we will present and discuss the previous work related to various aspects of our research. The following section describes the experiment setup and depicts the various aspects related to subliminal stimuli in a virtual 3D tutoring system. The obtained results will follow the experiment section leading to the last section where we conclude and present future work.
2 Related Work To the best of our knowledge, only a handful of papers in various fields have claimed the use of subliminal priming as a support for memory in the HCI community. The first and most referred to is Wallace’s text editor program [11]. In this experiment, Wallace and colleagues put forward two important findings: (1) the projected stimuli must take into account the specifications of the computer such as screen resolution and refresh rate (2) that the frequency at which subjects requested help was much lower when the requested information was projected subliminally. The Memory Glasses by [5] used wearable glasses that projects subliminal cues as a strategy for just-in time memory support. The objective was to investigate the effect of various subliminal cues (correct and misleading) on retention in a word-face learning paradigm and compare recall performance. Another use of priming for memory support can be found in the thesis of [12] where the author assesses the effects of brief subliminal priming on memory retention during an interference task. Finally, our most recent work showed the positive impact of subliminal stimuli on the learner’s performance [10]. Besides seeming to impact memory, subliminal priming can also have an emotional consequence on learners. Indeed, subliminal priming can have an emotional impact on the self-attribution of authorship of events [13]. Subjects were asked to compete against a computer in removing non words such as “gewxs” from a computer screen in the fastest time possible. However, after a determined amount of time, the computer would remove the word. Subliminal primes of self-associated words like “I” and “me” before an action increased the personal feeling that it was the participant that eliminated the non word and not the computer, thus increasing the feeling of selfauthorship of events. Furthermore, visual subliminal stimulus has been neurologically proven to have an impact in many physiological signals, namely the galvanic skin response (correlated to arousal) [14].
Optimal Affective Conditions for Subconscious Learning in a 3D ITS
41
Since we also use physiological sensors to monitor the emotional reactions of the learner, it would be relevant to sum some of the work related to using physiological sensors to record and analyze emotions that can occur in a learning environment. Physiological signals are generally correlated with emotions by associating specific signals, such as skin conductance and heart rate, to valance and/or arousal [15]. Indeed, the Empathic Companion is a good example where multiple physiological sensors, namely galvanic skin response (also referred to as skin conductance), heart rate and respiration were taken in real-time to analyze and adapt the tutor to the emotional reactions of the learner in a virtual 3D ITS [16]. Further research has analyzed a more detailed and relevant emotional significance of physiological signals, either in complex learning or gaming [17-19].
3 Experiment The current experiment uses precise and timed subliminal projections in a 3D intelligent tutoring system while monitoring the physiological reactions of the learner. In the same time we record the actions on the screen as well as the facial movements of the learners. Those visual recording are crucial to remove noise and identify events of special interest. Moreover, we constructed the subliminal cues in a way which would accelerate the learning process by triggering and enhancing an already possessed knowledge. 3.1 Design of the Experiment Indeed, the focus of the experiment is to visually teach, in a virtual 3D environment, the construction of an odd magic square of any order with the use of neither a calculator nor one mental arithmetic operation. A magic square of order n is a square containing n2 distinct integers disposed in a way such as all the n numbers contained in all rows, columns or diagonals sum to the same constant. The first part of Fig. 1. below depicts such a square. Magic Square
Trick #1
Trick #2
Trick #3
Fig. 1. Experiment design : Magic square and the three tricks taught
To construct the following square, one must successively apply three simple tricks. These tricks are illustrated in Fig. 1 and labelled trick 1 to 3 respectively. We decided to show the learners multiple examples of each trick without explaining how the trick works. As an example, the first trick to construct any magic square is to place the following number one square above and two squares to the right of the previous one (exactly like a knight’s move in chess). If we look at the second picture of Fig. 1, we notice that number 15 is placed one square above and two squares to the right of
42
P. Chalfoun and C. Frasson
number 14. The same logic applies to numbers 1 and 2, 4 and 5 and so forth. Instead of giving away the answer to the first trick, we ask the subjects to deduce the rule by themselves. This is where the subliminal stimulus comes into play. We will have two groups, one group will take part of the experiment without subliminal stimuli (control group) and the tutor will subliminally send the answer to the other group. We will then compare performances, trick completion time, question completion time as well as physiological signal variations. The teaching material is separated into parts, or PowerPoint-like slides, and displayed at a slow rate to give every learner an equal chance at fully reading each “slide”. The subliminal stimuli and threshold were carefully chosen following the neural bases of subliminal priming [9]. Each stimulus was preceded by a 271 ms pre-mask of random geometrical figures, a 29 ms prime and a 271 post-mask of random geometrical figures. The subliminal stimuli that will be presented to one of the two groups will be displayed at significant places before and after specific slides. The experiment intends to “boost” learning by priming the answer before showing the corresponding slide. Fig. 2. shows a diagram of the way subliminal priming will take place between slide 1 and slide 2 when learning to deduce the inner working of the first trick.
271 ms
mask
29 ms prime 271 ms Slide 1
mask
Priming between slides
Slide 2
Fig. 2. Subliminal priming of the solution between 2 slides
The learners were instructed to answer a series of two to three questions following each learned trick to test their knowledge. The learners were instructed to finish the experiment as quickly and efficiently as possible. No time limit was imposed. A base line for the physiological signals preceded all monitored activities. A questionnaire preceded the experiment aiming at collecting demographical data as well as the gaming experience of the subjects. Another series of questions were asked at the end of the experiment to evaluate the learner’s appreciation and more importantly their overall appreciation of the system. 3.2 The 3D Virtual Environment Learning takes place in a game-like environment called MOCAS [20] as show in Fig. 3. The experiment has three rooms like the one illustrated on Fig. 3. Each room teaches one trick. MOCAS takes place in full-screen mode for a better immersion and less window distracting events. Furthermore, the system clock is hidden so users don’t get distracted by continuously monitoring the time they have spent on each lesson. The interactions between the avatar’s learner and the pedagogical agents are done via mouse clicks. This interaction is important because learners have a time
Optimal Affective Conditions for Subconscious Learning in a 3D ITS
43
window of 30 seconds to answer each question. If they feel that time was not enough, they can simply re-click on the agent and the question restarts. This re-click factor was important in distinguishing good from bad learners. The learners are instructed to continue once they are convinced they have discovered the inner working of each trick. They are then asked to answer a series of questions (two to three) by another set of visually different pedagogical avatars. Each question is related to the last trick learned. The agent asks the user to correctly place a number in a magic square. The learner responds by choosing the path that correctly answers the question. Physiological signals of the learners were also monitored in real-time and saved for further analysis. The used signals were heart rate, galvanic skin response, respiration rate and skin temperature. The signals are managed by the ProComp Infinity encoder [21].
Path leading to the Question Agents
Avatar of a pedagogical agent
Clicking on the Pedagogical agent starts the lesson.
Teaching material
Avatar of the learner
Fig. 3. 3D virtual learning environment
3.3 Learners Tested A total of 31 healthy volunteers, 16 men and 15 women, took part of the experiment. One participant had to be removed because of a major recording issue in the videosignal synchronisation module. The sample’s mean age was 26 (SD = 4.51). Only two volunteers had extensive video gaming experience. All the others gaming experience ranged equally anywhere from weak to moderate high. A repartition of the learners can be found in table 1. Table 1. Participants’ distribution
Group A : no subliminal stimuli Group B : primed with subliminal stimuli Total
Men 8 7 15
Women 7 8 15 30
44
P. Chalfoun and C. Frasson
4 Results The first aspect we wanted to examine was the existence, if any, of a relationship between affective variations and subliminal projections while learning the tricks. Fig. 4 shows the average quantitative affective variations of learners with regards to valence and arousal when learning all three tricks with and without the subliminal module. The signal used and correlated with valence is the heart’s inter-beat interval (IBI) and galvanic skin response was used and correlated with arousal (GSR) [15]. These signal values are normalized by mean-shifting, that is subtracting each signal’s value from the signal’s baseline mean then dividing the result by the signal’s standard deviation. This widely accepted technique enables us to compare learners’ results for it solves the problem of extra-individual physiological differences. Fig. 4 shows the average affective values for a period of 4 second following every subliminal stimulus. The full brown bars represent the average value of the signal for all subliminal stimuli at the precise moment the stimulus was projected (t=0s, s is for seconds). The horizontal dashed bars represent the same averaged value except for it’s computed for the 4 seconds following that projected stimulus (T=t + 4s). Since group A was not primed with subliminal stimuli, we placed markers for each learner at the precise moment where subliminal cues would have been projected if these learners would have been taking the course with the subliminal module. For example, the first four numbers from the left (-0.7, -0.7, 0.3 and 0.6) represent the following situation: on average, all learners in the experiment have had a normalized valence change of zero (-0.7 at moment t=0 and -0.7 after 4 seconds) when learning without the subliminal module compared to a normalized valence increase of +0.3 (0.3 at moment t=0 and 0.6 after 4 seconds) when learning with the subliminal module. Average Quantitative Affective Variations Of Learners While Learning The Tricks Group A
Normalized Mean-Shifted Values With Regards To Baseline
8.0
6.8 6.7
7.0
Group B
Group A
6.0 5.0
4.4 4.4
5.5
Group B
4.0
Group Group B A
3.6 3.2 2.9
3.0
Group B
2.0 1.0
Group A 0.3
Group B
1.7
Group A
0.6
Group B
2.5 2.5
1.0
0.8
Group A
1.1 0.5
0.4
0.0 Valence
-1.0
-0.7 -0.7
Arousal
Valence
All Learners
Arousal Best Learners
-0.9 -0.9
Valence -0.6 -0.6
Arousal Worst Learners
All 3 Tricks
-2.0 Valence And Arousal For All Tricks Broken Down By Best, All and Worst Learners
Without Subliminal Module - Average Value At t=0s
Without Subliminal Module - Average Value At T=t+4s
Fig. 4. Quantitative affective variations when subjected to subliminal stimuli while learning
Optimal Affective Conditions for Subconscious Learning in a 3D ITS
45
The results shown in Fig. 4 are not only statistically significant (p<0.001, α=0.05) but very important for they enable us to distinguish between the best and worst learners in terms of valence and arousal but also in terms of how much variation is considered optimal for success. In this case, having an average positive valence variation increase of about 0.8 and arousal increase between 2.5 and 2.9 is what our system should be looking for. In fact, we can clearly see at the far right part of Fig. 4 that the worst learners, those who made the most mistakes, were the ones who had a negative valence variation. Checking the results with Lang’s two dimensional space [15] informs us that a negative valence leads to a negative emotional state and thus not optimal for learning. Since subliminal projections increase valence variations, our system could then detect this negative emotional state and start projecting more stimuli until an optimal state is reached. The second aspect we wanted to investigate was the affective state of learners when answering questions. It is important to mention that no subliminal priming took place when answering questions. Group A and group B can then be compared without bias. Fig. 5 displays the quantitative affective variations when answering questions grouped by trick. The horizontal dashed bars represent the values for group B, that is the group projected to subliminal stimuli during learning. The results are surprising and the difference showed here is more than statistically significant for alpha is equal to 0.01 (p<0.001, α=0.01). Not only does the subliminal module help increase valence and arousal variations to optimal levels as discussed previously, but the effect seems to last throughout the experiment. Indeed, we can see that the best learners from group B are twice less stressed when answering all the questions and much less aroused than the best learners from group A. Affective Variations Of Learners (Valence And Arousal) When Answering Questions
Normalized Mean-Shifted Values With Regards To Baseline
14.0
11.9
12.0
10.0
9.5 8.2
8.0 6.9 6.3
6.3
6.0
5.5
5.3
5.4
5.2
3.9
4.0
3.2
2.0 -0.1
-0.3
-0.5
-0.6 -0.2
-0.1
-0.5
0.0 Valence
Arousal
-0.9 All Learners
-2.0
Valence
Arousal
-1.1 Best Learners
1st Trick Questions
Valence
Arousal
-0.9 All Learners
Valence
Arousal
-1.0 Best Learners
Valence
-0.9 All Learners
2nd Trick Questions
Without Subliminal Module
Arousal
Valence
Arousal
Best Learners
3rd Trick Questions
With Subliminal Module
Fig. 5. Affective variations of learners while answering questions
The results of the two previous figures give us an insight into the complex innerworkings of unconscious perception. It seems that positive arousal variations (between 2.5 and 2.9) and positive valence variations (between 0.6 and 0.8) seem to yield
46
P. Chalfoun and C. Frasson
the best results when answering questions. Furthermore, subliminal projections seem to produce a cumulative positive effect because of the observed results for the third trick. Indeed, the last trick is the most difficult because it requires the use of the first two, henceforth the very high arousal variations for group A. We demonstrated in [10] that the subliminal module helped reduce dramatically the number of mistakes made. Fig. 5. has just explained why in terms of valence and arousal. Impact Of Affective Variations On Perfomance When Answering Questions 80.0 73.0
70.0 61.9
60.0 49.1
50.0 41.0
40.0 34.0
41.5
41.0
39.0
35.0
33.8
30.0
25.0
24.9
20.0
17.0 12.0
10.0
13.0 10.0
9.0
7.0
0.0 1st Trick Questions
2nd Trick Questions
3rd Trick Questions
All Learners
Total Reclics - Without Subliminal Module Total Errors - Without Subliminal Module Average Completion Time (s) - Without Subliminal Module
Total Reclics - With Subliminal Module Total Errors - With Subliminal Module Average Completion Time (s) - With Subliminal Module
Fig. 6. Impact of the affective variations on results, broken down by trick
The affective variations examined throughout this section lead us to carefully examine the effect on performance when answering questions of the third trick. Indeed, we can surmise that a learner can correctly construct a magic square if he successfully answers all questions but more specifically the third trick questions. Fig. 6. displays the results of our analysis by comparing group A’s results with group B’s with regards to total re-clicks (number of times learners asked to restart the question), total errors and more importantly completion times. The results present another argument for the combination effect of subliminal stimuli. Indeed, we can see that the total re-clicks for group B is almost the same as group A’s for the first trick but dramatically decreases as the subliminal projections increase throughout the experiment. The same phenomena can be observed with errors and completion times. The last aspect, completion times, is very important because it represents the foundation of what we are aiming at: faster and more efficient learning. We are very pleased with the results because they clearly show the significant contribution of the subliminal module (p<0.001, α=0.05) to the dimension of performance as well as emotions.
5 Conclusion We presented in this paper the optimal affective conditions for subconscious learning in a 3D virtual ITS. Subconscious learning is done with the use of carefully
Optimal Affective Conditions for Subconscious Learning in a 3D ITS
47
engineered subliminal projections aiming to accelerate and enhance learning. We also illustrated the importance that subliminal stimuli can have on the long term process of learning and pattern recognition. In contrast to the previous work regarding the use of subliminal priming and emotions, our work differs in three ways. First, the subliminal priming was used in an HCI to teach a more complex lesson than simply face recognition or item association. Second, in contrast with the vast majority of researchers, we discussed and placed forward the use of quantitative, and not just qualitative, affective variations in building an effective HCI to detect the best learners from the rest. Third, we placed forward the long-term impact that subliminal cues can have on the entire learning process and not just simply on simple tasks such as face memorizations or item pairing. As future work, we intend to use these results to construct an intelligent real-time HCI system that detects the emotional variations of the learner and adjusts the subliminal projections accordingly. Acknowledgments. We acknowledge the support for this work from the Fond Québecois pour la Recherche sur la Société et la Culture (FQRSC).
References 1. Villon, O., Lisetti, C.L.: A User-Modeling Approach to Build User’s Psycho-Physiological Maps of Emotions using Bio-Sensors. In: IEEE International Symposium on Robot and Human Interactive Communication, Session Emotional Cues in Human-Robot Interaction. Human-Robot Interaction, United Kingdom (2006) 2. Damasio, A.: Descarte’s Error - Emotion, Reason and the Human Brain. Putman Press, New York (1994) 3. Blanchard, E., Chalfoun, P., Frasson, C.: Towards advanced Learner Modeling: discussions on quasi real-time adaptation with physiological data. In: 7th IEEE conference on Advanced Learning Technologies: ICALT 2007, Niigata, Japan (2007) 4. McQuiggan, S.W., Lester, J.C.: Learning empathy: a data-driven framework for modeling empathetic companion agents. In: International Conference on Autonomous Agents, Hakodate, Japan (2006) 5. DeVaul, R.W., Pentland, A., Corey, V.R.: The Memory Glasses: Subliminal vs. Overt Memory Support with Imperfect Information. In: IEEE International Symposium on Wearable Computers. IEEE Computer Society, New York (2003) 6. Dijksterhuis, A., Nordgren, L.F.: A Theory of Unconscious Thought. Perspectives On Psychological Science 1 (2006) 7. Watanabe, T., Nanez, J.E., Yuka, S.: Perceptual learning without perception. Nature 413 (2001) 8. Nunez, J.P., Vincente, F.D.: Unconscious learning. Conditioning to subliminal visual stimuli. The Spanish Journal of Psychology 7 (2004) 9. Del Cul, A., Baillet, S., Dehaene, S.: Brain Dynamics Underlying the Nonlinear Threshold for Access to Consciousness. PLoS Biology 5 (2007) 10. Chalfoun, P., Frasson, C.: Subliminal Priming Enhances Learning and Performance in a Distant 3D Virtual Intelligent Tutoring System. In: AACE World Conference on Elearning in Corporate, Government, Healthcare, & Higher Education: E-LEARN 2008, Las Vegas, Nevada (2008) 11. Wallace, F.L., Flaherty, J.M., A. K.G.: The Effect of Subliminal HELP Presentations on Learning a Text Editor. Information Processing and Management 27 (1991)
48
P. Chalfoun and C. Frasson
12. Schutte, P.C.: Assessing the Effects of Momentary Priming on Memory Retention During an Interference Task. Computer Science, Vol. Master Of Science. Virginia Commonwealth University, Virginia (2005) 13. Dijksterhuis, A., Preston, J., Wegner, D.M., Aarts, H.: Effects of subliminal priming of self and God on self-attribution of authorship for events. Journal of Experimental Social Psychology 44 (2008) 14. Tranel, D., Damasio, A.: Knowledge without awareness: an autonomic index of facial recognition by prosopagnosics. Science 228 (1985) 15. Lang, P.J.: The emotion probe. American Psychologist 520 (1995) 16. Prendinger, H., Ishizuka, M.: The Empathic Companion: A Character-Based Interface That Addresses Users’ Affective States. Applied Artificial Intelligence 19 (2005) 17. Conati, C.: Probabilistic assessment of user’s emotions in educational games. Applied Artificial Intelligence 16 (2002) 18. DiMello, S.K., Taylor, R., Graesser, A.C.: Monitoring Affective Trajectories during Complex Learning. In: Trafton, D.S.M.J.G. (ed.): Proceedings of the 29th Annual Cognitive Science Society, Austin, TX (2007) 19. Picard, R., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Transactions Pattern Analysis and Machine Intelligence 23 (2001) 20. Blanchard, E., Frasson., C.: Easy Creation of Game-like Learning Environments.: Workshop on teaching with robots and agents. In: conjunction with ITS 2006, Jhongli, Taiwan (2006) 21. Thought_Technology (2008), http://www.thoughttechnology.com
Computer-Based Learning to Improve Breast Cancer Detection Skills Yan Chen1, Alastair Gale1, Hazel Scott1, Andrew Evans2, and Jonathan James2 1
Applied Vision Research Centre, Department of Computer Science, Loughborough University, United Kingdom 2 Nottingham Breast Institure, Nottingham City Hospital, Nottingham, United Kingdom {Y.Chen3,A.G.Gale,H.Scott}@lboro.ac.uk
Abstract. In breast cancer screening it is important both to improve and maintain cancer detection skills at their highest levels. The introduction of digital imaging enables computer-based learning to be undertaken outside breast screening centres using a range of different devices. The potential for providing computer-based interpretation training using low-cost devices is detailed. The results demonstrated that naive observers can be trained to recognise certain key breast cancer appearances using a low cost display monitor along with a range of HCI techniques. Keywords: mammogram interpretation, training, eye movement, visualization, Human-Computer Interaction (HCI).
nationally. Additional training for individuals which targets such known discrepancies can then be proposed which should help in maintaining skill levels of cancer detection. Currently most training in the interpretation of screening mammograms needs to be undertaken where there is a mammo-alternator (a large viewing device, where several hundred cases can be loaded, which comprises a back-illuminated surface on which numerous mammographic cases can be presented simultaneously for examination) or other suitable light box (where only one single mammographic case can be presented at a time) on which to view the mammograms. This then limits the locations and availability where training can take place. However, digital breast screening, where images of the breast are digitally acquired and the resultant images examined using very high resolution and expensive dedicated workstations in breast screening centres is gradually being introduced nationally. Typically these workstations have a dual monitor set up where each has a resolution of 5 Mega pixels (2048 x 2560) and is capable of displaying 10-bit greyscale images with a high contrast ratio. Each monitor can display various combinations of mammographic views and is used primarily to display a single mammogram which can then be zoomed into and manipulated. The widespread adoption of such digital technology renders additional training opportunities using other less expensive and lower resolution displays and in various locations possible for certain purposes if suitable Human-Computer Interaction (HCI) techniques can be derived which are acceptable to radiologists. Primarily when examining digital mammographic images for diagnostic purposes then an appropriate workstation is fundamentally required. However it may be possible to target training of key mammographic appearances using other lower resolution displays which would make web delivered training using a range of viewing devices both potentially feasible as well as acceptable. This research investigates the utility of employing low-cost devices to provide individualised training with the support of HCI techniques and then determines what types of image interaction and manipulation techniques are required by potential end users. It was hypothesised that using a lower resolution display would hamper the identification of small signs of breast cancer (e.g. microcalcifications) but that larger signs (e.g. masses) would still be visible. Different HCI approaches were used as an initial approach to training based on facets of typical routine clinical training or reporting. In particular, the possibility of using aspects of visual search (as an HCI technique) to assist in mammographic interpretation training is explored.
2 Methods Firstly, a questionnaire was sent to all (601) UK screening personnel to determine their satisfaction with current training opportunities and their views on the potential for individualised training as well as on the use of other image display devices for examining the high resolution mammographic images. This included details of current screening (e.g. profession; experience of digital mammography) and the individual’s current usage of mammographic interpretation training (e.g. forms of training available; the amount of training opportunities; any difficulties of current training; advantages and disadvantages of current training).
Computer-Based Learning to Improve Breast Cancer Detection Skills
51
Secondly, as a ‘worst case’ scenario investigation twenty experienced breast screening radiologists were shown a series of mammograms on a small PDA (screen size 3.5”) and questioned as to whether they thought such displays could ever be used for training purposes in breast screening and if possible then what types of HCI image manipulation techniques would they envisage requiring (Figure 1).
Fig. 1. A radiologist views mammograms (MLO views) on a 3.5” screen PDA
An experimental investigation was then undertaken which examined different aspects of computer based training for breast screening interpretation. Initially, both the eye movements and the audio description of an expert breast screening radiologist were recorded whilst a series of recent screening cases were examined. From these eye movement data key fixations were identified (based on visual dwell measures) which fell on both abnormal areas as well as other normal image areas which attracted visual attention. Twenty naïve participants were then first given a short standardised introductory computer-based presentation about breast cancer; mammograms and the appearances of two key mammographic features; masses and calcifications. These two features were deliberately chosen as whilst both can be difficult to detect on a mammogram, masses are generally relatively large and calcifications are fairly small irregular abnormalities which can appear singly or in clumps and of various size. This was then followed by an image examination exercise. During this, each participant was required to examine 21 recent mammographic screening cases (7 normal cases, 7 cases containing a mass and 7 cases containing calcifications) on two occasions. Their performance in identifying abnormal appearance was measured as they first examined these cases, then they either undertook a short training exercise or were assigned to a control condition and then re-examined the cases. On both occasions the participants were asked to identify whether each case contained
52
Y. Chen et al.
either of the two features. If they thought an abnormality was present then they also had to indicate its location. Participants’ eye-movements were recorded throughout. The different forms of training used were: whole mammogram presented but with regions of interest highlighted; only regions of interest (where only a magnified portion of the mammographic image around a potential abnormality site was presented); playback of the expert’s visual search behaviour, and playback of the expert’s audio instruction on each case. Participants Twenty naive observers participated (seventeen research students in various subjects, and three university employees) were involved in the study. None of them had any experience in mammography reading. Participants were split into four experimental groups and a control group (without undertaking any training) with the experimental groups undertaking different forms of computer based training. Materials Visual stimuli: 21 sets of recent digital mammographic images were used. Each image set comprised both the Medio Lateral Oblique (MLO) views and Cranio Caudal (CC) views of both breasts. Fourteen of the pairs featured a specific abnormality which had been grouped into two types (namely: Mass and Calcification) with the abnormality visible on either one or both views. Seven of the sets featured no abnormality.
Fig. 2. Schematic of the different training and control approaches
Computer-Based Learning to Improve Breast Cancer Detection Skills
53
Hardware: The experiment was run on a PC with a LCD monitor (display size: 21.5”; resolution: 1680 x 1050 ) for displaying images; eye movements were recorded using a remote oculometer – a Tobii X50 eye-tracker. This records eye movements with no attachment to the participant and allows them some degree of free head movement (within 30 x 16 x 20 cm ; W x H x D) at 60 cm from the device. It has a reported accuracy of 0.5-0.7 degrees of visual angle. Design Training sets: each training set included the 21 cases comprising both MLO and CC views of each case but were presented in four different formats (Figure 2). These formats were: 1. T1 – whole image: the MLO views of both breasts were presented then, where appropriate, the feature area was highlighted by a circle with descriptive text, followed similarly by the CC views. 2. T2 – image with portion: the MLO views were presented (as in format T1) but then the area of interest around the abnormality was highlighted followed by this area being magnified and shown alone (size: 8”x 8”). This was followed similarly by the CC views. 3. T3 - eye movements: the MLO views were shown and then overlaid with annotated fixation locations of where the initial expert radiologist had looked and in the order in which visual search had been performed. Also, the area of abnormality was highlighted. This was followed similarly by the CC views. 4. T4 - comments: This was similar to format T1 above but with the addition of the expert’s audio descriptions concerning the case. 5. control: a control activity was undertaken which took the same length of time as the other training sessions. Procedure For each participant the eye tracker was first calibrated. Each was then given a short introduction concerning breast cancer, mammogram images, and were familiarised
A
B
Fig. 3. A shows a participant taking the test viewing the MLO mammograms of a case.; B shows a participant undertaking the audio training whilst viewing the CC images of a case.
54
Y. Chen et al.
with the appearance of the two different key breast cancer features in such images (masses and calcification). They then visually examined two practice cases. Subsequently they completed a computer-based image examination task whilst their eye movements were discretely recorded (Figure 3). During this task the participants first fixated on a small centrally presented fixation cross and then this was replaced with the MLO views of the case which in turn was followed by the CC views. Participants were asked visually to examine each case view, identify whether the view was normal or contained an abnormality. If the latter, they also had to specify the feature type (i.e. mass or calcification) and its location.
3 Results For the questionnaire study some 273 questionnaires were returned (a 45% response rate) which covered the main professions in breast-screening film reading, e.g. 152 consultant radiologists, 78 advanced practitioners (in the UK these are specially trained technologists), and others. These all indicated that screeners considered that whilst existing training was adequate they would like to have individualised and targeted training which is only really possible using digital images. The responses concerning the most useful mammographic interpretation training type is shown in Figure 4.
No.PPT consier it 'Most Useful'
100 90
Review of individual film reading data Arbitration/consensus PERFORMS MDT's Interval cancer review Discussion/interaction Radiology meetings/sessions CPD course Conference
1 2 3 4 5 6 7 8 9
80 70 60 50 40 30 20 10 0 1
2
3
4
5
6
7
8
9
Training Type Fig. 4. Participant’s response on ‘The most useful training type’
‘Interval cancer review’, ‘Arbitration/consensus’ and ‘PERFORMS’ were considered to be the fist three most useful. According to the characteristic of these three training types, it shows that being able to read a large amount of images followed by other forms of information, such as, expert’s opinions on the case, other people’s feedback on the case, etc. were considered to be the key useful training approaches.
Computer-Based Learning to Improve Breast Cancer Detection Skills
55
25% % of All correct answer
T est Session 1 T est Session 2
20% 15% 10% 5% 0% Visual
Audio
Control
Training type Fig. 5. Percentage of both correct answers (feature & location) of the training types (both test sessions)
Test session one Test session two
Fig. 6. Mean performance values of each participant group (both test sessions)
56
Y. Chen et al.
With regard to viewing images on the PDA, initially all participants were highly sceptical as the display had approximately 1/10th of the resolution of a digital mammographic workstation but it was determined that such a display could be useful for training (but not for diagnostic) purposes if suitable simple HCI techniques were available which would enable rapid image pan and zoom with contrast level manipulation. In the experimental study of naïve observers some 880 responses were collected and of these only 124 correctly identified both features and locations on both views of a case. Initially the training types were considered as either visual (whole image, portion, eye movements), audio (comments) or control - details are shown in Figure 5. In audio, performance decreased after training whereas for both the visual and control conditions there was a slight, non significant, increase. The performance of these participants in the first and second test was then attributed points concerning whether they correctly identified location and features on both views (Figure 6) where a score of 4 indicates identifying both the correct location and feature on both views. Most participants scored 1-1.5 indicating that they only correctly scored either location or feature in either view. In the three visual conditions there was a slight increase in performance in test 2, in the audio condition performance decreased and in the control condition there was little variation. In all conditions participants were significantly better in identifying features correctly than in identifying correct location (p<.05). The performance in identifying features between test 1 and test 2 significantly improved (p<.05). In terms of mammographic features then masses were better identified after any visual training and only slightly increased after audio training. Calcifications were detected worse after all three types of visual training as well as the audio training. In the control condition calcification identification improved. Normal cases were reported worse on every second trial. In the control group the correct identification of normal cases dropped on the second test but mass and calcification detection increased. In the audio condition calcification and normal identification fell on the second test but mass identification improved slightly. In the three visual conditions only mass identification improved.
4 Discussion From the returned questionnaires, these all indicated that screeners considered that whilst the existing training was adequate they would like to have individualised targeted training [5]. With regard to viewing images on the PDA then for this to be at all useful, easy to use HCI techniques which would allow rapid image manipulation were clearly required [6]. Using a PDA for some training purposes would be possible if this was done in a appropriately lit environment with a low ambient light level. The experimental study reported here was an initial investigation utilising a single low cost monitor to deliver mammographic interpretation training as compared to using one or two high resolution workstation monitors. Different training regimes were developed which presented observers with images enhanced using different HCI approaches which were considered suitable in this domain. The approaches used were firstly simply showing the full views of both breast simultaneously which mimicked the display on a workstation but at a much lower resolution. Secondly the full view
Computer-Based Learning to Improve Breast Cancer Detection Skills
57
was show followed by a magnified view of the area of interest to mimic basic image manipulation on a workstation. The third approach taken was to utilise areas of interest which attracted the visual attention of an expert together with the scanning path which were overlaid on the image. The fourth approach tried to replicate an expert describing how they examine a case for abnormalities which is a commonly adopted approach in real screening as shown in the questionnaire survey. The study set out to use short training approaches to examine their effects on naïve observers. This demonstrated that such observers can be trained to recognise certain key breast cancer appearances using a low cost display monitor along with a range of HCI techniques. Two mammographic appearances were studied; small calcifications because these can be difficult to detect in routine breast screening and larger masses. Calcifications were not detected well presumably due to the shortage of image manipulation techniques used here. Naïve observers were used in this study to see how they responded to the different training types.
5 Conclusion Overall, it is argued that these findings taken together indicate that low cost devices can be used for training purposes in digital breast screening with appropriate HCI techniques. These then extend the opportunity for training beyond the clinical workplace. Ongoing research is further investigating the delivery of computer-based training using a variety of HCI methods to breast screeners.
References 1. World Health Organization “Cancer”, Fact sheet 297 (2006) (Accessed: 01 March, 2009) 2. National Statistics Online, Breast Cancer: Incidence rises while deaths continue to fall, Office for National Statistics (2007) (Accessed: 01 March, 2009) 3. Gale, A.G.: PERFORMS-a self assessment scheme for radiologists in breast screening. Seminars in Breast Disease 6(3), 148–152 (2003) 4. Gale, A.G., Scott, H.J.: Measuring Radiology Performance. In: Michell, M. (ed.) Breast Screening. Contemporary Issues in Cancer Imaging – Breast Cancer. Cambridge University Press, Cambridge (in press) 5. Chen, Y., Gale, A.G., Scott, H.J.: Mammographic interpretation training profile in the UK: current difficulties and future outlook. In: Manning, D., Sahiner, B. (eds.) Image Perception, Observer Performance, and Technology Assessment. Proceedings of SPIE (in press) 6. Chen, Y., Gale, A.G., Scott, H.J.: Mammographic interpretation training: how useful is handheld technology? In: Manning, D., Sahiner, B. (eds.) Image Perception, Observer Performance, and Technology Assessment. Proceedings of SPIE, vol. 6917, pp. 691712– 691712-10 (2008)
Virtual Classroom and Communicability: Empathy and Interaction for All Francisco V. Cipolla Ficarra1,2 HCI Lab. – F&F Multimedia Communic@tions Corp. ALAIPO: Asociación Latina de Interacción Persona-Ordenador 2 AINCI: Asociación Internacional de la Comunicación Interactiva Via Pascoli, S. 15 – CP 7, 24121 Bg, Italy [email protected] 1
Abstract. We present the main empathy components in the design of interactive systems aimed at classroom and E-learning education. These components have a bidirectional relationship with communicability and usability. Each of them depicts an intersection of communication, semiotics, interface design, software engineering, usability engineering and human-computer interaction. Additionally, we present a table which can be used as a communicative quality guide. Its content is the result of 20 years of design and heuristic assessment of on-line and off-line interactive systems, mainly for Americans and European users. Keywords: Education, Virtual Classroom, Hypermedia, Communicability, Design, Information, Empathy, Interaction, Accesibility.
Virtual Classroom and Communicability: Empathy and Interaction for All
59
phones in Italy, for instance, and yet they keep on demanding more functions), multimedia systems are based on partial design models, with primitives not unanimously accepted in the computer context, of human-computer interaction, usability, etc. [5]. Consequently, the lack of a unique model causes in many cases time loss of human resources. The only way to solve these problems is by resorting to basic notions of the historic concepts within international circulation. In the current work there is a brief description of the state of the art at the beginning, then follows the eradication of ambiguities in the essential notions of communicability, information and communication. Later on the empathy and the interaction is analyzed from a diachronic perspective with examples and finally an heuristic table is presented for the first assessment of the empathy applied to the educative and interactive systems, on-line and off-line.
2 Eliminating Ambiguity in Hypermedia Systems It is important to carry out in some cases the anchoring operations of the relationship of signification and significance of the terms, as is the way in semiotics framework and linguistics [6]. For instance, in the later stage of the evolution of interactive multimedia systems, we have the notion of hypermedia, which is an hybrid word between hypertext and multimedia. Here are assembled the advantages of both technologies inside the multimedia communication process. As has been observed in the origins of hypertext, where the textual aspects of the first systems prevails (including static graphics, in a broad sense), where it is established the associative character in the structure of information (these aspects generate a less frequent denomination in hypermedia systems, such as is the case of the electronic book). Whereas in multimedia and through the intersection of media there is a dynamic content of information: video, computer and audio animations. This dynamic component entails the time or synchronization factor among diverse means, which in our case we call panchronism. (From Greek ‘pan’ meaning all and ‘chronos’ meaning time [5]. A panchronic approach to, say, language is an approach including every aspect or all dimensions of time. Measurable quality criteria is very important in the communicability of on-line and off-line contents of hypermedia systems [7]. Consequently, hypermedia in a classical sense of the notion allows: selected access to those parts determined beforehand by the user, a greater degree of detail in the structure of information, that is to say, it is possible to resort to the richness of the content, in regard to several means used in the transmission of messages and reinforcing the communication process [8]. For instance, it is present in the use of CD/DVD-Roms and the DVDs for education, commerce, tourism, the ecological and cultural patrimony of many cities, regions or countries of our planet, etc. As a side note, in our projects in this area, until now we have undertaken our research only with users without physical disabilities. We hope and wish to include a wider range of users to study hypermedia systems in our future research. Along the evolutionary path of hypermedia, it can be said that it has become an interactive extension of multimedia. Consequently, this is the reason why the notion of multimedia is used indistinctly with that of hypermedia in many research works,
60
F.V. Cipolla Ficarra
although the relationship between the signification and the significant may be not strictly symmetrical according to Ferdinand de Saussure [6]. In hypermedia the synchronism of the active and static and dynamic means is essential. Besides, the amount of interaction required by the system and control of the fruition of the user over the system are two quality criteria for making assessments in current hypermedia systems [7]. 2.1 Communication, Information and Communicability In the context of the factual and formal sciences there are plenty of theoretic and practical works which talk about communication [9]. In this research it is easy to observe several definitions, where there are common elements among them. However, the momentum of verbose communication in the daily and colloquial environment has generated a sort of deficiency in its scientific sense, that is to say, a terminological status is missing. This deficiency becomes even more obvious when we talk about communicability in multimedia systems. Therefore, we will try to rank different points of view on both notions. In the case of communications, inside the context of the social sciences and more specifically in the area of social communication, some experts traditionally made a study of the differences between communication and information (Abraham Moles, David Berlo, Frank Dance, Luka Brajnovic, Rau Birdwhistell, Wilbur Schramm, etc. [10]). Others, in contrast, exclude the notion of information. Currently in the design of interactive systems the word communication is linked to contents, that is to say, the communication of contents. A short analysis of the word communication from its genesis leads us to the noun “communication” and the verb “communico”. Both have their origin in the word “communis” formed by cum (with) munis (duties-lins) [11], entailing implicitly the meaning to unite, to bind, to link to, that’s to say, there is a relationship with another person or system, in our case. There is an implicit dynamic process. For instance, through the relationship with the stored information in multimedia on-line and off-line systems, human beings can expand their knowledge. Regarding this, Schramm [6] considers communication as a real established relationship which consists in the discovery of the “I”, the “other” and “others”, and a donation of content which entails a duality of terms between emitter and receptor, who coexist in a contained environment and which is the foundation to its corresponding process. Within the notion of an emitter, we understand it to be the designer of multimedia systems, whereas the receptors are the potential users of that interactive system. Now, Schramm shows the need of cultural understanding between both to obtain optimal results in the communication process. On his side Luka Brajnovic tells us about communication and information differentiating each one in the following way: “communication consists in getting in touch with two or more people, things or bodies, in their different combination possibilities and its meaning can be manifold, and have diverse procedures and effects” [10]. It is a sort of direct channel or creative encounter that ties presences and distances, sometimes, without the informational goal, although it can be a vehicle of information. Therefore, information can be included in communication. When the information is incorrect there is no feedback, in the interactive communication process, for instance, because human communication is a psychosocial process.
Virtual Classroom and Communicability: Empathy and Interaction for All
61
Moreover, if we consider that communication is a psychosocial phenomenon, we must admit that this phenomenon is a given in the human being with all his rational and creative possibilities to organize the message and interpret its reception. That is to say, that from the point of view of social communication, it is not a simple direct channel of relationships, but rather a socialization process. Therefore, communication is a social fact, updated by the human being, through a process among which the integrants who make it up, where the interchange of experiences entails the enrichment of the participants, through the internalization of the messages that have been expressed in a given space and time. Among human beings there is a will to encounter in communications and a desire to have or create something in common. Spontaneous binding achieves the integration of man with nature, cultural patrimony, the community he inhabits, etc. We not only communicate data and facts but additionally we can communicate our ideas, experiences, feelings, and real or imaginary events, the objective and the subjective, the presence of two or more people in their different combinatory possibilities. Once we have quality in communication we are faced with the notion of communicability. That is to say, that communicability automatically includes quality. In our case, we sometimes use both notions simultaneously and redundantly to strengthen the idea of quality in the process of interactive communication. However, communication and communicability are two synonymous terms in the current multimedia theoretical framework. In the design of interactive multimedia systems it is easy to detect the presence of communicability, although as happens with the notion of beauty, it requires time to describe each one of the elements that make it up. In short, it is starting from this interrelation “cum” technological and “cum” the individual when the relationship in other research environments take place, such as the cognitive models used in the development of interfaces. At the moment of the design of an interactive system, the designer must consider the cultural factors, the types of users, the geographical location of the system, etc. [12], [13], [14], [15]. These variables mean that since the software sector at the start of the nineties the need of incorporating sociologists, anthropologists and sociologists has been affirmed to improve the quality of interactive systems. However, in the new millennium it is necessary to talk of experts in communicability [7].
3 Empathy and Interaction: Interactive Design We can define the empathy in the interactive design as the interactive systems designer's mental ability to put himself in the shoes of the potential user. It is the result of the triad confirmed by the cultural knowledge, mental ability to occupy the place of the other in the communicative process and the competence in advancing the user's behaviour in front of certain situations [16], [17]. In multimedia design traditionally we talk about cognitive models, that is to say, the solution would be to frame it in the psychological context. Obviously, it is a valid alternative for the first hypertext and multimedia systems in the late eighties and the decade of the nineties. With the advent of the use of information networks, whether it is Internet or extranet from international entities, since the late nineties it has been a matter of communicability. A communicability which stems from the design process in the
62
F.V. Cipolla Ficarra
interactive systems and is translated to its usability. If we analyze some multimedia products aimed at the education of the nineties, we can find how in the design of their structure one resorts continuously to two quality attributes such as are prediction and self-evidence [16]. A priori, prediction and self-evidence can seem similar, but it is not so. In selfevidence, the navigation of pages with dynamic elements (i.e. audio, video, animation, etc.) and the structure of the system can be anticipated by the user from the first moment, even if the user has scarce experience in the use of hypermedia. On the contrary, in prediction the user must have previous ability in order to navigate efficiently and overpass complex situations, after having previously navigated the hypermedia system. In prediction there is more previous experience in the navigation or knowledge of structural parts. Nevertheless, in some hypermedia, where the designers resort to reusability of information, prediction and/or self-evidence can be prejudiced if the behaviour of the dynamic media is different according to the place that it has in the structure. For example, in some sites the user can interrupt the animation and go to another page of the structure, while in others he cannot do so until all the animation is finished [18]. These two attributes are related to the concept of isotopies inside the context of communication. That is to say, those elements that must be maintained continuously in each one of the design categories to favour the interaction of the users with the content of the multimedia system [18] [19]. For instance, the location itself of the navigation keys in the different screens. The same modes of activating and disactivating the dynamic means, the synchronization between the audio and the images in movement, regardless of whether they refer to a video or an animation, etc. The presence of the isotopies in the interactive design indicates a high degree of empathy towards the potential users. In the first multimedia off-line systems it is observed a certain local or national empathy at the expense of international empathy. The reason of this failure in the internationalization of the multimedia contents lies in the fact that these systems counted with a design team for human factors, where would be located the empathy with psychologists and anthropologists. The origin of this deformation in communicative empathy and in the future of communicative evolution lies in software engineering. Some authors in the nineties maintained that in order to increase the quality of the software it was necessary to include some representatives of the social sciences, referring only to psychologists and anthropologists [20]. Obviously, some commercial products of multimedia software have followed these rules but they have forgotten the social communicators, whose training is in many cases an intersection of the fact and social sciences. Before making these statements to increase the quality of the software it is necessary to know beforehand the university study plans which are usually different in each of the continents in our planet. It is through the social communication professionals that a bigger international empathy is reached, which makes easier the circulation of on-line multimedia content. Throughout the history of interactive systems interfaces there have been some works that have made apparent the need to overcome cultural barriers among the different international users and the designer.
Virtual Classroom and Communicability: Empathy and Interaction for All
63
These works refer to the design categories presentation and content of the hypermedia systems. The acceptance of these premises of the interfaces design are related to the operative systems of the computers, that is to say, the interface and the PC Macintosh icons are not the same as compared with a PC that works with the last Windows version. In this regard, the contribution made regarding compatibility matters, such as the meaning of icons, that has been undertaken by manufacturers of operating systems in order to increase the empathy and interactivity with the multimedia systems, should be praised. Although some icons and functionalities of these have changed in the last years, this does not mean that they are objects of criticism because from the point of view of their usability and communicability it is necessary to consider them from a diachronic view. To make an heuristic and synchronic study of these; without considering the temporal variable only mistaken results are obtained. Making a diachronic study of the different interfaces used for the teaching of languages and those aimed at the virtual campuses, we can see how the interactivity increases during the teenage years in off-line systems, for instance. The interactivity with the system is related to the designer's empathy. It is for this reason that in the categories of presentation and content of the components of the interfaces for children we include the use of primary colors, sounds and melodies that draw the attention, characters from tv series, etc. [21] [2]. The more that the user's age increases, the designer will increasingly try to immerse him/her in real life situations to go deeper into the knowledge acquired in the learning process of a language. The relationship between user and computer shows some variables during interaction [22]. These variables are related among the individual user, the interactive system, and the context. In the first case, we have to consider elements like age, physical condition for autonomy of interaction, education or previous experience with multimedia systems and the aims of fruition, above all. In the second case, we have to consider access to information and the kind of support for the information, basically. The last variable is the time to fruition through the use of an interactive system.
4 Lessons Learning In the design process of a multimedia system the aim is to communicate content. Now, communication implies behaviour. Consequently, it is an activity that the designer develops with the purpose of obtaining a behaviour from other people through the information he has about the potential users. However, communication has a goal. This goal is flexible, since the very communicative process is modified, or rather, that goal is being determined progressively during communication. Therefore, communication does not only modify behaviour, but also the goal of the action to be communicated. That is why some of the quality attributes of a multimedia system refer to self-evidence, prediction, etc. The notion of wealth as a quality attribute is a solution that the designer has when he lacks enough information in front of the potential users of the system [23]. Nonetheless, that wealth entails a metamorphosis in the categories of navigation and structure of the system, that is to say to adapt the content and the ways of presenting the information in the interface of the computer
64
F.V. Cipolla Ficarra
screen, iPod, E-Book, etc. in front of the eventual users, globalization and the future of standardization [24], [25], [26]. It is here where the ability of the designer is seen and his empathy, that is to say, the bigger is the experience the more you can see how the goals are being determined with regard to the options that are presented in a progressive way. In other words, the goal of the interactive communication must be adapted in a constant, progressive and detailed way, as the users' behaviour indicates a bigger interaction with the system’s contents. Therefore, currently it is mistaken to set all the goals to be implemented at the beginning of a communicative process in a rigid way, for instance, eliminating the attribute of wealth or richness, whether it is via bidirectional navigation among the nodes of a referential link, activation or disactivation of dynamic means, etc. A parallel of this situation exists in the communication process in a classroom, when the teacher lays down the goals of a lesson at the beginning of the class. Although the goals are shown in a synthetic way, there must also exist the malleability of this order and/or contents with regard to the feedback inside the interaction process with the students [27], [28]. Therefore, communication always entails an action of a previous flow chart, where it is implicitly laid down that is intended to be obtained in the human computer interaction, for instance, repetition of the most frequent sentences in a language, overcoming the self-assessment tests, etc. that is to say, the fruition of the greatest possible amount of information stored in the database, explicitly, such as using the multimedia information to increase the cultural level of a student. In the next graphic some of the main goals are summarised, accompanied by interfaces where there is an excellent empathy towards the potential users of encyclopedia and English course. In figures 1 and 3 we find the richness atribute and a menu of the options. The users can active or desactive the sounds, transition of the frames and dimension of the video, for example.
Fig. 1. Main empathy components in the design of interactive systems (content and presentation categories)
Virtual Classroom and Communicability: Empathy and Interaction for All
Fig. 2. Enciclopedia de la ciencia [29]
Fig. 4. Richness option
65
Fig. 3. Kiyeko –5 languages for children [30]
Fig. 5. Learning English Kids –www.britishcouncil.org/kidsenglish
5 Conclusions The designer's empathy in face of potential users is the key issue for interactive systems of the following types –multimedia on-line, off-line, immersion multimedia, Web 3.0, virtual reality, etc. The current training of the designer and heuristic assessor
66
F.V. Cipolla Ficarra
of interactive systems demands a real intersection of factual and social sciences. Besides, being a communicability expert is also necessary. That is to say, to reach an utmost quality in the design of interactive systems, in the least possible time and with reduced costs. That is why the communicability guide presented is a constructive addtion to partially reaching that goal. We say partially because the current guide is aimed at users without physical disabilities, however it will extend and adapt itself to users with disabilities in the future. It is in the educational process that empathy gains a paramount role, since it is an essential element in boosting the motivation of potential users. Aditionally, a good design of multimedia teaching materials with an excellent communicability level allows E-learning education to be offered not only inside the borders of a country, but also other adjacent countries or those who have a language in common. Consequently, empathy is an economic factor, whether it is because of the gains or losses which can pertain to the educative or industrial entity due to the generation of multimedia content. A way of detecting its presence or absence is through the users' satisfaction at the moment of interaction with the interactive systems. The intersection can be measured through the time spent on the websites or with the classical CD-ROMs or DVDs in the case of languages, for instance. The more interaction, the more communicability, and additionally it means there is a greater designer's empathy at the moment of designing the multimedia system. Acknowledgments. A special thanks to Emma Nicol (University of Strathclyde) and Maria Ficarra (Alaipo & Ainci – Italy and Spain) for their helps. Also, I would like to thank all producers and distriibutors of multimedia/hypermedia off-line: Electronic Arts (Madrid –Spain), Microsoft (Madrid –Spain), ZetaMultimedia (Barcelona – Spain), Ubisoft (Paris –France) and DeAgostini (Novara –Italy).
References 1. Bruner, J.: The Culture of Education. Harvard University Press, Massachusetts (1996) 2. Reeves, B., Nass, C.: The Media Equation –How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press, Cambrigde (1998) 3. Botto, F.: Multimedia, CD-ROM & Compact Disc. Sigma Press, Wilmslow (1992) 4. Nickerson, R., Landauer, T.: Human-Computer Interaction: Background and Issues. In: Handbook of Human-Compputer Interaction. Elsevier, Amsterdam (1997) 5. Cipolla-Ficarra, F., Cipolla-Ficarra, M.: Interactive Systems, Design and Heuristic Evaluation: The Importance of the Diachronic Vision. In: New Directions in Intelligent Interactive Multimedia, vol. 142, pp. 625–634. Springer, Heidelberg (2008) 6. Nöth, W.: Handbook of Semiotics. Indiana University Press, Indianapolis (1995) 7. Cipolla-Ficarra, F.: Communicability design and evaluation in cultural and ecological multimedia systems. In: Proc. 1st ACM Workshop CommunicabilityMS 2008, pp. 1–8. ACM Press, New York (2008) 8. Lang, M., Fitzgerald, B.: Hypermedia Systems Development Practices –A Survey. IEEE Software 22, 68–75 (2005) 9. Bunge, M.: The Science –Your method and your philosophy. Siglo XXI, Buenos Aires (1981) 10. Dance, F.: Teoría de la comunicación humana. Troquel, Buenos Aires (1973)
Virtual Classroom and Communicability: Empathy and Interaction for All
67
11. Birdwhistell, R.: Diccionario de las ciencias sociales. Aguilar, Madrid (1974) 12. Shih, E., et al.: IT Diffusion in Developing Countries. Communications of the ACM 51(2), 43–48 (2008) 13. Fernández, T.: Global Interface Design –A Guide to Designing International User Interfaces. Academia Press, Boston (1995) 14. Kellog, W., Thomas, J.: Cross-Cultural Perspective on Human-Computer Interaction. SIGCHI 25, 40–45 (1993) 15. Dubberly, H., Pangaro, P., Haque, U.: What is Interaction? Are There Different Types? Interactions, pp. 69–75 (2009) 16. Cipolla-Ficarra, F.: HEDCDEH: A Heuristic Evaluation Disk for Communication and Design in Hypermedia. In: CD-ROM Proc. HCI International 2005, Las Vegas (2005) 17. Preece, J.: Empathic Communities. Interactions 5, 32–43 (1998) 18. Cipolla-Ficarra, F.: An Evaluation of Meaning and Content Quality in Hypermedia. In: CD-ROM Proc. HCI International, Las Vegas (2005) 19. Cipolla-Ficarra, F.: Dyadic for Quality in Hypermedia Systems. In: CD-ROM Proc. Applied Human Factors and Ergonomics, Las Vegas (2008) 20. Basili, V., Musa, J.: The Future Engineering of Software –A Management Perspectiqe. IEEE Computer 24, 90–96 (1991) 21. Cipolla-Ficarra, F.: Multimedia and Languages for Children: Semiosis for Universal Access. In: CD-ROM Proc. HCI International, Las Vegas (2005) 22. Marcus, A.: When is a user not a user? Who are we? What do we do? Interactions 5, 28–34 (2003) 23. Cipolla-Ficarra, F.: Evaluation Heuristic of the Richness. In: Proc. International Conference on Information Systems Analysis and Synthesis, ISAS, Orlando, pp. 23–30 (1999) 24. Sears, A., Lund, A.: Creating Effective User Interfaces. IEEE Software 14, 21–24 (1997) 25. Marcus, A.: Icons, Symbols, and Signs: Visible Languages to Facilitate Communication. Interactions 10, 37–43 (2003) 26. Robson, R.: Globalization and Future of Standardization. IEEE Computer 39, 82–84 (2006) 27. Mason, R., Rennie, F.: Elearning –The Key Concepts. Routledge, New York (2006) 28. Edmundson, A.: Globalized E-learning –Cross Cultural Dimension. VDM Verlag, Saarbrücken (2008) 29. Enciclopedia de la ciencia CD-ROM. Dorling Kindersley-ZetaMultimedia, Barcelona (1998) 30. Kiyeko CD-ROM. Ubisoft, Paris (1995)
Communicability for Virtual Learning: Evaluation Francisco V. Cipolla-Ficarra1,2, Miguel Cipolla-Ficarra2, and Pablo M. Vera2 HCI Lab. – F&F Multimedia Communic@tions Corp. ALAIPO: Asociación Latina de Interacción Persona-Ordenador 2 AINCI: Asociación Internacional de la Comunicación Interactiva Via Pascoli, S. 15 – CP 7, 24121 Bg, Italy [email protected], [email protected] 1
Abstract. An assessment is made of the work of design from the perspective of communicability and usability in multimedia aimed at E-learning, mainly through the off-line interactive systems (commercials) and on-line (open software). The method used is accompanied by a series of heuristic results along time to stress the validity or not of some of the design components. Besides, a novel strategy of organizing the textual content is presented for teenagers and the young: the truncated inverted pyramid. Finally, those quality attributes are mentioned that are related to the dynamic and static means, at the moment of heuristically assessing the communicability of a hypermedia system which has as its main goal college education. Keywords: Virtual Learning, Evaluation, Content, User-Centered Design, Open Software.
professors and students. When the instruments are presented in a rigid way towards the teachers, it will be difficult to implement e-learning in the time planned. As is the case with other platforms aimed at e-learning, in almost all the centres where Moodle has been used,there has been a need to create seminars, demonstrations, interchange of experiences, etc. for the use of the platform, training courses for the staff who will use the Moodle, assistance in the design of the courses (even in the transformation of the traditional material that normally use the professors for their schoolroom teaching), and activation of a help desk. It is very important that the help desk be located inside the university and that it is not an outsourced activity [4]. There is an endless series of negative experiences in this sense inside the software context and computers because the user needs access to a person who can explain or solve the problem. In Northern Italy, for instance, the assistance is only available by phone in the commercial world, and involves having to wait until an external technician arrives at the premises where the problem has originated. The costs are high because they include the commuting to the place where the problem is to be found. Outsourcing is avoided as an ideal solution in the software [4]. The current research work is structured in the following way: a brief state of the art of the software used in Elearning college education currently. Later on, the advantages of organizing textual information under the form of truncated inverted pyramid are considered. Then the diachronic results obtained from the heuristic assessment of the off-line multimedia systems off-line are compared, from the point of view of usability and those components of communicability are considered which have a great validity in the current on-line systems whose main purpose is teaching.
2 E-Learning and Open Software University centres keep spending an important sum of their financial resources for the software licenses and the maintenance of the hardware of large servers. In contrast Moodle is a free web application that educators can use to create effective on-line learning sites. Today, the great universities are adopting Moodle for the realization of distance courses or support to presential lessons. Perhaps the main advantage lies in the fact that many applications with a cost equal to zero already exist. These have been evaluated and corrected in other university teaching centres. However, a series of problems to be solved persists before implementing those systems can happen. These problems are related to issues known as human factors of software engineering; guaranteeing an efficient use of the technological instruments from the point of view of usability; helping teachers in the design process, elaboration, assessment, implementation and management of a distance course; to use a design of interactive system that does not change according to the contents and types of studies made inside the academic curriculum; that is to say, that the interface used in a technical engineering in computers has to be similar for superior engineering and a master in design and credibility of the information in the web (for instance), inside the university teaching centre itself. Now we understand by design not only the distribution of the contents of dynamic and static means on the computer screen, that is to say, the presentation, not only navigation, the structure, panchronism, etc. In
70
F.V. Cipolla-Ficarra, M. Cipolla-Ficarra, and P.M. Vera
order to satisfactorily overcome each one of these tasks it is necessary to count on an expert in communicability of interactive systems [5]. In e-learning, it is always necessary, from a financial point of view to use multimedia technologies within the reach of the greatest number of local users and in particular to the potential international users in the emerging areas. Therefore, it is incomprehensible and mistaken that a university under the teaching model "dual mode" plumps for iPod in E-learning, when the dean and over 75% of the professors do not have a digital culture. This entails resorting to outsourcing for the generation of teaching material and even the use of servers for academic management or ERP such as AS/400, creating teaching platforms in Lotus Domino, with very high costs in maintenance and management of software, hardware, professor training, content creation, etc. [6]. There are currently excellent gratis models [7] based integrally on Linux [8] in some Spanish regions as is the case in Extremadura. In this region, Elearning usually has a modest impact from the financial point of view (yearly computer budget) compared to some realities of "dual mode" universities in Lombardy (Bergamo city, for example). These costs may also have an impact on the ondine laboratories dedicated to teaching which exist in the main universities of a country or a region. These laboratories have as their main goal the support of lessons outside of teaching hours to go deeper into subjects, small lessons tending to the stretching of the students, etc. In some cases these activities are under the CAI initials, –Computer Aided Instruction–. CAI may include activities referring not only to the students, but also to the professors. In the case of the professors it has as its essential goal the updating of the latest technological news and knowledge of the potential of the technologies for a qualitative, straight and fluent communication with the students [2], [9], [10].
3 Truncated Inverted Pyramid and Textual Contents On-Line Historically, the textual contents and images are very important in educational multimedia systems [11], [12], [13]. The communicability proposed an efficient way of organizing the textual content in the hypermedia systems: the truncated-invertedpyramid. In the inverted pyramid the most important things are at the beginning, while the least important ones are at the end [14]. But this organization of the text was not detected in the multimedia systems analysed, even if it is more important than the usability of the systems. The use of the inverted pyramid is useful to increase the speed of interaction between user and computer, above all in the case of encyclopaedias, where usually the text prevails over the images. In the textual static elements analysed broadly prevails the organization of the text as a normal pyramid, especially for dictionaries and encyclopaedias. The normal pyramid admits a division in five related areas: 1. 2.
Presentation or exposition of the theme (also called lid). In this area there is the content of the rest of the textual element (textual sememe). Explanation of the principal idea. It is used to enlarge the presentation of the theme.
Communicability for Virtual Learning: Evaluation
3. 4. 5.
71
Sub-themes. It presents other aspects related to the principal idea. Contextual information or background. It creates a context for the principal data of the theme presented. More information about the presentation of the theme or lid. In this area there is a redundancy of information that consequently increases the space of storage of the information in the database. At the end, there are the least important data or a conclusion.
Areas two and three of the normal pyramid have been unified in the inverted pyramid. Therefore there are four related areas: 1. 2. 3. 4.
The lid is the principal area where there is a summary or a conclusion of the theme in no more than thirty words. Explanation of the principal idea and development of the sub-themes. Contextualization of the information. Amplification of the lid.
The use of the inverted pyramid is a writing technique that simplifies the content, as the user has access to the textual element core from the beginning. This is the reason why it is better to organize information in the form of inverted pyramid in the hypermedia systems, particularly when the user has no experience in using computers, as for example the systems developed by UOC –Open University of Catalonian [14]. A graphical representation of the areas of a normal, inverted and inverted-truncated pyramid is shown in figure 1:
Fig. 1. Normal, inverted and inverted-truncad pyramids
In each one of the areas of the pyramid it is necessary to determine the extension of the paragraphs. From the experience carried on in the UOC [14], it is suggested that in the normal pyramid the number of the words in the first area should be between 15 and 20. For the inverted pyramid, the words should be about 30. While in the other areas the paragraphs should not be composed by more than 20 words. But usually this method is not followed in the multimedia systems as you can see in the following image representing the textual element of the definition of the “Hypermedia” – English language in the Wikipedia:
72
F.V. Cipolla-Ficarra, M. Cipolla-Ficarra, and P.M. Vera
Fig. 2. Definition of the “Hypermedia” in the form of a normal pyramid with 70 words in the lid (first paragraph)
An analysis of the organization of the texts is important in order not to compromise the communicability of the multimedia system, as the understanding of the text can be difficult for the inexpert user, especially when the sentence of the text is composed of more than 20 words and there are only a few verbs (one or two). A good proportion is seven words for every verb [14]. The textual elements can be divided according to the type of pyramid presented for a detailed analysis of its content. Nevertheless, the analysis of the content should be different from the presentation or visualization of the content of the textual elements, as now the trend is to title or divide the paragraphs in order to fractionate the content of a large textual element, as for example the information concerning a country. To follow, there is an example where the textual information of the element Australia in the Zanichelli encyclopaedia [15] is divided in paragraphs with direct access. The text can be accessed directly from the titles of the paragraphs located in the inferior part of figure 3:
Fig. 3. Textual element divided and presented in paragraphs
Currently the so-called truncated pyramid is imposing itself in the texts of digital newspapers. The truncated pyramid structure is used when you have to combine the interest that comes from the facts with the need of giving a chronological order to the
Communicability for Virtual Learning: Evaluation
73
account. It consists of the following elements: headline, starting paragraph, main body of the news (main data in decreasing order of interest and secondary data in a growing order of interest), and new essential data. In the truncated inverted pyramid the principles of the lid are maintained with the extension of the words that make up the paragraphs. However, it is necessary to cut down to 50% the explanation of the main ideas and the contextualization of information, which can be replaced by referential links. These four areas of the inverted pyramid are ideal for adult and senior users of interactive systems, who have had previous experience in the off-line multimedia systems and are readers of the traditional books in paper support, including on-line. The young users feel boredom in the face of some of the components of the design of a hypertext structure, and consequently the interest towards the fruition of the content drops. Two examples may be: guided links with texts in the inverted pyramid format and of lineal structure, or rather textual referential links in the truncated pyramid format. Even in the case of using the truncated pyramid modality, many of them only read the lid or first paragraph. Therefore, it is necessary to keep in mind the use of the inverted or truncated pyramid, avoiding the use of the normal pyramid. In the first pyramids in it is necessary to sum up as far as possible the important information at the beginning, placing titles and subtitles that draw the attention of the potential users. Obviously, a senior or adult user whose purpose is research or seeing information online will be grateful in the end for new essential data or referential links.
4 Systematic Assessment Systematic assessment is a set of abstract tasks defined by communicability specialists [5]. These tasks were carried out by a group of users who are experts in the computer environment in particular, in operating systems, or hypermedia systems. A systematic assessment in our case may consist in evaluating four categories of design such as presentation, content, navigation and panchronism. Obviously, the problems that were put to the users have required a previous analysis of the inconsistencies detected in the design [16], [17]. The analysis was the responsibility of a specialist in heuristic assessment in multimedia/hypermedia system. These inconsistencies or problems can refer to the guided tours [18]. A problem that can occur within them is when you can't have access to the nodes that make them up as it happens with the system. At first a guided tours is selected and the navigation through the nodes is made in a directional way. However, there is the possibility of going back to the visited node by choosing the historic listing option, if that component is present in the design category. The latter can only be known at the moment of navigating through the application. For instance, in a guided tours that refers to an artist's collection in a given period of his life, sometimes the paintings are presented with the node number that is being visited and the total of nodes that make up the collection. (node 7 out of a total of 45, for example). In some zone of the interface we can read 7/45. This helps the user in the orientation of the navigation. However, it is necessary that the information presented in the interface agrees with the structure of the nodes and links, that is to say, that you are really in node 7 of a collection of 45 nodes. Inside a set of nodes belonging to a group, joined by a common denominator from the
74
F.V. Cipolla-Ficarra, M. Cipolla-Ficarra, and P.M. Vera
point of view of the content, for instance a historic era, a painting style, a study subject, etc. there are many nodes that have been used, that is to say, inserted in other groups but which do not adapt correctly to the new context. Finally, there can be dynamic means inside the guided links in which the advance or regression of each one of the nodes, can leave active these means: video, animation, music, etc. and prevent navigation, or there is no synchronism among them, thereby affecting the category of the panchronism. An expert in communicability will carry out the inspection of the multimedia system for the latter definition of a set of activities in a heuristic assessment with users. These activities are called abstract tasks. The communicability expert will consider the list of abstract tasks and the list of potential problems the user will find. With this information the assessor defines a series of actions or tasks to be developed by the user. These tasks that are enumerated and described next, may refer to an educational multimedia/hypermedia encyclopedia: 1. To find the geographical data of a country and the continent where it is to be found; 2. To locate guided links and make them navigate forwards and backwards; 3. To determine the borderline countries of that country; 4. Go to the screen of a region or province of that country to then be able to activate or de-activate the dynamic means, for example. Before the realization of the described tasks, it was explained to the users which were the objectives to be reached and the kind of multimedia system that had to be analyzed. In the laboratory a computer system is available for the automatic recording of the time spent and of the answers of the users at the moment of carrying out the tasks. The tables listed the averages of the values for each tasks. Besides, the users' mistakes were indicated with an asterisk (*). If the user ignored the way to carry out the task, the situation was indicated with the letters (i). These letters mean that the user did not know how to do it. The data in the tables have made it possible to identify which were the critical points during the interaction person-computer. The user test has made it possible to check in this way the problems detected in the systematic inspection by the expert. Knowing the time spent and the total number of actions carried out by the users in each task, it can be seen how the problems found by the assessor in the systematic inspection that had previously made. 4.1 Communicability Diacronism and Heuristic Evaluation With the purpose of comparing communicability diacronism and taking account of the evolution of design of the multimedia systems, the MEHEM methodology and the MECEM metrics [19] have been applied. Now the main results related to the application of the table of heuristic assessment are being presented, as described in [20], and the experiments carried out with users to verify the presented methodology. The results of the table are related to the universe of study which has enabled the elaboration of the suggested method. This randomly chosen universe of study is made up by 40 commercial CD-ROMs of international distribution (science and technology, ecology, travels, arts, history, social sciences, hobbies, sports, and games). In table #1
Communicability for Virtual Learning: Evaluation
75
Table 1. Multimedia Systems Off-line Analyzed All Movie Guide; Anatomia 3D; Argentina Vision; Astronomical explorations; CD atlas; Autodesk Texture Universe; Castelos de Portugal; Cdull; Cuba; Curso de Windows; Diccionario de la Lengua Española; Einstein; El Museo Thyssen Bornemisza; El teatro mágico; Encarta; Encicopledia del cine español; Enciclopedia del Universo; Encicopledia Zanichelli; Garden Encyclopedia; Green Bear; Historia Universal del Arte; Instrumentos Musicales; Kiyeko; La aventura de los dinosaurios; La máquina del tiempo; L’Egitto dei Faraoni; Maps Facts; Multimedia Beethoven; NN’n Toy Markers; Peter y los números; Rayman; Red Rhino; The Complete Herman; Travel Talk; Velázquez; Wild Board Games; World’s Greatest Classic Books; Peter Gabriel’s Secret World, Van Gogh and Zoo.
are described the off-line multimedia systems which have been used to analyze the static and dynamic graphic components that have been analyzed. The verification process of the presented methodology consisted firstly in the assessment in a usability lab and secondly in the elaboration and assessment of the applications developed for the UOC [14]. Other heuristic assessors and users have intervened in the verification. Some of these assessors have worked inside a usability laboratory. The users who have taken part in the assessment belong to two categories: expert and inexpert in the use of multimedia systems. Next, the obtained results and in brackets the percentage of the presence of the design component that has been assessed is depeicted: Table 2. Heuristic Evaluation of Communicability Design • • • • • • • • • •
The sound is present in most of the analyzed multimedia systems (75%) The sound prevails in relation to the animations and the video (25%) The sound is the most used device to draw the attention of the user in the navigation (20%) The audio is the device most used to foster the navigation in the analyzed systems (20%) In the base texture of the screens there prevails a combination of texts, colors and images (25%) The lighting effects that prevail are of the ambient kind (50%) The 3D simulations prevail in the 3D with shadow edges (45%) The 3D objects are simply presented with a final rendering (25%) The 2D animations are the ones that prevail in the interface (55%) The explanation of the functioning of the system lies in an help option (75%)
These results which encompass several design categories also enable the designers to have a state of the art summing-up of the commercial multimedia systems off-line in the decade of the 90s. We can see the quality results by categories of design in the annex # 1. The results obtained in the assessment with users inside the laboratory have made it possible to elaborate a list of actions and conclusions which are next presented: • Usability must be divided into several levels. One can start from very general problems which are common to all the interactive systems and go to the specific problems of specific systems. That is to say, at a first level there are the interactive systems in general terms; at a second level the categories of systems such as are the hypermedia or the multimedia and at a third level the kinds of systems, such as for instance an information stand at a museum. In each level there must be a preparatory stage, a listing of abstract tasks and a set of assessment criteria.
76
F.V. Cipolla-Ficarra, M. Cipolla-Ficarra, and P.M. Vera
• Upon evaluating a multimedia/hypermedia system it is necessary in the first place for an inspection by an expert in heuristic assessment of multimedia systems. The results of the inspection are: some application outlines, a list of the potential problems and a set of abstract tasks. The tasks must be specific activities for users in the heuristic assessment. • Empirical evaluation is more effective in relation to the quality of the results and cost when there is a list of abstract tasks. • The inspection of the expert must be efficiently combined with the empirical assessment.
5 Conclusions The heuristic assessment carried out by the communicability and usability expert in the current systems and those in off-line multimedia supports have confirmed the obtained results with the tests made in the usability laboratories in the past decade. Counting on a usability and communicability expert is very positive since it allows the tasks the users must carry out at the moment of the test to be established beforehand. Having precise tasks available allows money to be saved since the designer knows beforehand the possible inconveniences that the users will find at the moment of interaction. In the current work it has been detected how in the multimedia systems made with open source software, they maintain many of the design components of the off-line multimedia systems of the nineties. Therefore, in the current work it has been seen that it is important to consider those educative systems and adapt them to the on-line contents. In the case of the text, which is one of the design categories that prevail in the current on-line interactive systems, it is necessary to resort to organization in the shape of an inverted truncated pyramid. The presented quality attributes and the multimedia design components that have lasted throughout time will be immediately applied in the realization of multimedia products for a virtual campus. Acknowledgments. Thanks to Emma Nicol (University of Strathclyde) and Maria Ficarra (Alaipo & Ainci – Italy and Spain) for their helps.
References 1. Beverly, A.: Instructional and Cognitive Impacts of Web-based Education. Idea Group Publishing, Hershey (2000) 2. Paymal, N.: Pedagogia 3000. Editorial Brujas, Cordoba (2008) 3. Burleson, W., Picard, R.: Gender-Specific Approaches to Developing Emotionally Intelligent Learning Companions. IEEE Intelligent Systems 22, 62–69 (2007) 4. Iacovou, C., Nakatsu, R.: A Risk Profile of Offshore-Outsourced Development Projects. Communications of the ACM 51, 89–94 (2008) 5. Cipolla-Ficarra, F.: Communicability design and evaluation in cultural and ecological multimedia systems. In: Proc. 1st ACM Workshop CommunicabilityMS 2008, pp. 1–8. ACM Press, New York (2008)
Communicability for Virtual Learning: Evaluation
77
6. Willcocks, L., Sykes, R.: The Role the CIO and IT Function in ERP. Communications of the ACM 43, 32–38 (2000) 7. Campbell-Kelly, M.: Will the Future of Software be Open Source? Communications of the ACM 51, 21–23 (2008) 8. Torvalds, L.: The Linux Edge. Communications of the ACM 42, 38–39 (1999) 9. Hammnd, N., et al.: Educational Multimedia for Conceptual Understanding. In: Proc. ACM Multimedia, pp. 447–456 (1995) 10. Larochelle, M., Benarz, N., Garrison, J.: Costructivism and Education. Cambridge University Press, Cambridge (1998) 11. Pea, R.: Learning through Multimedia. IEEE Computer Graphics & Applications 1, 58–68 (1991) 12. Garrand, T.: Writing for Multimedia. Focal Press, Boston (1997) 13. Stephenson, J.: Teaching and Learning Online: Pedagogies for New Technologies. Kogan Page, London (2001) 14. Cipolla-Ficarra, F.: Evaluation and communication techniques in multimedia product design for on the net university education. In: Multimedia on the NetVienna, pp. 151–165. Springer, Heidelberg (1996) 15. Zanichelli Enyclopaedia CD-ROM. Zanichelli Publishing, Bologna (2004) 16. Nielsen, J., Mack, R.: Usability Inspection Methods. Willey, New York (1994) 17. Lewis, J.: Sample Sizes for Usability Tests: Mostly Math, Not Magic. Interactions 13, 29– 33 (2006) 18. Trigg, R.: Guided Tours and Tabletops : Tools for Communicatiing in Hypertext Environment. ACM Transactions on Office Systems 6, 398–414 (1988) 19. Cipolla-Ficarra, F.: Communication Evaluation in Multimedia –Metrics and Methodology. LEA 3, 567–571 (2001) 20. Cipolla-Ficarra, F.: Table of Heuristic Evaluation for Communication of the Multimedia Systems. In: Proceedings of the HCI International, pp. 940–944. LEA, Crete (2003)
Annex #1: Evaluation of Multimedia Systems
Attention and Motivation in Hypermedia Systems Francisco V. Cipolla Ficarra1,2 and Miguel Cipolla-Ficarra2 HCI Lab. – F&F Multimedia Communic@tions Corp. ALAIPO: Asociación Latina de Interacción Persona-Ordenador 2 AINCI: Asociación Internacional de la Comunicación Interactiva Via Pascoli, S. 15 – CP 7, 24121 Bg, Italy [email protected], [email protected] 1
Abstract. We present the results of a heuristic analysis of a set of multimedia off-line systems aimed at boosting the mental skills of the users through reflexes, maths, etc. To this purpose two metrics have been created to assess the motivation of the users and the degree of help implicit in the multimedia system. The metrics are based on a group of primitives aiming at increasing the communicability of commercial multimedia systems and targeted to the public at large, regardless of age and previous experience in the use of computers. Keywords: Attention, Motivation, Hypermedia, Design, Navigation, HumanComputer Interaction, Help.
support. From 1995 onwards plenty of small and medium businesses sprang up with the purpose of digitalizing the contents in paper support. Here takes place the first transference of the concepts of the graphical arts to the design of computer interfaces. Obviously, the dimension of the screen was bigger than the classical A4 folio. Many principles of topography, colour, disposition of the images, with their texts, etc., responded to the graphical arts of the paper [4]. To design a screen at that time was to organize the background of the screen, that is to say, all the elements in the screen that contribute to the appearance and the behaviour of the interface. The background carried all the burden of the design because it covered a great part of the screen. The background, from the point of view of motivation and the design fulfills two functions: First, it has an influence in the appearance and the visual balance, and the position of all the elements. Second, it fills the void space so that other elements are not flying objects. In both cases, all the elements must be differentiated, but at the same type an isotopy must be created among them [5]. Currently, we find university websites where the background is textual, full of colours that do not respect any chromatic order for the interfaces design [6] and the text shifts vertically, joined to dissolution transitions by dots among the frames of the different nodes, at the moment of navigation. That is to say, a style from the early 90s for off-line multimedia systems but which can't be applied in the era of Web 2.0 and 3.0, especially in the university environment. An interface with such backgrounds immediately undermines motivation of the contents' fruition.
2 Motivation: Dynamics and Statics Means The motivation for this study is the whole set of dynamic means and structural resources that boost the navigational quality of a multimedia/hypermedia system. With dynamic means and structural resources, it is intended to increase or maintain interest towards the system, especially when the user is in a student role [7], [8]. The motivation seeks to focus the user’s attention on the need to continue fruition of the system. Attention on the screen must be sustained, that is, the user keeps an attitude of permanent expectation with regard to the system. It is mainly virtual resources that trigger very good results in attention and acceptance towards the system, especially when there is no previous computer experience by the user. Without going deeper into psychological aspects, there are two factors related to the content and structure which allow attention to be drawn to an interface. If the content of the information is specialized, it means that the target of the multimedia system is an expert, and therefore the information must be relevant and be well-organized. Here the attention on the interface is of a cognitive character [8]. There is an “affective” link of the user towards the system because of the linearity of the message. This lineal factor forces the user to utilise the system until the end of a sequence. As a rule, the establishment of a sequence series through the use of structural methods is due to the fact that the user studies the content. These cognitive or affective factors can be used to draw attention to a multimedia/hypermedia system in a joint or individual way [9]. Choosing one way or the other will depend on the kind of user and the purpose of the system, such as educational, consultation, entertainment, etc. But both factors are present in the motivation or boosting of navigationn multimedia systems one resorts continually to dynamic means to motivate fruition:
80
1.
2.
3.
4.
F.V. Cipolla Ficarra and M. Cipolla-Ficarra
Music; Sound effects that are activated upon reaching a goal, such as are: to pass an educational subject with success, to go through a higher level inside a tale, to win a game, etc. Voice; The narrator or speaker incorporates sentences to promote advancment through the system. For instance, in the fragmented and gradual assessment of content in a computer-assisted teaching system. Text and images linked to transitional effects, such as blurring, sweeping, fadeins, fade-outs. The textual sentences have expressions of the kind: “excellent”, “go on,” and “continue”. 2D and 3D animations of characters and objects (in the universe of study they are 2D animations, made with Flash [10]).
Fig. 1, 2. Interfaces with 2D animations in Allenamente DVD-ROM
Examples of structural resorts which maintain and boost interest in the system: The outlines that at the end of a guided link make it possible to switch to other entities; An Index, and nodes that link to other components of the same entity. The existence of resources to promote or maintain motivation have a positive influence on the richness of a multimedia system [11]. The relationship between the attributes of wealth and motivation are based mainly on resources that belong to the visual component, such as the images and interactive help. Interaction with the system can generate a series of mistakes due to either hardware or software. Hypermedia systems usually have means to solve these hardware errors as well as those of software. There are several ways of presenting help in the interface. This help can be classified in regard to the access and the presentation in the form of tutorials, outlines and exploratory texts: 1. 2.
3.
Help in the shape of a tutorial consists of a collection of screens with an index on the first of them (can be activated from all the screens in the system –fig. 3 [12]). Help of the outline kind is that in which on a single screen are explained the functions of the main components of the interface, as it can be observed in figure 4 [13]. The exploratory help is that in which as the cursor moves along the keyboard, the explanation is activated, for example, in figure 5 [14]. In the automatic correction
Attention and Motivation in Hypermedia Systems
81
of Gmail spelling, for instance, sometimes we see how the first word remains hidden, thereby preventing a quick visualization of the possible correct words. (figure 6). A good example of this is the interface of the Cinemanía system as it can be seen in the figure 4.
Fig. 3. Help in the form of a tutorial
Fig. 4. Help of the outline
Fig. 5. Upon placing the cursor on the clock icon the meaning appears
Fig. 6. Gmail spellchecker –the first word remains hidden
82
F.V. Cipolla Ficarra and M. Cipolla-Ficarra
There are several ways to activate help for the explanations of the components of a screen, such as the shifting of the cursor on the interface; through an option as has been seen in figure 5 or in the presentation of the system. In this last case it is shown in the shape of an automatic animation before it appears on the main screen. However, 63% of the analyzed multimedia systems lack available aids on all their screens. Access to these aids should be constant in all the screens of the system regardless of the place within the structure in which the user finds themselves. Although the purpose of the aid is to facilitate interaction, there are systems in which the help is not a solution to the user, since they increase disorientation and demotivation within the interaction. The use of some kind of help or another at the moment of design will depend on the user of the system [15]. Among the different kinds of access to help, the most positive to the expert user are the outline and exploratory modalities. The outline modality allows users watching on a single screen all the explanations of the components in the shape of an outline which speeds up reading, but obviously avoiding the failure in figure 5. For the inexperienced user, it is positive to have a simple movement of the mouse from which the meaning of the subject indicated on the screen is revealed through exploration. However, for an expert user the exploratory help can turn out to be negative by watching information he/she already knows. This is due to redundancy of information, and to the fact that exploration requires a longer time at the moment of interaction with the system. For these kind of users it is advisable to use other means of help.
3 Attention and Users The treatment of the attention stimuli must pursue a simplification, hence the organization of the framing of the visual dynamic means is important. Creating excessively informative situations must be avoided in order not to wear down the user. This can only fully handle a small fraction of the whole information that reaches his/her senses. Through the capacity for selective attention the user pays attention discreetly to some stimuli while ignoring others. In any development in which a great amount of elements are combined, as it happens in any message of the visual dynamic means, this process takes place. If a big part of the development of the speech is left to the user's selective attention ability, you run the risk of not being able to establish an effective communication, because it is liable to trigger many limited interpretations and in some cases overlapping interpretations. Here we do not refer to the participation margin, to the stimulation of the user's imagination that any hypermedia must present [9], [16] but to the lack of structuring or conductive order of the sense. In this sense the camera movements (video or animation) are important. For instance, in the horizontal movements there is a tendency to split the screen in two halves, one to the left where we place the I and now, and another to the right, where we place the “you”, the future and any pictorial object located in this part tends to look bigger. Therefore, the movements from left to right symbolize a movement from I to you, the movement in direction to the world, the progressive movement. Its perception demands lesser effort than those aimed at the right. The movements from right to left are related to traditionalism and interiorization, it is a conservative movement, in direction to the origin, to the I. The direction of the movement also affects the
Attention and Motivation in Hypermedia Systems
83
perception speed by the user [17], in general the descending motions and from left to right are faster than ascending ones, and those from right to left. Here is the reason of the importance of the elements that make up the visual composition of the dynamic means. The sound is another element to be combined during the composition has a similar treatment, in front of the attention and the stimuli that make it work, for example in games [18], or the audio channel, by establishing relations to the visual side, constitutes a semantic and/or expressive boost in the whole articulation of the multimedia content. The obtained results make it clear that in the set of analyzed systems, motivation is only of 20% value, and the quality of the help is equal to 35% of the analyzed cases, as revealed through heuristic techniques. Consequently, the analyzed systems are not communicable to those users who do not have previous wide experience in the use of computers. 3.1 Attention, Users and Audio-Visual Factors With the passing of time and the reduction of the size of screens it was necessary to include other components from the dynamic means in order to appeal to the attention and the motivation of the users. One of those was the video. A good image that speaks and moves has a greater communicative power than that of a photograph or a graphic, for instance. However, it loses that ability to draw attention and motivate the user if the content does not convey a message or a captivating experience. Evidently, the fact that the longest video segments require a greater variety and more attention to the coordination of those segments (something missing here!). Twenty seconds with an anchorman who makes you think of a speaking statue may seem like eternity to a teenager, whereas a 75 second long action sequence will seem short as compared with the first. Now, from the point of view of design of the interface it is necessary that the video merges to some extent with the rest of the objects that make it up. In multimedia off-line systems the use of video was cut down to a minimum for several reasons: the room occupied in megabytes in the PC, the low quality of the full screen reproduction with which they were cut down to 10-12 cm. long small squares or rectangles (the user was accustomed to seeing these kind of images in the screens of computers which were over 20 inches big), the copyrights, etc. The computer animations through the video games consoles would breed another generation of users of multimedia systems dedicated to a pastime who didn't use the keyboard or the PC mouse [17]. In those systems, the animations tried to draw the attention of the users through the audio, that is to say, the music, the special effects, etc. , which were especially created for each one of the interactive games. In order to successfully integrate the sound in the interface it was necessary to pay attention to the synchronism among the music levels, the human voice in the narration (where it exists) and the sound effects. All of them were well-balanced in order to achieve the desired effect on the user, especially in the design category known as panchronism. In the combination of the audio with the animations there was an excellent coherence of the colours, lightning and scenography, the characters the script of the plot or adventure, etc. Many of these design strategies in the dynamic and static means have been boosted through lighting, for instance.
84
F.V. Cipolla Ficarra and M. Cipolla-Ficarra
4 Visual Attention Evidently it is very important that the illumination component and special effects (FX) enter into learning process independently of the contents and animations 2D and/or 3D. We have three main effects: • Lighting effects: The effects stemming from lighting such as are shadows, reflections, diffusion, etc., must also remain constant in the whole hypermedia system. The lighting effects derive from the kind of light that is used in scenes with tridimensional objects. This kind of light can be environmental or directional. In environmental lighting the objects that receive the light generate shadows, whereas in directional lightening a beam of light is emitted over a given object of the scene. When there is more than a beam of light, lighting is multidirectional. In the following figures extracted from the off-line multimedia systems Braincity [18] and Universal History of Art [19], there are examples of lighting. The correct use of environmental light can be seen in all the scenes that make up the application and in the Universal History of Art. Through directional lighting the main options of the system are shown and activated.
Fig. 7. Directional lighting (center of the image)
Fig. 8. Environmental lighting
• The effects of the reflection or shining simulate the arrival of the light to a spot in an object. This light can be depicted in the shape of a star. The shine can be either static or dynamic. It is dynamic when an animation is associated with it. The animation may consist in a switch of 360º over its axis or in an increase or diminution of the shining. This effect is used mainly when in the background of the screen the black colour prevails, the scale of grey and blue shades, and all the metalized colours such as gold, silver, copper, etc. The shine means prestige or superiority of an element in relation to the others that make up the interface. They can also be useful to draw or attract the attention to an element in the screen. In the Movie Guide we can find a dynamic shine (the intention is to stress a “search wizard” which can be activated from any place in the structure of the system). • Relief effect: There was an attempt in the 90s to emulate the shadows produced by environmental lighting in the screen elements. The purpose in these first
Attention and Motivation in Hypermedia Systems
85
hypermedia systems was that these bidimensional elements “acquire” tridimensionality. Emulation is obtained by correctly darkening the edges of those elements that are intended to be stressed in the interface. Today it is a common element in the web 2.0 and web 3.0 Between these two elements the navigation keys can be mentioned, typography, picture frames, etc. It is a positive effect in all the navigation keys, since they are easier to recognize in an interface, as it can be seen by comparing figures 9 [21] and 10 [22]. The interface of a multimedia system has a greater degree of realism with the emulation of the shadows, which favors their acceptance.
Fig. 9. A greater degree of realism
Fig. 10. A normal degree of realism
Without any doubt, lightning, colours, shapes, the organization of the contents, etc. have a very important role in the motivation and in the attraction of the users in the on-line and off-line multimedia systems. A good or bad lightning of the components that make up the interface can boost or not the interaction of the potential users, regardless of their content and the goal of the interaction: entertainment, educational, informative, etc.
5 Heuristic Evaluation: Results In our case we have assessed as a representative example a set of 15 multimedia systems in DVD-ROM support from the Allenamente collection [10], which theoretically favors language, logic, memory, etc. Besides, it contains an endless series of games and exercises for self-assessment the more the user advances in the contents. The search for the presence of the motivation and the attention in the ensemble of the analyzed off-line multimedia systems has led us to create a series of metrics. For their elaboration we have resorted to the primitives stemming from the evolution of the interactive systems: hypertext, multimedia and hypermedia. The listing of primitives and the design categories to which they refer are: in annex #1. The graphic showing the obtained results. The graphic demonstrates the scarce quality from the motivation point of view and the attention of the analyzed systems.
86
F.V. Cipolla Ficarra and M. Cipolla-Ficarra
6 Conclusions The motivation and the holding of the users' attention in the content of interactive systems: multimedia, virtual reality, etc. is one of the main goals at the moment of the design. Now, the 3D graphic informatics has more available resources in this sense than 2D informatics. The dynamic means play a very important role at the moment of drawing and keeping the users' attention. The metrics presented in this work have made it apparent how in the off-line multimedia systems with 2D animations there is a scarcity of animated graphic resources and a lack of creativity in the planning of the contents. An interesting content without resorting to communicability may turn out to be negative for the potential users to keep on navigating through the different systems that make up a wide multimedia systems collection. The motivation and the attention to the design of an interactive system must not only involve the content, but also each one of the following categories: presentation, navigation, structure and synchronization of the dynamic means, that is, panchronism. All these categories must include the quality attributes for multimedia/hypermedia systems off-line and on-line. Acknowledgments. Thanks to Emma Nicol (University of Strathclyde), Maria Ficarra (Alaipo & Ainci – Italy and Spain), Quim Romo-Ferrer and Carlos for their helps.
References 1. Soh, J., Tan, B.: Mobile Gaming. Communications of the ACM 51, 35–39 (2008) 2. Cipolla-Ficarra, F., Cipolla-Ficarra, M.: Interactive Systems, Design and Heuristic Evaluation: The Importance of the Diachronic Vision. In: New Directions in Intelligent Interactive Multimedia, pp. 625–634. Springer, Heidelberg (2008) 3. Silva-Salmerón, J., Rahman, M., El Saddik, A.: Web 3.0: A Vision for Bridging the Gap between Real and Virtual. In: Proc. 1st workshop communicability design and evaluation in cultural and ecological multimedia systems, pp. 9–14. ACM Press, New York (2008) 4. Kahn, P., Lenk, K.: Principles of Typography for User Interface Design. Interactions 6, 15–29 (1998) 5. Cipolla-Ficarra, F.: Evaluation of Multimedia Components. In: Proc. IEEE Multimedia Conference on Multimedia Computing Systems, pp. 557–564 (1997) 6. Cipolla-Ficarra, F.: HECHE: Heuristic Evaluation of Colours in HomepagE. In: DVDROM Proc. Applied Human Factors and Ergonomics, Las Vegas (2008) 7. Cipolla-Ficarra, F.: Evaluation and communication techniques in multimedia product design for on the net university education. In: Multimedia on the Net, pp. 151–165. Springer, Heidelberg (1996) 8. Edwards, A., Holland, S.: Multimedia Interface Design in Education. Springer, Berlin (1992) 9. Card, S., et al.: The Pychology of Human-Computer Interaction. Hillsdale, New Jersey (1993) 10. Allenamente DVD-ROM. GreenTeam, Bologna (2008) 11. Cipolla-Ficarra, F.: Evaluation Heuristic of the Richness. In: Proc. International Conference on Information Systems Analysis and Synthesis, ISAS, Orlando, pp. 23–30 (1999)
Attention and Motivation in Hypermedia Systems 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22.
87
Enciclopedia de la Ciencia CD-ROM. ZetaMultimedia, Barcelona (1998) Cinemania CD-ROM. Microsoft, Madrid (1996) Musée d’Orsay CD-ROM. Montparnasse Multimedia, Paris (1996) Dworman, G.: Arbitration of a Help System. Interaction 14, 39–42 (2007) O’Neill, S.: Interactive Media –The Semiotics of Embodied Interaction. Springer, London (2008) Terzopoulus, D.: Artificial Life for Computer Graphics. Communications of ACM 42, 32–42 (1999) Raghuvanshi, N., et al.: Real-Time Sound Synthesis and Propagation for Games. Communications of the ACM 50, 66–73 (2007) Braincity CD-ROM. Digital Illusion, Barcelona (1995) Historia Universal del Arte CD-ROM. Espasa-Calpe, Madrid (1996) Explorama CD-ROM. Anaya, Madrid (1995) All Movie Guide CD-ROM. Corel, Ottawa (1995)
Annex #1: Primitives and Graphic Results Table 1. (P)rimitives for heuristic analysis of the motivation and attention (design catogories: content, dynamics, structure, presentation and panchronic) Analepsis (P): Content and Dynamic; Continuum (P): Content and Dynamic; Element (P): Content; Element Type (P): Dynamic and Structure; Entity (P): Content and Presentation; Frame (P): Presentation, Content, Dynamic and Panchronic; Frame Principal (P): Presentation, Content, Dynamic and Panchronic; Guided Tour (P): Presentation, Structure and Dynamic; Hierarchical Links (P): Structure and Dynamic; Index (P): Structure and Dynamic; Hypertrails (P): Structure and Content; Keyword Links (P): Content; Link (P): Structure; Node (P): Content; Polytopes (P): Presentation and Content; Polysemy (P): Content; Referential Links (P): Structure and Content; Sememe (P): Content.
A Web-Based, Interactive Annotation Editor for the eCampus Development Environment for SCORM Compliant E-Learning Modules Benedikt Deicke, Jan-Torsten Milde, and Hans-Martin Pohl University of Applied Sciences Fulda Competence Center Human Computer Interaction [email protected], [email protected], [email protected]
Abstract. The eCampus development environment was created in an interdisciplinary project at the University of Applied Sciences Fulda. Today it is a fully webbased application for the easy creation of E-Learning modules complying the SCORM standard. The webbased, interactive annotation editor for the eCampus development environment is used to both automatically and manually annotate existing OpenOffice documents in order to transform them into E-Learning modules. The editor is build using Open Source software and frameworks such as Ruby on Rails. Keywords: E-Learning, Web, SCORM, eCampus, OpenOffice, Ruby, Ruby on Rails, JavaScript, user friendly, annotation, transformation.
1 Introduction The creation of standard compliant E-Learning modules is a complex task. In order to simplify the process, the University of Applied Sciences Fulda created the eCampus development environment [5]. It is based on OpenDocument files which are transformed into SCORM [1] compliant E-Learning modules using XSL Transformations. The transformation relies on annotations added to the OpenDocument file using OpenOffice.org [3]. Nevertheless, the handson experience showed that the annotation process using OpenDocument styles within OpenOffice.org is still too complicated for most of the users. With the “eCampus” development environment now being a web-based and integrated system for the generation of SCORM compliant E-Learning modules, the creation of an interactive, web-based annotation editor is the next logical step. To simplify the annotation process even more an automatic analysis and annotation process is implemented, too.
A Web-Based, Interactive Annotation Editor for the eCampus Development Environment
89
the eCampus project at the University of Applied Sciences Fulda. In this interdisciplinary project, content is being created for the faculties Nutritional Sciences, Food and Consumer Sciences, Applied Computer Science, Nursing & Health Care as well as Food Technology. Within the first years of the project, many E-Learning modules were produced. All modules address prominent introductory courses of the particular program of study. This was only possible by using the eCampus framework. The central target of the project is the implementation of a tool that decreases the complexity of the process of learning module production. The central objectives of the learning module production are an increase in the interactivity of the course, as well as an increase in the quality and quantity of self learning.
3 The Tools The annotation editor is based on the existing eCampus web application which is built using Ruby [8] and the Ruby on Rails framework [7]. It is running as an E-Service on a centralized server. In order to use it, the author has to create a user account. This is required to protect uploaded documents and generated E-Learning modules from unauthorized access. The user interface is built using XHTML and CSS. It is enhanced by JavaScript utilizing the Prototype and Script.aculo.us frameworks [4].
4 The Process Using the webbased annotation editor the process of creating E-Learning modules from OpenDocument files is condensed into five simple steps: Uploading of the unannotated OpenDocument file, automatic analysis and annotation of the file, manual annotation and correction of the resulting document, transformation into a SCORM compliant E-Learning module, downloading of the finished module. 4.1 Reading and Modifying OpenDocument Files In order to read and manipulate the OpenDocument a library has been developed. It encapsulates the required steps within a simple API. In order to read the contents of the document it is extracted using RubyZIP. Afterwards the content. xml file, which contains the actual content, and the styles.xml file are parsed using Ruby’s built-in XML library [6]. Utilizing Ruby’s dynamic nature, the library creates a class for every node type contained in the document’s content and instantiates it as needed. This provides the possibility to dynamically overwrite the default behavior for specific node types. 4.2 Automatic Analysis and Annotation The process of the automatic analysis and annotation is separated into different annotation engines. The engines analyze every stylable element in the OpenDocument’s content structure using different approaches. If the analysis step is
90
B. Deicke, J.-T. Milde, and H.-M. Pohl
not successful, the next annotation engine tries to analyze the current element. This is continued until no annotation engines are left or an approach is successful and annotates the element. This process is repeated for every annotated element of the input text. Figure 1 visualizes this concept. Currently five annotation engines are implemented: Keep, Tags, Styles, Bayes and Default. The Keep engine stops the analysis chain if the element is already annotated with eCampus styles by checking each elements formatting style. The Tags engine analyzes the type of the current element and its context and tries to map this onto eCampus styles. For example text:h nodes with an outline level of 1 are annotated with EC_Modul, nodes with outline level 2 as EC_Session, etc. The Styles engine simply translates default OpenDocument styles into eCampus styles where possible, such as Text body to EC_Text. The Bayes engine tries to analyze the current elements style properties in order to detect elements like headlines or important terms which were just changed visually, but not by using using built-in OpenDocument styles. In order to achieve this, the Bayes engine is trained with correctly annotated documents. Based on the annotation and the visual style, the engine learns possible visual variations. While visiting each node in the document the Bayes engine builds “sentences” from the nodes style attributes. This results in sentences like “12pt bold italic” which are handed to a bayes classifier that returns a corresponding eCampus style or unknown if no style could be identified.
Fig. 1. The automatic annotation process
As well as the final transformation into a SCORM compliant E-Learning module, the automatic analysis and annotation takes several seconds (or even minutes, depending on the documents size) to finish. Therefore it needs to run asynchronously in the background. This is done marking the documents as waiting for analysis in the database. See Figure 2 for a complete overview of possible states. A separate processes, triggered by CRON, checks for waiting documents and runs the analysis on them. On the client side the user can see the status of the transformation. It is automatically updated using a polling AJAX request.
A Web-Based, Interactive Annotation Editor for the eCampus Development Environment
91
Fig. 2. Possible states of a document
4.3 The Interactive, Web-Based Annotation Editor The front end of the annotation editor for manual annotation is implemented on the client side using JavaScript. It uses Ajax techniques [2] to save changes to the document on the server. The GUI is designed to be simple and clean and focusses on the important functions needed to annotate the document. Figure 3 shows a screenshot of the final GUI. It consists of a toolbar on top, which includes buttons for navigating through the document and buttons for saving and exiting the editor. The tooltip on the right is used to actually annotate the currently highlighted element. The user is able to select the desired style by selecting it from the select-box. The JavaScript implementation is designed to be as modular as possible. It utilizes the observer pattern to separate the various GUI components from the underlying logic. On startup of the annotation editor, every component registers itself with the AnnotationEditor core class. Based on the users interaction with the GUI the elements fire events on the core class, which executes the requested functionality and fires events on all registered components as a result. This enables the possibility of additional annotation methods, other than the tooltip described above. Imaginable annotation methods are: dragging and dropping the styles on the elements or a fill format mode like known from OpenOffice or Microsoft Word. To display the OpenOffice document it is read on the server and transformed into a simple HTML representation. Each HTML elements relation to a node in the original OpenOffice document is described by adding an attribute containing the nodes XPath to the element. The nodes annotation (such as EC_Definition, EC_Section, EC_Image, etc.) as well as its family (Paragraph, Text or Graphic) are represented using CSS class attributes. Additional every HTML element that can be modified later the CSS class annotatable is added. During the manual annotation the underlying CSS class attributes identifying the annotation are added, modified or removed. On modification of an element it gets
92
B. Deicke, J.-T. Milde, and H.-M. Pohl
marked as changed. When the user decides to save the changes, all marked elements are collected and serialized into JSON before they are transfered to the server using an Ajax request. On the server the JSON is deserialized and the changes are applied to the OpenOffice document using the library described above.
Fig. 3. Screenshot of the Graphical User Interface
5 Results and Visions The combination of a simple user interface for the annotation and the automatic analysis and annotation process the creation of standard compliant E-Learning modules out of existing content simplifies the creation of standard compliant ELearning modules. Being integrated into the eCampus web application the annotation editor inherits useful features like easy updating and extending. The annotation editor completes eCampus to a full web based development environment for SCORM compliant E-Learning modules. Future development could include the possibility to rearrange elements within the document or even change the content. Additionally the current user interface needs to be fully evaluated during end-user tests. As described above other annotation methods could be implemented as well. Furthermore the automatic annotation engines offer a broad scope for new additions.
References 1. ADL. Scorm, http://www.adlnet.gov/scorm/ 2. Garret, J.J.: Ajax: A new approach to web applications (August 5, 2008), http://www.adaptivepath.com/ideas/essays/archives/000385.php 3. OpenOffice.org. The free and open productivity suite, http://www.openoffice.org/ 4. Porteneuve, C.: Prototype and script.aculo.us. The Pragmatic Programmers, 1st edn. (2007)
A Web-Based, Interactive Annotation Editor for the eCampus Development Environment
93
5. Pohl, H.-M., Tulinska, P., Milde, J.-T.: Efficient creation of multi media eLearning modules. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007. LNCS, vol. 4558, pp. 457–465. Springer, Heidelberg (2007) 6. REXML. Ruby standard library documentation (August 18, 2008), http://www.rubydoc.org/stdlib/libdoc/rexml/rdoc/index.html 7. Thomas, D., Heinemeier-Hannson, D.: Agile Web Development With Ruby On Rails, 1st edn. The Facets Of Ruby Series. The Pragmatic Programmers (2005) 8. Thomas, D.: Programming Ruby - The Pragmatic Programmers’ Guide, 2nd edn. The Pragmatic Programmers (2005)
An Innovative Way of Understanding Learning Processes: Eye Tracking Berrin Dogusoy1 and Kursat Cagiltay2 Computer Education and Instructional Technology Department, 1 Mersin University, 2 Middle East Technical University {bdogusoy,kursat}@metu.edu.tr
Abstract. This paper aims to present findings on the use of eye-tracking technology as a new approach from an educational perspective. The studies in this paper on relationship between learning and eye-movements have focused on concept-map formation, learning from multimedia materials, designing materials with different cognitive strategies, multimodal comprehension of language and graphics with and without annotation, computer games and cognitive style effects of computer based interfaces and hypertext environment. The results of the Middle East Technical University (METU) Human Computer Interaction (HCI) research group’s eye-tracking based research studies presented and discussed how this approach helps educators to better understand learning processes of humans. Understanding and using this innovative approach is important for both educators and researchers in terms of comprehending learning processes deeply. Keywords: Eye-tracking, web based learning processes, concept maps, computer games, learning from multimedia.
An Innovative Way of Understanding Learning Processes: Eye Tracking
95
potential to give a broaden perspective for the researchers providing both application opportunities and the different analysis techniques. This paper aims to provide information on this new and innovative approach from an educational perspective. Our studies on learning and eye-movements have focused on concept-map formation, learning from multimedia materials and computer games. The results of eye-tracking based research studies [5] were presented and discussed how this approach may help educators to better understand learning processes of humans. Understanding this innovative approach is important for both educators and researchers in terms of comprehending the learning process deeply by using its new techniques. 1.1 Related Eye-Tracking Studies There are many studies which were conducted by using eye-tracking approach. Eye tracking approach gives researchers opportunity to collect information on user behavior in definite tasks. This device also provides researchers various data related with the user through the process of task; task completion duration, transition number, fixation count, average fixation duration, time to first fixation, hotspot, fixation order, gaze time and gaze replay data [6]. Under METU HCI group, eye-tracking studies have been done by using a usability laboratory environment. Baran, Dogusoy and Cagiltay (2007) conducted an eyetracking study to understand how adults solve Tangram (Chinese puzzle) based geometry problems [7]. The sampling of this study consists of twenty participants. To analyze the problem solving process, the eye movements of the participants were recorded while they were working on two problems with different levels. The results showed that the participants use different strategies to solve problems given in two different difficulty levels. The researchers indicated that there might be relationship between the task completion duration and the difficulty levels of the problems. Moreover, eye-tracking device enabled the researchers to use the Area of Interests (AOI) option to determine different areas on the screen and these areas were used while analyzing the users’ behaviors. In this study, the AOIs were used by dividing the screen into two parts (geometric object screen-the objects given in this screen and problem screen-the solving process is eventuated) in order to determine the different behavior patterns of the participants in problems with two different levels. The researchers argued that there is a relation between the problem complexity level and the focused place on the screen. The participants tended to focus on the problem screen rather than the geometric objects screen. This was explained by the researchers as participants used both inductive and deductive strategies for solving the problems with different levels. This study have potential in terms of giving beneficial clues to educators both using different strategies and support the learners in the development of strategies for solving the problems in different difficulty levels. User analysis is an important issue for instructional designers and in another study the main issue which was explored is the presumptive effects of cognitive style on the interaction ways while using computer based interfaces [8]. Since cognitive style is frequently researched issue by the researchers especially the potential influence of cognitive style on the interaction with interfaces, they grouped the participants by using Cognitive style test, Group Embedded Figures Test (GEFT) and they were
96
B. Dogusoy and K. Cagiltay
informed about the process. The process includes studying the predetermined web site and after that the participants were required to take a test about the web site content. The results showed that although there wasn’t a statistically significant difference between the participants in terms of the eye-tracking data, there were differences between the two groups in terms of the fixation duration and fixation places. It was suggested by researchers to study with wider groups and using think-aloud protocol during the process, studying the website in further studies. Another eye-tracking study was related with the multimodal instructional materials and the behavior of the users was conducted [9]. The main purpose of this study was to investigate the potential effects of the instructional environments which were designed by using multimodal information on the participants’ behaviors. Fifteen participants’ behaviors were recorded and their eye movements were analyzed by using eye-fixation counts and durations. The researchers indicated that the participants used both slide and video presentations in the study. Also, the participants much more focused on the video screen than the slide presentation screen which was obtained from eye-fixation duration data. They suggested replicating the study by using different combinations of the slide and video materials, that it might be possible to offer effective designs for this kind of instructional materials. Eye-tracking is used as a beneficial approach in terms of observing the general patterns of the participants while completing a specific task. In another study, this approach is used for observing the general pattern of the prospective teachers during the concept map construction [10]. With the aid of eye-tracking device it was aimed to determine whether there was a general pattern among the participants while developing a concept map. Especially the focusing time on concepts, links and cross links were explored. Also, the quality of the created concept maps was considered while analyzing the eye-tracking data on process. Sixteen prospective teachers attended this study from the science education related departments. The results showed that the participants tended to develop hierarchic concept maps, and synchronize strategy while writing the links and concepts. The researchers indicated that with the aid of this study, it may be possible to give some hints to teachers about the effective use of concept maps and their construction process. Usability is a crucial issue in education that design process is directly affected by the usability related problems. In another research study, the main aim was to gather information on the issues related with the computer games especially usability aspect with the aid of eye-tracking device [11]. They used a computer game which was not familiar for the participants and tracked the behaviors of them during the learning process of a new game. Sixteen participants attended to the study. The researchers pointed that they used not only quantitative but also qualitative methods. Eye tracking approach provided information related with the fixation number, total duration and gaze time of the participants. In addition to this, the comments of the participants were taken for determining the main usability issues related with game. Researchers explained that during the learning process the participants tended to use a trial and error approach. Although there were no gender and level differences among participants in terms of education effect on participants’ learning process, the participants marked many usability issues related with the game. Also, they emphasized that the game should be re-designed with the aid of the information gathered from the users.
An Innovative Way of Understanding Learning Processes: Eye Tracking
97
The researchers in another study aimed to investigate the cognitive style effects on navigation patterns during an interaction occurred within a hypertext environment [12]. The sampling of this study consisted of twenty participants and they were grouped with the aid of the Group Embedded Figures Test (GEFT) and also the knowledge and ability of the participants on computer was another factor. A predetermined task was given to the two groups (Field-dependent and field-independent) and they were asked to navigate within a hypertext environment. Data was gathered by using eye-tracking device; the fixation duration and gaze points of the participants which were recorded during the task completion process also the navigation complexity, visited and revisited page numbers were traced. The researchers observed the participants and interviewed with them. Also for strengthening the process, think aloud protocol. This protocol was used to comprehend the process. It was explained that these data would help to verify the eye-tracking data. Moreover, the participants took a recall test to understand if there was a difference among groups. According to the results of this study, there was no significant effect of cognitive style on the participants’ navigational patterns. The researchers suggested that this study should be done with a wider sampling and this may give more reliable results in terms of the cognitive style differences. Ozcelik, Arslan and Cagiltay (unpublished manuscript), proposed a study about the effectiveness of signaling the pertinent keywords or sentence on the performance of the participants [13]. In this study, for determining the influence of signaling on reading they used eye-tracking device. The sampling consisted of twenty-eight undergraduate students, one group was given signaled keywords and the other group was given signaled sentence. The results of this study showed that there is no difference between the groups in terms of their performance. The retention and transfer scores were not influenced from the different designs. However, the eye-tracking data contributed the results of the study that signaling affected the students in a positive way, increased attention, less visual search and gave attention to the necessary information from the material. In the same vein, a similar study on signaling and its impacts on learning was conducted [14]. In this study, the researchers used two groups, one of them is signaled and the other one is nonsignaled. They preferred to use color signaling in their study, the transfer test was an indicator of their study in terms of the performance of the participants. The researchers pointed out the inconsistencies in the literature about the retention and transfer. Also, they emphasized the importance of eye-tracking in the study with giving the four main reasons that the signaling contributed to the meaningful learning. First of all, they stated that it was supported by the number of fixations on the necessary information. Secondly, they emphasized that much more time spent on the persistent information given that the gaze duration and average fixation supported this argument. Thirdly, they explained that the time of the visual search was decreased. Lastly, in terms of the effective positioning the relevant information, the data was beneficial since increased fixations gave valuable information about the process. Yet another study was conducted on the effects of color coding on multimedia learning and visual search of the participants [15]. The purpose of this study is to differentiate whether color-coded split texts and split format text make a change of the participants’ recall, transfer, matching performance and their eye tracking data. The researchers explained that color-coded materials enhance the recall and transfer
98
B. Dogusoy and K. Cagiltay
performance. On the other hand, they found that there was no significant difference between participants in terms of the matching performance. The eye tracking data contributed the results with giving information on the participants’ eye fixations on color-coded materials which was longer than the other material. The researchers emphasized the existent literature on the fixation duration and its relation with cognitive processing. Also, they stated that the positive effects were related with the cognitive processing. They suggested that future research should be done on this issue to find our whether there are other reasons or not. Dogusoy and Cagiltay (2007) studied on another issue by using eye-tracking approach [16]. They pointed out the importance of finding information and search engines’ role in this process. In this study, the researchers aimed to examine the possible search behavior of the participants with the aid of eye-tracking approach. An unfamiliar search engine was chosen and nine participants used this search engine for completing four predetermined tasks. According to the results of this study, it was observed that the participants generally had problems during the searching process with the search engine. They tended to behave in a centralized manner while looking for the necessary information that the eye-tracking data supported this result. In addition to this, it was reported that the participants seem to use an “icon-based approach” that the eye tracking data showed that bigger and colored icons were much more focused on the screen than the other icons. They suggested that there needs to be more research on visual based search engines and their formations. Since the sampling is limited, the future studies might be conducted with wider and diverse sampling. They explained that these kinds of studies may give hints about the effective design processes especially the icon and keyword based structures and gathering data from users might help for constructing more effective visual search engines. Another research study was conducted on an interesting issue, multimodal comprehension of language and graphics graphs with and without annotation [17]. They explained their aim as to determine the function of “annotated textual elements on graphs in multimodal documents” [17]. At this point, eye tracking was used for gathering data on the participants’ eye movements and this may help them to have information on the visual attention of the participants. In this study, the sampling consists of thirty-two participants and they were taken verbal and spatial working memory span tests. The results of this study showed that the eye movement characteristics changed in accordance with the annotations on graphs. Also they stated that if the fixation duration data showed the participants’ processing difficulties, this was supported by the results of the study that the participants’ fixation duration is greater on the Graph AOI in graph-only format than the annotated format. They explained it with the possibility of the cognitive attempts of the participants in the process. Also, it was evidence that the participants’ gaze time on the Graph AOI was greater in the graph-only format. They emphasized that the findings of this study is on the same parallel with Split Attention effect [18]. They proposed the future works importance and introduced some of the possibilities. They explained that a detailed analysis of the Graph AOI is essential and with this, it might be possible to comprehend the annotated textual elements function in the process. Moreover, experimental studies might be replicated to have more observable results and lastly post-test might be included to the procedure for determining the individual differences and the connections between the features of the eye-movements of participants.
An Innovative Way of Understanding Learning Processes: Eye Tracking
99
2 Conclusion and Discussion The results of the eye-tracking based research studies under the METU HCI group were presented [5] in this paper. Since eye-tracking enables the researchers to study on different dimensions of human and cognitive schema, this may provide beneficial clues for both educators and researchers in order to understand the learning process in depth. The studies which were presented in this paper aimed to show less explored aspects of the educational applications and the attempts on using different techniques and strategies. Learning is a very complex process and it is related with the human beings’ complex thinking structure. The main issue which was pointed out in this paper was the design of better learning environments. Many papers focused on this issue and they tried to represent different design possibilities and their effectiveness. It can be seen that eye-tracking approach has a potential to provide information on learning processes and especially the general view on user’s behaviors. Although many studies resulted in with “no significant difference”, eye tracking approach has potential to provide information on the process which helps to represent a bigger picture for analyzing the process more easily. Also, new developments in technologies and more studies in this field, especially the analysis process, enable us to use this device more effectively and more accurately. At this point, eye-tacking approach may also give beneficial information on the process of understanding the nature of the cognitive processing. The studies generally focused on the data related with the fixation duration and place which gives information about the visual attention. Moreover, the gaze replay data is beneficial to the researchers to analyze the process deeply which enable them to observe the participants in the task completion process. Major limitation in eye-tracking studies is related with the small sampling. It is really hard for researchers to generalize the results with limited sampling. Also, in the literature the limitation of the eye-tracking studies were explained as they might give some answers about the users’ behavior and their ocular behavior but understanding the reasons under this behavior is not an easily understandable by researchers [19]. Video recordings which provide data on the participants’ facial expressions and verbal protocols of participants may help researchers in eye-tracking research studies. Although there are limitations related with this approach, the potential of these kinds of studies should not be ignored so that they may assist instructional designers and researchers in order to better understanding the learning process of human beings with predicating the deficiencies of the designs and how they can be strengthen. Acknowledgments. This study was supported by TUBITAK under grant SOBAG 104K098 and METU Human Computer Interaction research group (http://hci.metu. edu.tr).
References 1. Just, M.A., Carpenter, P.A.: Using eye fixations to study reading comprehension. In: Kieras, D.E., Just, M.A. (eds.) New methods in reading comprehension research, pp. 151– 182. Erlbaum, Hillsdale (1984)
100
B. Dogusoy and K. Cagiltay
2. Rayner, K.: Eye movements and cognitive processes in reading, visual search, and scene perception. In: Findlay, J.M., Walker, R., Kentridge, R.W. (eds.) Eye movement research: Mechanisms, processes, and applications, pp. 3–21. Elsevier Science, New York (1995) 3. Rayner, K.: Eye movements in reading and information processing: Twenty years of research. Psychological Bulletin 124, 372–422 (1998) 4. Just, M.A., Carpenter, P.A.: Psychological Review: A Theory of Reading: From Eye Fixations to Comprehension. American Psychological Association 87, 4 (1980) 5. Middle East Tehnical University (METU) Human Computer Interaction (HCI), METU HCI Research Group (2007), http://hci.metu.edu.tr 6. Clearview User Manual. Tobii, Sweden (2006) 7. Baran, B., Dogusoy, B., Cagiltay, K.: How do adults solve digital tangram problems? Analyzing cognitive strategies through eye tracking approach. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 555–563. Springer, Heidelberg (2007) 8. Yecan, E., Çagiltay, K.: Cognitive Styles and Students’ Interaction with and Instructional Web-Site:Tracing Users through Eye-Gaze. In: The 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, Netherlands, 5-7 July (2006) 9. Yecan, E., Sumuer, E., Baran, B., Cagiltay, K.: Tracing Users’ Behaviors in a Multimodal Instructional Material: An Eye-Tracking Study. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 755–762. Springer, Heidelberg (2007) 10. Sancar, H., Dogusoy, B., Cagiltay, K., Cakiroglu, J.: Exploring Cognitive Process of Concept Map Construction: An Eye-Tracking Study. In: Association for Educational Communications and Technology (AECT), Orlando, USA (2008) 11. Sancar, H., Karakus, T., Cagiltay, K.: Learning a New Game: Usability, Gender and Education. In: 3rd Technology-Enhanced Learning Enlargement Workshop: Young Researchers for the European Future of the Technology-Enhanced Learning, Sofia, Bulgaria (2007) 12. Karakus, T., Sancar, H., Cagiltay, K.: An Eye Tracking Study: The Effects of the Individual Differences on Navigation Patterns and Recall Performance on Hypertexts Environments. In: ED-MEDIA, World Conference on Educational Multimedia, Hypermedia & Telecommunications, Vienna, Austria (2008) 13. Ozcelik, E., Arslan, I., Cagiltay, K.: Signaling the Keywords or Sentences: An Eyetracking Study (unpublished manuscript) 14. Arslan, İ., Ozcelik, E., Cagiltay, K.: Effect of signaling on multimedia learning: An eye tracking approach (unpublished manuscript) 15. Ozcelik, E., Karakus, T., Kursun, E., Cagiltay, K.: Color Coding Effect on Multimedia Learning and Visual Search: An eyetracking approach. In: The American Educational Research Association (AERA), Chicago, USA (2007) 16. Dogusoy, B., Cagiltay, K.: Evaluation of Visually Categorized Search Engine. In: 3rd Technology-Enhanced Learning Enlargement Workshop: Young Researchers for the European Future of the Technology-Enhanced Learning, Sofia, Bulgaria (2007) 17. Acarturk, C., Habel, C., Cagiltay, K., Alacam, O.: Multimodal Comprehension of Language and Graphicsgraphs with and without Annotation. ECEM. In: 14th European Conference on Eye Movements, Potsdam, Germany, August 19-23 (2007) 18. Chandler, P., Sweller, J.: The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology 62, 233–246 (1992) 19. Pan, B., Hembrooke, H., Gay, G., Granka, L., Feusner, M., Newman, J.: The Determinants of Web Page Viewing Behavior: An Eye Tracking Study. In: Proc. ETRA (2004)
A Set of Rules and Strategies for UNSAM Virtual Campus Jorge Fernández Niello1,2,5, Francisco V. Cipolla Ficarra3,4,5, Mario Greco1,5, Rodolfo Fernández-Ziegler1,5, Silvia Bernatené1,5, and Maria Villarreal1,5 1
Universidad Nacional de San Martín Comisión Nacional de Energía Atómica 3 ALAIPO: Asociación Latina de Interacción Persona-Ordenador 4 AINCI: Asociación Internacional de la Comunicación Interactiva 5 Ayacucho 2197 (1650) San Martín Provincia de Buenos Aires - Argentina [email protected], [email protected], {mario.greco,rziegler,sbernatene}@unisam.com.ar 2
Abstract. We present a first set of strategies for the establishment of a virtual campus. Additionally, a set of pedagogic and communicative rules is established with the purpose of achieving a better diffusion of the university contents among the students. These rules are constantly updated according to the teaching supply and demand, and the requirements of the teachers and the students. Additionally the different components of interactive education are present as well as their main functions in the virtual campus. Keywords: E-learning, Campus Virtual, Pedagogy, Design, Hypermedia, Moodle, Software, Hardware.
campus. Consequently, before setting in motion these projects it is necessary to analyze each one of the cost variables to avoid unforeseen circumstances in the future. Obviously, some of the main goals of distance teaching are: to increase the quality of education, to widen the teaching offered and to raise the social level of the community to which the university belongs, among others. In the current work a state of the art of the software and hardware within E-learning education and the first strategies and rules to foster the use of the virtual campus is presented. Simultaneously, the interface of the current system is made public as well as a series of attributes of quality to boost communicability.
2 Software and Hardware The university costs in these projects entail the training of the teaching body in the use of these technologies, the promotion inside and outside the university through corporative publicity, to generate multimedia material, to dedicate space and personalized staff in the management and maintenance of the software-hardware, etc. With regard to the software the virtual campus manages, currently the costs are minimal as compared to the investments made by other American, Asian and European universities in the mid-nineties. The price reduction lies in what is known as open software [7], [8]. Thanks to this there are systems such as Moodle (Modular Object-Oriented Dynamic Learning Environment) aimed at E-learning. Concerning this there are colleges that have implemented this software with excellent results among students and teachers. This saving in the budget can be aimed at the realization of multimedia teaching materials of excellent quality and which guarantee the constant updating throughout time. When it comes to hardware, in our case, one might have two mirror servers. that is to say, that in the case that one fails, the second is automatically activated, using Linux as an operating system, thus avoiding additional costs through the yearly renewal of licenses in software material as it happens with other operative systems. Both servers should have firewalls available. Two PCs with the same configuration may be a transitory solution in the case that one of them breaks down. All these computer devices and network must have available a continuous energy supply in the face of potential electrical blackouts and in rooms that comply with the quality rules for the correct functioning of the computer system, that is to say, air conditioners. We see how the costs in computer material have also gone down as compared with the realities of the past decade. In spite of this, there are still European universities, for instance, that continue using servers for the internal management of the university with the campus or virtual classroom, with very high maintenance and great updating costs with regard to server hardware components [9]. In our case, we have available a magnificent system for SIU-Guaraní academic management, with the possibility of making excellent detailed statistic graphics in real time. However, no aspect related to classrooms or virtual campuses is developed yet. Consequently, we have started to project the main strategies to reach that target in a short term. The SIU-Guaraní is a system of college information that administers academic management since the students enroll in college in a standardized way and profiting
A Set of Rules and Strategies for UNSAM Virtual Campus
103
Fig. 1. SIU-Guaraní system –database interface for academic gestion
data to the maximum. It has a single federal database, that is, it allows information to be added and for it to be reviewed in a decentralized way. Some of its main functionalities are planning, fees management, classroom management, exam management, administration and expedition of certificates, management of pupils polls, diffusion of memos and communiqués through e-mails or SMS (students, teachers, and academic authorities), management of several interfaces: follow-up of the students who have finished their studies, accountancy management, etc. and even has an interface with Moodle, but which is in the development stage and will be aimed at E-learning. Access is through the university intranet and internet. It fosters cooperative and cooperation work, starting from the interaction among the different sectors and actors (authorities/student staff/teachers/students) [10]. Additionally, here also are to be found accessibility bases; since it is an on-line system it makes possible its availability around the clock, 365 days of the year. From the point of view of software engineering it has the main attributes of quality, that is to say, privacy of information, reliability, flexibility, adaptability, etc. [11], [12], [13], [14]. Some of the operations the students can do are: examinations inscription, reviewing the study plan and academic record, seeing the chronology of the partial assessments, class agenda, requesting certificates, updating of personal data, reception of files and messages, etc. On their side, professors can look up the students who have registered to their examinations, to register the exam grades in the records, to consult the schedule of the exams, etc. In short, the SIU-Guaraní system is a very interesting tool of control over academic management and which makes easier to some extent some of the activities that the tutors/teachers have to carry out and the future students in the virtual classrooms.
104
J. Fernández Niello et al.
3 Human Factors In common with other public or private entities with a high number of people who interact among themselves, the human factor is essential to set in motion the projects, in order to avoid resistance to it. Within the context of the new or latest technologies in the education sector there are those whom Umberto Eco would call apocalyptic or integrated [15], that is to say, you are either in favour of or against their use. In the metropolis of the economically developed societies these problems virtually do not exist. It is a population that constantly includes new technologies in their daily life activities [16], [17]. In other places it is necessary to boost the motivation among its participants, always trying to establish a horizontal communication, among peers, for which it is necessary to count on accurate rules to be followed by the virtual community that is generated with e-learning, and so that the democratization phenomena of distance teaching may reach each one of the most distant places in a national and international territory [17], [18] since we live in Mc Luhan's ‘global village’ [20]. It is necessary to remember that we have overcome the era of computer use and we are going through the quality era of communicability [21]. It is also necessary to remember that many students do not finish their mid-level studies, therefore, a campus or virtual classroom can motivate them for such a purpose, resorting to the public institutions. There are countries in Southern Europe where these studies can be finished in two years resulting in some cases even to obtaining a masters degree. Oddly enough, these private teaching centres guarantee the finalization of studies in a 100% of cases. Obviously, this is a reality that belongs more to the commercial sector than to education. Now through social programs and having available sufficient economic and financial means, the causes of school-quitting might be traced, and thus put at the disposal of the potential students a whole range of pedagogical and technological resources to end the studies and later on, if they want to, they might follow their university studies. The idea here is to introduce to the students a quick way to study and even if they are working, they have the alternative of finishing the studies for which they have registered. In other words, a well-structured and systematically functional virtual campus can speed up the timing in the process of teaching and learning. Simultaneously, a virtual classroom may overcome the problems originated by the workload complaints of the university teaching and non-teaching staff and which usually end up in some places in long strikes. The students who have the virtual material can follow their exercises, see the exam schedule, see the grades, etc. It is a way of not damaging the students. A way to overcome the difficulties that the students usually have at the moment of having access for the first time to the virtual environments is through the implementation of a call center. The call center has to be active 24 hours per day, along all the weeks, including holidays. Perhaps a good strategy to overcome the human factors barriers in the first moments for the use of the instruments of the virtual classroom is to explain their main advantages, both for the teachers and the students, through a corporative or institutional information campaign, activation of an exclusive website with multimedial demos which explain the functioning of each one of the main
A Set of Rules and Strategies for UNSAM Virtual Campus
105
components of the virtual campus, training of tutoring students for the virtual community, etc. In the annex #1 a list with the main components. All these lines of action have to be coordinated and follow the common goal of the democratization of the use of the virtual campus, inside and outside the college facilities [3], [22], [23], [24], [25].
4 Campus Virtual UNSAM: First Step Next a series of active interfaces that have allowed navigation, and the first experiences with the main professors involved in the project: One of the solutions to set in motion projects of this stature consists in external advice through other university realities (in this case, PUC of Valparaíso). However, we must remember that it is feasible for us to insert a foreign model 100% in the university context since the sociocultural variables such as the mentality of the peoples changes from one state to another. However, the experiences of the bordering countries will always be more positive and adaptable to home reality than those coming from overseas. Without any doubt,
Fig. 2. UNSAM Virtual Campus: A simple iconography and minimal style desing of the interfaces. Two email options: intern. between students and professors/tutors (adove) and extern between the users and system administrator of virtual campus (down).
106
J. Fernández Niello et al.
Fig. 3. A ‘classic’ chat system and menu of the colors for texts. It is a public area for all users with autorization –virtual campus system aministrator.
Fig. 4. Marks and person responsable for evaluation, additional information, observations and details. In the right zone a list with all active virtual classrooms. Neat data and content disposition.
A Set of Rules and Strategies for UNSAM Virtual Campus
107
Fig. 5. Notes, summaries, practical homework, etc. option. Eassiness for attach files, add contents description and/or remarks, and selection of the categorie.
the current virtual campus contains activities that are currently carried out in the virtual environments aimed at E-learning. These are instruments which are not complicated to use, but which necessitate time for its learning by both tutors and students. The presented interfaces have made it clear that they have a very simple design, easy to use and not ambiguous [2], [26], [27]. The communicability for an expert user is optimal, however it is necessary to think of those users who have little experience in the use of computers. The activation and disactivation of the static and dynamic means, as well as their functioning, are similar to the main components of the Windows operating system [28].
5 Conclusions Establishing a virtual campus entails following some general and secondary targets akin to the former, where achieving the common good prevails among each one of its
108
J. Fernández Niello et al.
members whether it is as course organizers, multimedia content generators, Elearning tutors, active members of the virtual community, etc. The rules to be followed must be determined in a protocol of principles for the whole university community. These rules must respect usability and communicability of the interactive contents aimed at E-learning. A priori, the set of rules and strategies may seem to the final user of the system as a kind of normal pyramid, that is, at the top are the academic authorities, the professors, tutors, etc. However, it is not so, because it is precisely in the opposite sense. The switch of the pyramid is achieved thanks to the virtual community that is born from the E-learning process. That is to say, with the passing of time and through the use of the instruments provided by the new technologies of the virtual campus, we are in front of an inverted pyramid where we find the students at the summit. Besides, the use and circulation of free software make it possible to speed up this process. Axiomatically and undisputedly, it is necessary at the beginning to count with the greatest possible amount of available information, even from similar experiences in other universities and the cooperation of some outsourcing services. However, the final goal must be to transfer even these outer services inside the university context. In our case, we have available an efficient federal academic management system, but for the moment it lacks a virtual campus. Consequently, the set of rules and strategies presented are not static but rather dynamic, such as E-learning communication and education process. Finally, the costs concerning software and hardware have considerably lowered nowadays as compared to the past decade to set in motion a project of virtual campus. Acknowledgments. The authors wish to acknowledge all colleagues in our Universidad Nacional de San Martín and PUC of Valparaíso for the support and their helps.
References 1. Saussure, F.: Curso de Lingüística General. Losada, Buenos Aires (1982) 2. Cipolla-Ficarra, F.: Evaluation and communication techniques in multimedia product design for on the net university education. In: Multimedia on the Net, Vienna, pp. 151– 165. Springer, Heidelberg (1996) 3. Paymal, N.: Pedagogia 3000. Editorial Brujas, Cordoba (2008) 4. Eletti, V.: Che cos’è L’E-learning. Carocci, Roma (2002) 5. La Noce, F.: E-learning. Franco Angeli, Milano (2001) 6. Harasim, L., et al.: Learning Networks. MIT Press, Cambridge (1998) 7. O’Reilly, T.: Lessons from Open-Source Software Development. Communications of the ACM 42, 32–37 (1999) 8. Schlesinger, D.: Working with Open Source: A Practical Guide. Interactions 14, 35–37 (2007) 9. Lamborghini, B. et al.: European Information Technology Observatory. FrancoAngeli, Milano (2005) 10. Mamtyla, K.: Blending E-Learning: the Power is in the Mix. McGraw Hill, New York (2001) 11. Ternoven, I.: Support for Quality-Based Design and Inspection. IEEE Software 13, 44–54 (1996)
A Set of Rules and Strategies for UNSAM Virtual Campus
109
12. Frakes, W., Isoda, S.: Success Factors of Systematic Reuse. IEEE Software 11, 14–19 (1994) 13. Ozkaya, I., et al.: Making Practical Use of Quality Attribute Information. IEEE Software 25, 25–33 (2008) 14. Safayeni, F., et al.: Requirements Engineering in New Product Development. Communications of the ACM 51, 77–82 (2008) 15. Eco, U.: Apocalittici e integrati. Bompiani, Milano (2001) 16. Beverly, A.: Instructional and Cognitive Impacts of Web-based Education. Idea Group Publishing, Hershey (2000) 17. Toyama, K., Dias, B.: Information and Communication Technologies for Development. IEEE Computer 41, 22–25 (2008) 18. Ramakrishnan, R., Tomkins, A.: Toward a PeopleWeb. IEEE Computer 40, 63–72 (2007) 19. Balasubramaniam, S., et al.: Situated Software: Concepts, Motivation, Technology, and the Future. IEEE Software 25, 50–55 (2008) 20. McLuhan, M., Power, B.: The Global Village: Transformations in World Life and Media in the 21st Century. Oxford University Press, Oxford (1992) 21. Cipolla-Ficarra, F.: Communicability design and evaluation in cultural and ecological multimedia systems. In: Proc. 1st workshop Communicability MS 2008, pp. 1–8. ACM Press, New York (2008) 22. Trentin, G.: Insegnare e apprendere in rete. Zanichelli, Bologna (1998) 23. Gatti, R., Gherardi, V.: Le scienze dell’educazione. Carocci, Roma (1999) 24. Norman, D., Spohrer, J.: Learner-centered Education. Communications of the ACM 39, 24–27 (1996) 25. Dziuban, C., Moskal, P.: Evaluating Distributed Learning in Metropolitan Universities. Metropolitan Universities 1, 41–49 (2001) 26. Nielsen, J.: The Usability Engineering Life Cycle. IEEE Computer 2, 12–22 (1992) 27. Sharp, H., Rogers, Y., Preece, J.: Interaction Design: Beyond Human-Computer Interaction. John Wiley, West Sussex (2007) 28. Shneiderman, B.: Designing the User Interface. Addison Wesley, Massachusetts (1998)
110
J. Fernández Niello et al.
Annex #1: First Set of the Components for a Virtual Campus 1) Hypermedia On-line and Off-line –Design Categories − − − − −
Content Navigation Panchronism Presentation or layout; Structure
2) Quality Attributes − Acteme is the study of communication as behavioural system. − Adaptability of the content, that is, variety of styles of navigation, plenty of different media (text, graphics, maps, animations, videos, music and sounds), a wide control over active media. − Behaviour-animated help analyses the attributes universality, simplicity, originality and humour in animated pedagogical agents. − Competence considers how the systems adapts itself to skills of the main groups of users for navigation. − Isomorphism sets down a range of constant formal features among the different components and design categories, i.e., content, layout, structure, navigation, etc., that is, it researches those components that remain equal in the different categories and especially the topology of the elements’ distribution on the frame. Isomorphism seeks ‘regularity in irregularity’. − Motivation is the set of the dynamic means and structure resources that strengthen navigation in multimedia systems. − Semiosis for universal access makes reference to the main components that converge in this attribute for systems multimedia: linguistic, iconic, attention, narration and education/formation. − The control of fruition is the degree of autonomy in navigation that the structure of the multimedia system gives the user. − The naturalness of metaphor assesses the user’s ability to understand the set of images that make up the structure of the interface. − The phatic function asserts the direct communication in the human-computer interaction process without generating mistakes. − The transparency of meaning analyses the usage mainly of terms (also, images and sounds related to the words) of the interface that do not cause ambiguities between content and expression. − The triple dynamic coherence analyses the relationship between text, image and audio in the dynamic media, nevertheless, this attribute does not consider the synchronization between audio and dynamic image (animation and video). − Unlimited iconic consists of those icons and their components that, depending on the national and/or international cultural setting, have different meaning, for example, the gestures, the colours, the words, etc. − Acteme is the study of communication as behavioural system.
3) Experts − − − − −
Communicability Human-Computer Interaction Human-Computer Interface Pedagogy Software (analysis and programming) and Hardware (systems)
4) On-line Learning –Main Elements − − − − − −
Call center: info and problem resolution support Interactive communication: Email, chat, netmeeting, etc. Portals (website –Internet and intranet) Teaching materials: book style and support (digital and/or analogic), production, publishing, etc. Tutors: virtual or real Virtual community : integrants and administrators
HCI Professional Involvement in k-12 Education: On Target or Missing the Mark? Martin Jelin, Adrian Sudol, Jeffrey Damon, Douglas McCadden, and David Brown Computer Science Department, Maine School of Science and Mathematics, 95 High Street, Limestone ME, 04750, United States {sjelin,asudol,jpdamon,mccaddend,dr.david.brown}@gmail.org
Abstract. The state of learning across geographic, socioeconomic, age, and gender boundaries can be enhanced greatly by Human-Computer Information (HCI) infusion into blended learning [1][2][3] or Course Management System (CMS) software [4][3][5]. The major thrust of this paper is to examine problematic issues examined in popular software such as Moodle™ in which the HCI community could be beneficial. By regarding the ultimate students’ goal, i.e. grades, and the desirable benefit of course material understanding, one can develop an understanding of what CMS software needs and CMS software users expect on the high school level. Keywords: blended learning, e-learning, Course Management System (CMS), Virtual Leaning Environment (VLE), Human-Computer Interaction (HCI), k-12 Education, magnet schools.
1.1 Blended Learning and Course Management Systems (CMS) “Blended learning” is a method where students are being taught by two or more sources of information, generally traditional classrooms and online classrooms [10]. Often times, blended learning is implemented by using a Course Management System (CMS). CMS, also known as Virtual Learning Environments (VLE), are online tools for teachers to interact with their students outside of the physical classroom [8]. Interaction may occur in a variety of educational modalities.
2 Review of Literature Blended learning and CMS software can improve learning for all students [1][2][3][4][5]. The use of blended learning has been increasing quickly over the past few years, both in school and businesses [7][11][12]. Many college campuses have realized this and moved to CMS environments to streamline and augment organization, reporting, teaching and learning [1][9][13][14]. These programs have been generally well received, with both students and teachers finding the programs beneficial [6][9][13][14]. These institutions are funded well enough to absorb the costs of retail offerings such as First Class™, Blackboard™, and WebCT™ (now owned by Blackboard™), as well as technology training programs. High school funding does not allow this luxury. Moodle™ is an open-source CMS commonly used in high schools that cannot afford more expensive programs [8]. The software vehicle establishes the platform and potential groundswell of increased activity with HCI and the high school communities [8]. Moodle™, educational communities are international in scope with as many diverse uses as cultural approaches to education [2][3][15]. As a vehicle, Moodle™ has tools such as blogs, wikis, online tests, instruction, email, and asynchronous collaboration [8]. It's flexibility has caused it to outshine many other open source CMS programs, and even Blackboard™ in some cases [16][17]. Moodle™ is well known for focusing on student interaction, often considered the main advantage of blended learning. The educational philosophy that advocates this is known as social constructivism [8][18]. According to Berger and Luckmann [22], this theory posits knowledge is actually created by students and teachers through interaction with themselves and their environment. They contend that social constructivism is important to learning [22]. However, despite this focus on interaction, there is sparse research on the actual content of online student interaction [19]. A study of a blended learning class at Blackpool and The Fylde College found that 77% of students in the class preferred a normal learning environment, and more reflective students tended to avoid the online portion of the class as much as possible [9]. Martin Dougiamas, the creator of Moodle™, did a similar study in which he found Moodle™ did not even allow social learners to properly interact with each other [18]. Hetlighen and Anderson surmise that information overload is another large problem facing CMS users. This occurs when the students are given too much information absorbing little [20][21].
HCI Professional Involvement in k-12 Education: On Target or Missing the Mark?
113
A 2000 study by The US National Center for Education Statistics found that of k12 teachers who used the Internet, only 39% used it for instructional material, and just 16% used it for gathering information to plan class lessons [6].
3 Methodology The sample, drawn from a magnet school population in the Northeastern United States, contains grades 10-12. Socio-economic statuses range from multimillionaire families to wards of the state. Gender distribution is approximately 66% male and 34% female in a population of 114 students. Diversity of the population results in reduced bias. It should be noted that all students at the school have Internet 75% of the day. Two computer labs are available for a required, introductory technology class (32 students). Increased exposure to technology and high academic rigor could bias the survey responses toward CMS software and blended learning. The school uses the open-source CMS Moodle™ [8]. Approximately two thirds of the population used the CMS for at least one of their classes at the time of the data collection. The students were given an anonymous survey on their perception of Moodle’s™ influence. This volunteer survey resulted in 63% response. NonMoodle™ users’ responses were excised. The survey consisted of two parts. In the first part, students gave information about their general experience with Moodle™. In the second, students gave information about their experience specifically within their class which used Moodle™ the most. 3.1 Semantic Differential Students used the following semantic differential to rate effects: Table 1. Semantic Differential Choice Large Negative Effect Medium Negative Effect Small Negative Effect Small Positive Effect Medium Positive Effect Large Positive Effect Not Applicable or Not Used
Acronym LN MN SN SP MP LP N/A
4 Results Students were first asked to give information about their experience with Moodle™ in general. The students were asked to furnish data concerning their experiences with Moodle™, specifically within the class that used the CMS the most.
114
M. Jelin et al. Table 2. Moodle™ General Results Question What is your gender? What grade are you in?
Results Male – 26 Sophomore – 11 Senior – 16
How many classes are you taking this semester? About how often do you log onto Moodle™?
Four – 15 Five – 17 Seven – 4 Eight – 1 Never – 0 1 to 3 Days Per Month – 7 1 to 3 Days Per Week – 23 4 to 6 Days Per Week – 8 Daily – 10
About how often do you have difficulties when using Moodle™? Overall, what effect has Moodle™ had on your grades?
Never –13 Rare – 20 Mildly Often – 6 Very Often – 9 LN – 0 MN – 4 SN – 8 SP – 26 MP – 9 LP – 1
Overall, what effect has Moodle™ had on your understanding of the subjects of your classes?
LN – 1 SP – 25
Female – 22 Junior – 21
MN – 2 MP – 11
Four – 15 Seven – 4
SN – 7 LP – 0
Table 3. Moodle™ Intensive Class Results Question
Approximately how many days a week does your teacher required you to log onto Moodle™ for this class? Approximately how many days a week are you advised to log onto Moodle™ for this class? Approximately how many days a week do you log onto Moodle™ for this class? Rate the effect Moodle™ has had on your grade in the class.
Results Zero – 19 Three – 5 Six – 1
One – 8 Four – 3 Seven – 2
Two – 5 Five – 5
Zero – 6 Three – 4 Six – 2
One – 7 Four – 1 Seven – 14
Two – 6 Five – 8
Zero– 6 Three – 4 Six– 2
One – 11 Four – 7 Seven – 7
Two– 8 Five – 3
MN – 3 MP – 9
SN – 8 LP – 3
MN – 2 MP – 7
SN – 5 LP – 2
LN – 1 SP – 5 N/A – 23
MN – 1 MP – 7
SN – 6 LP – 5
LN – 1 SP – 10 N/A – 23
MN – 0 MP – 5
SN – 7 LP – 2
LN – 0 SP – 19 N/A – 6 Rate the effect Moodle™ has LN – 1 had on your understanding of SP – 23 N/A – 6 the class subject.
Rate the effect Moodle's™ Assignment Page has had on your grade in the class. Rate the effect Moodle's™ Assignment Page has had on your understanding of the class subject.
HCI Professional Involvement in k-12 Education: On Target or Missing the Mark?
Rate the effect Moodle's™ Weekly/Topic Outline has had on your grade in the class. Rate the effect Moodle's™ Weekly/Topic Outline has had on your understanding of the class subject. Rate the effect Moodle's™ Discussion Board (forums) has had on your grade in the class. Rate the effect Moodle's™ Discussion Board (forums) has had on your understanding of the class subject.
LN – 0 SP – 12 N/A – 19
MN – 1 MP – 9
SN – 4 LP – 3
LN – 0 SP – 15 N/A – 19
MN – 3 MP – 7
SN – 4 LP – 0
LN – 2 SP – 13 N/A – 23
MN – 0 MP – 4
SN – 5 LP – 1
LN – 2 SP – 11 N/A – 23
MN – 1 MP – 5
SN – 3 LP – 3
115
Results of “N/A” on the last few questions indicate that the student responding did not use that particular feature of Moodle™ in his or her class which used the CMS most. 4.1 Graphs Only data relevant to the conclusion were graphed.
25 20 15 10 5 0 Never
Rare
Mildly Often
Very Often
Fig. 1. Frequency of Difficulty
30 25 20 15 10 5 0
E ffect o n G rade E ffect o n Understanding
LN
MN
SN
SP
MP
Fig. 2. Overall Effect of Moodle™
LP
116
M. Jelin et al.
20 O v e ra ll Ef f e c t
15
A s s ignm e nt P a ge ' s E f f e c t
10
We e k ly O ut line ' s E f f e c t
5 0 LN
MN
SN
SP
MP
LP
D is c us s io n B o a rd ' s E f f e c t
Fig. 3. Moodle’s™ Effects on Grade by Components 25 20
O v e ra ll E f f e c t
15
A s s ignme nt P a ge 's E f f e c t
10
We e k ly O ut line 's E f f e c t
5 0 LN
MN
SN
SP
MP
LP
D is c us s io n B o a rd's E f f e c t
Fig. 4. Moodle’s™ Effects on Understanding by Components
4.2 Significant Differences Using the Kruskal-Wallace test, a nonparametric that compensated for unequal cell sizes, the following significant differences were obtained: Table 4. Significant Differences Independent Gender Frequency of Difficulty Frequency of Difficulty Frequency of Difficulty Frequency of Difficulty Frequency of Difficulty Class Subject Class Subject
Dependent Frequency of Difficulty Effect of CMS on Class Grade Effect of CMS on Subject Understanding Effect of CMS' Assignment Page on Subject Understanding Effect of CMS' Weekly Outline on Class Grade Effect of CMS' Weekly Outline on Subject Understanding Effect of CMS on Class Grade Effect of CMS on Subject Understanding
HCI Professional Involvement in k-12 Education: On Target or Missing the Mark?
117
5 Conclusion Seventy-three percent of the sample perceived problems with the CMS. The significant differences found indicate the importance of enhancing ease of use, ease of navigation, clarity of command and menu structures, as well as other factors of HCI. Poole’s study found no students having difficulties with their CMS [9]. This contrasts starkly with the 73% of students at the secondary school. The large difference demonstrates that studies on higher education’s use of blended learning cannot be generalized to the high school level. Of approximately 40 articles reviewed for this paper, only four scholarly sources could be found which focused on k-12 education. Current research on blended learning does not address the needs of secondary schools or the HCI components that may need reconstruction or clarification. A new body of research is needed for high schools. The significant difference between the genders when crossed with perceived difficulties gives a direction to improve CMS in high schools. Both psychological and sociological HCI concerns should be addressed. Historically, young females have addressed computers as antagonists, with some trepidation [23]. The current generation of students is facing a greater risk of information overload [21]. If the CMS adds to the learning curve, rather than simplifying the experience, the learning process becomes obfuscated. Therefore, a CMS needs to be as simple relative to HCI standards as possible. In follow-up conversations with the student body, students did not notice or take advantage of help pages. In fact, 80.4% of the sample was unable to locate and access the help modules. The help pages were perceived as confusing and detracting from the subject matter. Although Moodle™ is used throughout the world due to its opensource nature; it is believed that HCI professional community can contribute to its efficacy by improving HCI concerns. Despite the problems facing blended learning, the literature signifies great potential. Previous studies conducted on blended learning on the college level have shown that college students appreciate the process of blended learning [8][9][13][18]. High school students in this study believed that blended learning helped increase their knowledge acquisition (See Figure 2). As such, blended learning as an idea is not flawed, but needs more data based underpinnings. Students found the broader aspects of the information delivered by the CMS to be more relevant to both grades and understanding than those components that fostered depth. This may also lead to the supposition that a better navigational experience would have enhanced the students’ use of the blended learning tools (see Figure 3 and Figure 4).
Acknowledgements The authors would like to thank Gary Parker for his initial contributions, the student body for their participation in surveys and questionnaires, the faculty who responded to surveys and interviews, and an administration that facilitated the financial aspects of delivering this paper.
118
M. Jelin et al.
References 1. Garrison, D.R., Vaughan, N.D.: Blended Learning in Higher Education: Framework, Principles, and Guidelines. Jossey-Bass, Hoboken (2007) 2. Hargadon, S.: Implementation Study #3: Moodle. K-12 Open Technologies (2008), http://www.k12opentech.org/implementation-study-3-moodle 3. U.S. Department of Education, http://www.ed.gov 4. Voogt, J.M., Knezek, G.A.: International Handbook of Information Technology in Primary and Secondary Education. Springer, Heidelberg (2008) 5. Awidi, I.: Critical Factors in Selecting a Course Management System for Higher Education in Ghana. Educause Quaterly (2008), http://connect.educause.edu/Library/EDUCAUSE+Quarterly/ CriticalFactorsinSelectin/46028 6. National Center of Education Statistics, http://nces.ed.gov 7. Chapman, G.: Federal Support for Technology in K-12 Education. Brookings Papers on Education Policy, 307–343 (2000) 8. Moodle.org, http://www.moodle.org 9. Poole, J.: E-learning and learning styles: students’ reactions to web-based Language and Style at Blackpool and The Fylde College. Language and Literature 15(3), 307–320 (2006) 10. Voos, R.: Blended Learning: What it is and Where it Might Take Us? Sloan-C View 2(1) 11. Mullins, J.: E-Learning in the Inspector General Community: The Federal Government Is Using E-Learning to Raise Inspector General Skill Levels, Reduce Training Costs, and Increase Training Hours and Training Opportunities-Strengthening the Community’s Core Competencies Overall. The Public Manager 35(1), 46–50 (2006) 12. Carlivati, A.P.: E-learning evolve. ABA Banking Journal 94(6), 49–56 (2002) 13. Crisp, P.: E-learning and Language and Style in Hong Kong. Language and Literature 15(3), 277–290 (2006) 14. Plummer, P., Busse, B.: E-learning and Language and Style in Mainz and Münster. Language and Literature 15(3), 257–276 (2006) 15. Computers in the Classroom: Uses, Abuses, and Political Realities. EMCParadigm Publishing, http://www.emcp.com/intro_pc/reading1.html 16. Graf, S., List, B.: An Evaluation of Open Source E-Learning Platforms Stressing Adaptation Issues. In: Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies, pp. 163–165 (2005) 17. Munoz, K., Duzer, J.: Blackboard vs. Moodle: A Comparison of Satisfaction with Online Teaching and Learning Tools. Humboldt State University (2005), http://www.humboldt.edu/~jdv1/moodle/all.htm 18. Dougiamas, M., Taylor, P.C.: Interpretive analysis of an internet-based course constructed using a new courseware tool called Moodle. In: HERDSA 2002 Conference (2002) 19. Ziegler, M., Paulus, T., Woodside, M.: Creating a Climate of Engagement in a Blended Learning Environment. Journal of Interactive Learning Research 17(3), 295–308 (2006) 20. Hetlighen, F.: Change and Information Overload: negative effects. Principia Cybernetica Web (1999), http://pespmc1.vub.ac.be/chinneg.html 21. Anderson, N.: 2008: Year of Information Overload? Ars Techina (2007), http://arstechnica.com/news.ars/post/20071226-interruptionsinfo-overload-cost-us-economy-650-billion.html 22. Barger, P.L., Luckmann, T.: The Social Construction of Reality: A Treatise in the Sociology of Knowledge. Anchor Books (1966) 23. Beckwith, L., Burnett, M., Grigoreanu, V., Wiedenbeck, S.: Gender HCI: What About the Software? Computer (2006)
A Language Learning System Utilizing RFID Technology for Total Physical Response Activities Harumi Kashiwagi, Yan Xue, Yi Sun, Min Kang, and Kazuhiro Ohtsuki Kobe University, 1-2-1 Tsurukabuto, Nada, Kobe, 6578501, Japan {kasiwagi,kang,ohtsuki}@kobe-u.ac.jp, [email protected], [email protected]
Abstract. In this paper, we present a method of integrating a CALL system with RFID tags in a classroom in order to provide the basic support for listening activities based on the concept of Total Physical Response (TPR) Approach. We designed and developed a prototype system with the function of providing corrective feedback. The prototype system has the following three features: (1) Real objects are used as the options for responding to audio questions. (2) A considerable amount of attribute information is used to increase the variation in the questions. (3) The system has a function of providing error messages and additional questions, depending on the degree of a learner’s mistakes. Results from the experiment suggest that integrating real objects into the learning system by using RFID tags has a potential impact on the language learners. Keywords: RFID, Language learning, TPR, Corrective feedback, Interaction.
used as the options for responding to audio questions. The respective object is linked to its attribute information. (2) A considerable amount of attribute information is also used to increase the variation in the questions. Thus, multiple target listening points are included in one question. (3) The system has a function that provides not only appropriate error messages depending on the learner’s incorrect responses but also provides additional questions as supplementary exercises, depending on the degree of the learner’s mistakes. The remainder of the paper is organized as follows. Chapter2 describes the basic concept of the system. In chapter 3, the outline and features of the prototype system have been discussed. Chapter 4 presents the implementation of the system, and chapter 5 explains the experiment and its results. Finally, the conclusions as well as future suggestions are presented in chapter 6.
2 System Concept In this chapter, the basic concept of the system has been described. This includes the concept of the TPR approach, the manner in which the questions and answer options should be arranged so as to cope with the variation in and difficulty level of the questions, and the manner in which feedback messages should be prepared in response to the learner’s mistakes. 2.1 The TPR Approach The TPR approach is a method in which teachers first provide a command in the target language, and then perform the corresponding action with the students. In this manner, the students learn not only by observing the actions but also by performing them[7]. In lessons that employ the TPR approach, teachers should avoid introducing new commands in quick succession. It is generally recommended that they present three commands at a time. As the students increase their knowledge of the target language, a longer series of connected commands can be provided. In this method, the actions clarify the meaning of the commands. Moreover, memory is also activated by connecting the actions with the language. From the teachers’ perspective, they will be able to immediately determine whether or not the students understand the command by observing their actions. The prototype system designed and developed in this study is based on this method. 2.2 Question and Feedback Structure Varying the Question Sentence Patterns and Controlling the Difficulty Level of the Questions. When creating listening exercises, it is necessary to consider the variation in the question sentence patterns and the difficulty level or complexity of the questions. The latter is related to certain aspect of language, i.e., vocabulary and phrases, grammatical structures, and length of sentences. Thus, a listening sentence that includes many new words and high-level grammatical structures will be difficult to comprehend. In addition to vocabulary and grammatical complexity, the difficulty level of a question will also be affected by the length of the sentence[10]. Therefore, vocabulary, grammatical structures, and length of sentences are the determinants of
A Language Learning System Utilizing RFID Technology for TPR Activities
121
the difficulty level of questions. However, the following can be employed to reduce the difficulty level. 1. Give specific examples: When using terms such as “carbonated drinks” in a question, provide specific examples such as “Coke” or “Sprite.” 2. Give external characteristics: When the word “glue” is mentioned in a question, provide supplementary information about its external characteristics such as "It has the shape of stick and has a blue cap." 3. Simplify the text: Consider the following sentence, "Please give me cold lemon juice in a plastic bottle." This text can be simplified by breaking it up into two sentences, such as "I want a cold drink. A lemon juice is good, and a plastic bottle would be better." Thus, providing examples, giving supplementary information, and breaking up the questions into manageable parts is helpful for the learners. Controlling the Difficulty Level of the Questions by Varying the Response Options. The second factor to be considered is the response options. Typically, teachers instruct mixed-level learners, and hence, by observing the level of each learner, they need to keep changing the difficulty level of the questions. However, we can also control the difficulty level of the questions by varying the response options to the questions. For example, when a question such as "Please choose cold lemon juice which costs ¥130" is presented, the learner will attempt to listen selectively for specific information such as cold, lemon juice, and ¥130. If the learner is required to choose a response from the following four options, he/she will have to listen to the question and identify all the key words: (1) cold, lemon juice, and ¥130; (2) cold, lemon juice, and ¥150; (3) cold, apple juice, and ¥130; (4)hot, lemon juice, and ¥130. However, if the learner is given the following options, he/she will be correct even if only one key word is correctly identified: (1) cold, lemon juice, and ¥130; (2) hot, orange juice, and ¥200; (3) cold, banana juice, and ¥150; (4) cold, apple juice, and ¥150. In this manner, the difficulty level of questions can be controlled, depending on the learner’s understanding, by merely changing the response options. This method would be helpful particularly when teaching lower-level learners. Thus, when developing our system, we introduced the above-mentioned concept of changing the objects that are used in the response options. Corrective Feedback. In general, teachers react to the mistakes made by learners in many different ways. One such manner is by providing feedback; it helps the learners in altering their outputs in a constructive manner. The typical types of corrective reactions have been described in Chaudron's research[4,5] in the following manner: _Emphasis: Using repetition, or questions, to identify the aspect that is incorrect _Explanation: Providing information on the different causes of the error _Original question: Repeating the original question _Altered question: Altering the original question syntactically, but not semantically _Verification: Assuring that the correction has been understood
122
H. Kashiwagi et al.
In addition, the following can be regarded as the other types of corrective reactions: _Selecting & reviewing: Selecting & reviewing important points in order to guide the learners toward the right direction _Breaking down: Breaking down the learning items into smaller parts/steps to ease understanding Based on the above-mentioned works, we considered the function of providing feedback in the prototype system.
3 Prototype System 3.1 Flow of Instructions Fig. 1 illustrates the prototype system developed in this study. When a learner scans a question card, the system reads out the corresponding message, e.g., “Please choose cold lemon juice in a plastic bottle” (Fig. 2).
Fig. 1. Prototype system
Fig. 2. Method to use the system
The flow of instructions has been presented in Fig. 3. When the learner responds to the question by choosing an appropriate object and allowing the scanner to scan it, the system will check his/her answer. For example, if the learner chooses a cold, orange juice can, the system will detect an error in the following two attributes: kind and container type. Thereafter, it will detect the degree of the error, i.e., the data pertaining to two attributes among the three targeted attributes are wrong. These errors are represented as follows: cold = cold, lemon ≠ orange, and plastic bottle ≠ can. Then, the system will read out an error message, e.g., “No, it is not lemon juice. It is not in a plastic bottle,” followed by an additional question regarding the wrong response, e.g., “Please choose lemon juice.” If the learner provides the correct response in this case, the system would state, “OK, good,” and then read out the second additional question for another wrong response, i.e., “Please choose a plastic bottle.” If the learner correctly responds to all the additional questions, the system would again read out the initial question, i.e., “Please choose cold lemon juice in a plastic bottle.” At this instance, when the learner responds correctly, the system will move on to the next question.
A Language Learning g System Utilizing RFID Technology for TPR Activities
123
Fig. 3. Flow of instructions
3.2 System Features The Use of Real Objects for Providing Responses. In this system, learners aree required to choose an appro opriate object after listening to the question. The systtem checks whether or not the object selected is correct. In order to realize this, real objects with RFID tags are used u as the response options. The respective objects are linked to a considerable amount a of attribute information that includes factual ddata about the object, such as kind, k price, container type, and size, as shown in Fig. 44. A set of correct attribute inforrmation is also prepared in advance for each question. T The system checks the responses made by the learners by comparing the difference between the attribute data of the t correct response and that of the selected response. Questions with Multiple Target T Listening Points. The attribute information of eeach object is also used to createe questions with multiple target listening points. Here, the target listening points referr to the points in a question that a learner is requiredd to listen to. For instance, in the t question “Please choose cold lemon juice in a plaastic bottle,” the target listening points are cold, lemon juice, and plastic bottle. As m mentioned above, a set of correect attribute information is prepared for each question. F Further, a question sentence paattern is also prepared in advance. For example, in Figg. 5, the attribute data—(1) cold d, (2) lemon juice, and (3) plastic bottle—are set as the correct attribute information n. When we frame a question using the above data, “Pleease choose [hot/cold] [kind of juice] j in a [container type],” the following question willl be read out: “Please choose co old lemon juice in a plastic bottle.” Error Messages and Addittional Questions. When considering feedback messages and questions, many of the exissting CALL materials, including the commercially availaable ones, only have a “Good-W Wrong” feedback function, leaving much to be desired. In this respect, more detailed feedb back and step-by-step support can be expected, dependingg on the learner's level of understtanding. Taking the above-mentioned into consideration, the prototype system verifies th he responses given by the learner and reads out correspoonding error messages as well as a additional questions. These error messages and additioonal questions are produced baseed on the degree of the learner’s mistakes.
124
H. Kashiwagi et al.
Fig. 4. Attribute information n of the objects
Fig. 5. Process of creating a question
Fig. 6 illustrates the man nner in which error messages and additional questions are selected and produced. 1. To count the number of incorrect i attributes (n) in a question 2. To calculate the ratio (n//m) between the number of incorrect attributes (n) and the total number of attributees in one question (m), by which the degree of the learnner's mistakes are graded baseed on the error level 3. To determine the scenariio pattern (feedback pattern) of the error messages and the additional questions, whiich are adapted to the learner's error level 4. To produce error messag ges and additional questions in the decided scenario pattern For example, when the learner listens to the question “Please choose cold lem mon juice in a plastic bottle” and chooses a cold, orange juice in a can, the system deteects that the attributes of kind and a container type are incorrect. It then calculates the degree of the error, i.e., two of the three targeted attributes are wrong, representedd as cold = cold, lemon ≠ orang ge, and plastic bottle ≠ can. In this case, the error leveel is determined to be Level B, and the scenario pattern decided is S3. The system tthen reads out the error messagee, “No, it is not lemon juice. It is not in a plastic botttle.” Thereafter, it reads out an additional question for the wrong answer, “Please chooose lemon juice.” If the learnerr provides the correct answer, it reads out the second quuestion, “Please choose a plasstic bottle.” If the learner answers all the additional quuestions correctly, the system reads r out the initial question again.
4 Implementation The prototype system is ou utlined in Fig. 7. The system consists of a PC, an RFID tag reader unit (RFID-RS232C C READ/WRITE BOARD), and question cards and rreal objects with RFID tags (Ph hilips, Hitag2). The tag unit can read data from and w write data into the RFID tags witthin a distance of about 5 cm. The system is implemennted using Java and JavaScript. The PC functions as a local web server in the system. As shown in Fig. 7, two thread ds, i.e., the thread of detecting the scanning activity of the RFID tags and that of obseerving the clipboard, are developed by Java; the questtion sentences, response checkin ng, error messages, and additional questions are develooped by JavaScript. JavaScript iss used because the patterns of the questions and feedbback messages can easily be addeed.
A Language Learning System Utilizing RFID Technology for TPR Activities
Fig. 6. Process of producing feedbacks
125
Fig. 7. Outline of the prototype system
The process of how the system works is described in Fig. 7. Here, one question sentence pattern and one feedback message pattern are implemented in the prototype system. The information about the question number and the kind of card (a question card or an answer option card) is transferred by using the method of “get.” The information representing whether the system produces an initial question or an additional question, correct attribute information for each question, and attribute information of the object that the learner chose as the response object, are transferred by using a “Cookie.” First, when a question card with an RFID tag is scanned, the related information is read by using the “get” method in Thread 1 of the Java application. The information about the kind of card and the question number are obtained and embedded into a URL. Then, Internet Explorer is activated with the URL (http://localhost/sample/index.html?k=40&p4=0001) in which the query string is added. In Fig. 7, k represents the kind of card, and p4 represents the question number. The file index.html provides a question or checks a response based on the information of the kind of card that is scanned. If a question is provided, the file quest.html is activated; on the other hand, when an answer is provided, the file judge.html is activated. Further, the file quest.html provides an initial question or an additional question based on the information maintained in the “Cookie.” If the initial question is to be produced, quest0.html is activated, and if the additional question is to be produced, quest1.html is activated. Similarly, with regard to the file judge.html, the “Cookie” determines whether the response is for the initial question or for the additional question. If it is for the initial question, judge0.html is activated, and if it is for the additional question, judge1.html is activated. In Thread 2, the system observes the clipboard, and produces questions or feedback messages in synthetic voices.
5 Experiment 5.1 Participants and Procedures Thirteen students were selected to evaluate the system; their evaluations were assimilated with the help of a questionnaire. There were seven undergraduates, five graduates, and one postgraduate. Two exchange students from the U.S. whose native language was English were among the seven undergraduates. They worked as
126
H. Kashiwagi et al.
teaching assistants in the free conversation room at the University. After providing them with the necessary instructions on how to use the system, each student attempted to use the system with two different feedbacks: (1) only with “OK, good”, and “Wrong, Once again,” without detailed error messages or additional questions, and (2) with detailed error messages and additional questions. Subsequently, they were required to complete a questionnaire on the usability of the system, the use of real objects as response options, and the feedback messages and additional questions depending on the degree of the learners’ mistakes. 5.2 Results and Discussion System Usability and the Use of Real Objects as Learning Materials. According to the results of the questionnaire, as shown in Table 1, all the participants were of the opinion that the system with RFID tags was easy to use. Further, twelve of them agreed to the question “Is using real objects as learning materials useful for learning vocabulary.” The results suggested that the system does have a potential impact on language learners, albeit further empirical studies are required. Detailed Feedback, Depending on the Degree of the Learner’s Mistakes. Participants were required to choose the statements that best described what they observed from among the options presented in Table 2, on the basis of the two abovementioned feedback functions. According to the results, (1) 70%–80% of the participants stated that when they used the system with detailed error messages and additional questions, they could follow the exercises by themselves; further, they found that the error messages were effective in identifying their mistakes. (2) Half of them believed that this function could enhance the interaction between the system and the learner. In fact, some of them also commented that the “Good-Wrong” feedback function was monotonous and that the detailed error messages and additional Table 1. Results of the questionnaire on system usability and use of real objects Questionnaire Yes Is the system with RFID tags easy to use? 13 (100%) Is using real objects as learning materials useful for 12 (92%) learning vocabulary?
No 0 (0%) 1 (8%)
Table 2. Results of the questionnaire on the two feedback functions Statements with regard to the two feedback functions We can identify our mistakes with the help of error messages. We can follow the exercises with the help of feedback messages. This function can enhance the interaction between the system and the learner.
“Good-Wrong” feedback 3 (23%)
Detailed feedback
4 (31%)
9 (69%)
2 (15%)
6 (46%)
10 (77%)
A Language Learning System Utilizing RFID Technology for TPR Activities
127
questions, depending on the degree of the learner’s mistakes, were good because each learner’s mistake is different from the others. Based on these results, it appears that the participants showed positive responses toward detailed error messages and additional questions, depending on the degree of the learner’s mistakes. At the same time, they also stated that variation in the feedback messages should be increased. Additional Questions. Finally, the participants were required to choose the statements that best described their opinion from among the options presented in Table 3, with regard to additional questions. In Table 3, the results are classified into two groups—native speakers (NS) and nonnative speakers (NNS). According to the results, the two exchange students (NS) agreed that one additional question was sufficient for multiple errors, while the seven NNS believed that one additional question was required for every error. Table 3. Results of the questionnaire on additional questions Statements regarding additional questions One additional question is sufficient for multiple errors. It is better to provide an additional question for the error. We can check each mistake with the help of additional questions. Additional questions are short and easy to understand.
NS (two participants) 2
NNS (eleven participants) 0
0
7
0
6
1
5
This result is consistent with the result that about 50% of the NNS felt that they could check each error with the help of additional questions. Further, five NNS agreed that the additional questions were short and easy to understand. Based on these results, it can be stated that feedback messages and short question sentences are helpful for learners who are unfamiliar with listening to a foreign language. It is expected that feedback comprising one additional question for an error is sufficient for helping foreign language learners.
6 Conclusion In this study, we proposed a CALL system with RFID tags in order to provide the basic support for listening activities in a classroom. Results from the experiment suggest that integrating real objects into the learning system by using RFID tags has a potential impact on language learning. Further studies on the variation in the feedback messages and additional questions are required to improve the system. It is assumed that the present prototype system can only be used by one user at a time. The individual user is not authenticated, and the learning history data are not saved. Using an RFID tag card as an identification card and saving history data on the RFID tags would help cope with these issues.
128
H. Kashiwagi et al.
References 1. Akiyama, I., Suenaga, S., Matsumura, Y., Minegishi, Y., Mano, S., & Iguchi, N.: The Structure of the IC Tag and its Impact. Soft Research Center (2004) (in Japanese) 2. Bandoh, H., Sato, H., Otsuki, Y., Baba, Y., Sawada, S., Ono, K.: Design and Trial Production of a Tool for Active Playtime in Preschools by Using RFID. Information Processing Society of Japan, IPSJ SIG Notes 2006(74) 2006-CE-85-(6), 41–48 (2006) (in Japanese) 3. Bhuptani, M., Moradpour, S.: RFID Field Guide. Prentice-Hall, Englewood Cliffs (2005) 4. Chaudron, C.: A descriptive model of discourse in the corrective treatment of learners’ errors’. Language Learning 27, 29–46 (1977) 5. Chaudron, C.: The role of error correction in second language teaching. In: Das, B.K. (ed.) Patterns of classroom interaction in Southeast Asia. RLC Anthology Series, vol. 17, pp. 17–50. Regional Language Centre, Singapore (1987) 6. Ito, E., Kojima, T., Takeuchi, H., Aoki, T., Miyazaki, S., Todoroki, S., Kitazawa, S., Yonekubo, S., Kazama, T.: Voice Output Communication Aid Used by the Picture Card with RFID. In: Proc. of the 21st Japanese Conference on the Advancement of Assistive and Rehabilitation Technology, pp. 327–328 (2006) (in Japanese) 7. Larsen-Freeman, D.: Techniques and Principles in Language Teaching. Oxford University Press, Oxford (2000) 8. Ogata, H., Akamatsu, R., Yano, Y.: Computer supported ubiquitous learning environment for vocabulary learning using RFID tags, TEL (Technology Enhanced Learning) 2004, France (2004) 9. Ogata, H., Yano, Y.: CLUE: Computer Supported Ubiquitous Learning Environment for Language Learning. Transactions of Information Processing Society of Japan 45(10), 2354–2363 (2004) (in Japanese) 10. Wilson, J.J.: How to teach listening. Pearson Longman (2008) 11. Yatani, K., Onuma, M., Sugimoto, M., Kusunoki, F.: Musex: A System for Supporting Children’s Collaborative Learning in a Museum with PDAs. The Transactions of the Institute of Electronics, Information and Communication Engineers J86-D-I(10), 773–782 (2003) (in Japanese)
Promoting Metacognition in Immersive Cultural Learning Environments H. Chad Lane University of Southern California Institute for Creative Technologies Marina del Rey, CA [email protected]
Abstract. Metacognition, defined as active control over cognitive processes during learning, is a critical component in the development of intercultural competence. Progression through stages of intercultural development requires self-assessment, self-monitoring, predictive, planning, and reflection skills. Modern virtual learning environments now provide a level of immersion that enable meaningful practice of cultural skills, both in terms of visual and experiential fidelity. This paper discusses their potential role in intercultural training, and the use of intelligent tutoring and experience manipulation techniques to support metacognitive and intercultural development. Techniques for adapting the behaviors of virtual humans to promote cultural learning are discussed along with the role of explicit feedback. The paper concludes with several suggestions for future research, including the use of existing intercultural development metrics for evaluating learning in immersive environments and on the balance between implicit and explicit feedback to establish optimal conditions for acquiring intercultural competence. Keywords: intercultural competence, metacognition, intelligent tutoring systems, immersive learning environments, experience manipulation.
way, training should be built around identifiable stages of development [3,8,9]. For example, if one is rushed to the point of behavior adjustment with limited or no understanding of the underlying cultural reasons, it could greatly hinder their overall development. True intercultural competence requires (at least) a heightened sense of self-awareness, an ability to self-assess, enhanced perceptive abilities, and a proclivity to reflect on experience. In other words, intercultural development requires concomitant metacognitive growth. This paper is about this process and how immersive learning environments and artificial intelligence can be used to promote intercultural learning. Metacognition involves active control over cognitive processes during learning. For example, when a learner is solving an algebra equation, cognition refers to the activities necessary to solve the problem, such as identifying rules to apply, applying them, finding a solution, and so on. Metacognition refers to a higher order of thinking that operates on these cognitive activities, such as planning, analyzing, assessing, monitoring, and reflecting on problem solving decisions and performance. Metacognition also enables more effective learning. A learner who is able to accurately gauge his or her own understanding is better equipped monitor his or her own progress and make productive decisions. This typically involves self-questioning and is part of the larger notion of metacognitive regulation [5]. Metacognitive skills can be taught. Numerous classroom studies have shown that explicitly teaching metacognitive strategies in the context of a specific domain (e.g., physics) can improve learning outcomes [4, p.19]. Strategies taught in these studies integrate metacognitive activities with cognitive and seek to make the steps of analyzing, planning, assessing, and reflection habitual in the learner. Studies have also shown that learning is more effective when learners explain worked out solutions to themselves. This phenomenon, which better learners do spontaneously, is known as the self-explanation effect. It is also possible to scaffold and promote the use of selfexplanations [6]. More recently, computer tutors focusing on teaching metacognitive skills have shown positive effects on learning behaviors (e.g., [1]).
2 Developing Intercultural Competence The Development Model of Intercultural Sensitivity (DMIS), developed by Bennett [3], is a model of intercultural development that has undergone rigorous validation. It is intended to explain how people construe cultural difference and how this ability
Fig. 1. The Developmental Model of Intercultural Sensitivity (DMIS) [3]
Promoting Metacognition in Immersive Cultural Learning Environments
131
becomes more flexible with time. By construe, Bennett is referring to Kelly’s [11] constructivist view that experience is a function of how one assigns meaning to events that occur in their lives. It is not simply a matter of being present during some event or set of events, but rather how those events are interpreted, encoded into memory, and later remembered. An underlying assumption of the DMIS is that as one’s ability to construe cultural differences evolves, intercultural competence also increases. According to Bennett, “it is the construction of reality as increasingly capable of accommodating cultural difference that constitutes development” [3, p. 24]. What constitutes a cultural difference for someone? It depends on that person’s cultural worldview, which is defined as the set of distinctions the person draws from to construe events in the world. A monocultural person – one who has primarily experienced a single culture in his or her life – will be unable to construe perceived differences from outside that cultural worldview. On the other hand, a person with a broader understanding is generally able to understand, even assume, other cultural worldviews. Hammer and Bennett summarize: “The crux of the development of intercultural sensitivity is attaining the ability to construe (and thus to experience) cultural difference in more complex ways” [8, p.423]. The DMIS is posits two broad worldview orientations: ethnocentrism and ethnorelativism. Each consists of three stages described below. The model is summarized in figure 1. Ethnocentrism is defined as an assumption that “the worldview of one’s own culture is central to all reality” [3, p.30]. Further, an ethnocentric person will implicitly assume that all others share this same worldview. The first ethnocentric stage is denial of difference in which the learner ignores or neglects differences. The next stage is defense against difference which includes recognition of cultural difference, but with negative evaluation. This stage is characterized by an “us vs. them” mindset and overt, negative stereotyping. The last ethnocentric stage is minimization of difference and includes the first signs of considering another cultural worldview. In this stage, the learner emphasizes similarities between cultures and recognizes only superficial cultural differences. Comments such as “we are all the same” are common at this stage. Guidance is especially important because some learners believe minimization is the ultimate stage of growth. When reality sets in that cultural differences are truly significant, there is a risk of withdrawal [3, p. 44]. The remaining three stages represent a shift to the ethnorelative orientation and are characterized by a basic understanding that one’s culture is but one out of many valid worldviews. The first ethnorelative stage is acceptance of difference in which the learner recognizes and appreciates cultural differences. Cultural difference evokes positive feelings, such as curiosity, in the learner for the first time. In the next stage, adaptation to difference, the learner makes an asserted effort to take the perspective of others. Because of this new perspective-taking or “frame shifting” ability, the learner can more easily interact with people from other cultures. The final stage is integration of difference: the learner has internalized multiple cultural worldviews and can easily assume different perspectives. Integration is an advanced stage often requiring years of experience to achieve. Metacognitive skills are critical for advancement through the DMIS stages. Given that the model is based on how one construes cultural differences, it follows that a learner must become aware of the construal process [11], how it works and how to assess their own ability to flexibly construe observed differences. The following metacognitive skills (at least) are related to the DMIS:
132
H.C. Lane
• Enhanced perceptive abilities are needed to consciously recognize cultural differences without reacting to them immediately. • Self-assessment shortcomings can hinder progress through the DMIS stages if the learner is not able to identify communicative failures or misunderstandings. This feeds into self-monitoring and tracking through the stages. • Cultural self-awareness, defined as an understanding of one’s own culture, is an important aspect of movement from ethnocentrism to ethnorelativism [2]. • Self-regulated learning is necessary for advancement: “Generally, people in the later phase of adaptation know how to orchestrate their own learning" [3, p. 59]. • Planning and goal-setting can support progression through DMIS stages, such as seeking to reach a specific stage or to understand specific cultural differences. Bennett addresses metacognition in his discussion of the final stage, integration of difference. He explains: “By being conscious of this dynamic process, people can function in relationship to cultures while staying outside the constraints of any particular one.” [3, p.60]. Of course, reaching this stage typically requires years of firsthand experience and is out of scope for any training program; but, it does make a strong case for nurturing an intercultural learner’s metacognitive skills. This idea is consistent with many intercultural training programs [9]. The rest of this paper explores how immersive learning environments may provide additional support.
3 Intelligent Techniques for Guided Cultural Learning 3.1 Immersive Cultural Learning Environments and Virtual Humans Technologies such as virtual reality and photoreal 3D graphics are particularly important when considering cross-cultural training. High fidelity simulations make it possible to create realistic portrayals of the products of different cultures, such as architecture, art, dress, sounds, language, attitudes, and even smells. This can promote the learner’s sense of immersion and provide a foundation for identifying cultural differences. Two such environments are shown in figure 2. The first is the Tactical Language and Culture Training System (TLCTS) developed by Alelo, Inc. [10]. TLCTS provides an experiential learning environment for acquiring language and cultural skills. In the mission environment (from Tactical Dari, shown on the left side of figure 2), a learner is free to explore an Afghani village, hear the sounds, speak to locals in Arabic using free speech, and make gestures. The clothing, buildings, and surroundings are realistic and thus can give a learner a sense of what it might be like to walk around an actual village. In this way, the system is already in a position to aid in the learner’s identification of cultural differences. The screenshot on the right in figure 2 is from the Adaptive Thinking and Leadership (ATL) simulation game [14]. ATL is a team-training system that uses human role players for both sides in intercultural scenarios. In-game assessment is performed by peers and instructors who observe play and after-action review (AAR) facilities are available to convey the outcomes to trainees. Learners are often assigned to role play as people from different cultures, with appropriate backstories and goals. Role-playing is a well-developed technique in the cross- cultural training literature and consistent with the DMIS with respect to the goal of understanding different cultural worldviews.
Promoting Metacognition in Immersive Cultural Learning Environments
133
Fig. 2. (Left) The Tactical Language and Culture Training System (Tactical Dari) by Alelo, Inc. [10] and (right) the Adaptive Thinking and Leadership simulation [14]. Reproduced with permission.
In multi-player environments, like the ATL system, inhabitants are human roleplayers. This can be costly and sometimes challenging to control from an educator’s point of view. Research on virtual humans provides an alternative or supplement to cultural team-training in immersive learning environments. Virtual humans combine artificial intelligence (AI) research in cultural and emotional modeling, speech processing, dialogue management, natural language understanding, and gesturing, among others, to enable natural feeling communication and interaction with computercontrolled characters that listen and respond to the user. Virtual humans are driven by detailed models of tasks, emotion, body language, and communication [16]. The underlying representations readily support explanation, which can be useful for learning [7]. In the case of intercultural education, it is important to endow virtual humans with the ability to explain their actions and reactions in terms of their cultural worldview. It is also important that their behavior be controllable in order to establish conditions that best promote learning. 3.2 Experience Manipulation and Implicit Feedback Generally speaking, computer simulations simulate real world phenomena as accurately as possible. There are circumstances when it is appropriate to consider goals other than fidelity when deciding how a simulation should behave and what events should occur. For instance, to enhance entertainment value, a popular basketball video game in the 80’s included special modes that allowed players to jump well over ten feet high and be “on fire”. In this case, the goal of entertaining the player trumps the goal of simulating basketball completely realistically. In the case of learning, the same idea applies: if a certain event or situation will promote learning, then the simulation should seek to produce that experience. The caveat is that in the case of learning, the adaptations should probably fall within bounds of feasibility. We refer to this general technique as experience manipulation and now discuss several ways it might be used to promote metacognitive growth and cultural learning.
134
H.C. Lane
Fig. 3. Expressions of anger, skepticism, and appreciation by virtual humans [7,16]
In a face-to-face intercultural situation, there are several ways that intelligent interventions could promote learning: • • • •
recognizing that an error was committed (or a good action was taken) finding a causal link between the action taken and the observed reaction understanding the reason(s) and culpable underlying cultural differences learning to avoid the same mistake in the future (or sustain good actions)
If a learner commits a cultural error, for example, and simply concludes to avoid the same behavior in the future, it will contribute little in their progression through the DMIS stages since this does not get to underlying causes. Also, the stage a person is in impacts how cultural differences are interpreted. Someone in the denial phase may not even be willing to accept the fact that a cultural error even occurred, for example. A learner in the other two ethnocentric stages (defense and minimization) may be aware an error occurred, but unwilling to take blame or perhaps place the onus on the virtual human to be the one who should adapt. Based on this understanding, the reaction of a virtual human to a cultural error should be appropriate for that learner. Feedback from the simulation itself, such as the oral and nonverbal reactions of virtual humans, is called implicit feedback. To support recognition of cultural errors, there are several strategies that can be used adjust implicit feedback to promote learning. One of the simplest is to accentuate verbal responses of characters to draw more attention to anger or negative feelings, in the case of an error, thus supporting the learner in recognizing a cultural difference that exists. Similarly, implicit positive feedback can be achieved by accentuating positive and laudatory responses to correct user actions. In some cases, it may even be appropriate for the virtual human to deliver an impassioned mini-lecture regarding the cultural issues in question. The choice of words by the virtual human can be designed to refer directly to actions taken by learner to support the pedagogical goal of linking cause and effect in the learner’s mind. In addition, the virtual human might also drop hints regarding the underlying cultural differences. Other forms of implicit feedback, such as body language and gestures, can have a dramatic effect on the communicative power of utterances. Figure 3 shows several virtual humans in different emotional states and displaying a variety of gestures. The timing and emphasis of these gestures can be adjusted to meet pedagogical goals in a way similar to the utterance content. Aside from body
Promoting Metacognition in Immersive Cultural Learning Environments
135
language, other features that might be adjusted in virtual humans are facial expressions, speech rate, intonation, and tone, emotional state, and personality traits. 3.3 Experience Management and Interactive Narrative The techniques described in the previous section all focused on emphasizing specific details of interacting with virtual humans to support the learner in recognizing cultural difference and improving their ability to self-assess in an interpersonal context. Of course, explanations for why certain behaviors are culturally offensive can be very complex. They may involve fundamental differences between worldviews, varying ethical standards, social structure, historical and geographical factors, and so on. A deep understanding of culture includes these advanced notions and may enable a learner to go beyond rote learning by providing the knowledge needed to reconstruct appropriate surface behaviors later. Cultural simulations should provide diverse cultural experiences that go beyond one-onone interactions and manage how events are presented and experienced by the learner. One such technology focusing on experience management is automated story directing (ASD) [15]. The goal is similar to that of a traditional tutoring system: allow users to feel as much freedom as possible, but keep them on certain paths that consist of certain experiences. The “path” in an interactive storytelling system is a storyline consisting of plots, arcs, events, and other narrative elements. Users may “break” a storyline by taking actions in the virtual world, and so ASD systems use a variety of techniques, like reactive planning, to repair storylines and re-plan when new events are deemed desirable. Often, the aim is to maximize engagement. For cultural training, the additional aim is to create situations and conditions that specifically address intercultural development of a learner and their specific areas of need. Learning issues involving metacognitive skills come into play when we consider the learner’s role in the narrative. She or he must be aware that the actions being taken are being observed by the AI agents in the simulation and that choices being made have observable outcomes. Just as behavioral details of interactions can be manipulated to highlight differences, so can story elements. For example, if a learner makes a gender-related error early in a game, the ASD may decide to propagate this knowledge through the social network so other story elements and characters can exploit the weakness. Here the goal is not only to teach gender specific cultural differences, but also to encourage consideration of unintended cultural consequences of earlier actions. This requires the metacognitive ability to continuously self-assess over an extended period, reaching back further than just the most recent action, and the ability to predict (another metacognitive skill). After dealing with negative consequences of actions, it is hoped that a learner will become more likely to anticipate possible unintended outcomes of actions before taking them. 3.4 Guidance and Feedback Inherent risks exist in unguided environments, such as inefficient learning, the formation of misconceptions, and development of incomplete or fragmented knowledge [12]. Experience manipulation and implicit feedback can certainly mitigate these risks to a certain degree, but to adequately address the needs of novice and intermediate learners, explicit feedback from a human tutor, pedagogical agent, disembodied
136
H.C. Lane
coach, or other form of intelligent tutor has the potential to greatly enhance learning. Explicit guidance can come in different forms in an immersive learning environment. A pedagogical agent who plays a role in the underlying simulation is a popular approach. The TLCTS [10] and the mission rehearsal exercise described in [16] both provide pedagogical agents in the form of a knowledgeable companion who can give hints on how to succeed. No matter what the modality, explicit feedback generally provides more direct and understandable guidance than implicit – this is especially important for novices [12]. Immersive learning environments can be overwhelming at times. With respect to cultural learning, explicit hinting and feedback can help learners in several ways: 1. 2. 3. 4. 5.
confirm a learner’s interpretation of observed virtual humans behaviors explain the cultural differences in play during specific interactions explain the “under the hood” reasoning of a virtual human hint about ideal actions to take or warn against certain risks suggest the learner identify possible outcomes and desirable end states
Explicit tutorial feedback removes a level of interpretation for the learner. Rather than guessing or inferring the cognitive and emotional states of virtual humans, a clear statement by a tutor can act as a strong scaffold for learning in immersive cultural environments. There are certainly cognitive aspects to the tactics listed above, but they also address the metacognitive demands of intercultural development. Tactics 1-3 encourage self-assessment by describing the impact of a learner’s actions on a virtual human. Because this is feedback being delivered in a real-time environment, the issues of distraction and cognitive overload need to be considered. Thus, it is ideal to keep “in-game” feedback short and precise, saving the longer explanations for postpractice reflection. Hinting (tactic 4) can be direct (and at the cognitive level), but also can be used to encourage the learner to think about pros and cons of taking different actions – this is especially important in ill-defined domains where assessment is inherently challenging [13]. The content of tactics 1-4 are precisely the what should be a part of the learner’s deliberations before acting in the environment. Such cognitive activities by the learner would constitute attempts at self-explanation. Tactic 5, identifying potential end states, is a purely metacognitive tactic that is geared towards supporting goal formation and identifying “what right looks like.” Encouraging the learner to “think before acting”, to engage in planning and simulate hypothetical actions, and “reflect after acting” are at the core of metacognitive growth and a fundamental requirement for advancement through the DMIS stages. Tutorial sub-dialogues are rarely possible in real-time environments, and so there is time only for very brief periods of reflection. However, once a practice session or exercise is complete, there is time to carefully target metacognitive skill development. Immersive learning environments should therefore provide supporting tools such as video playback. Reflective tutoring is an appropriate supplement to guide the use of these tools and to fill in the gaps from feedback that was delivered during practice. The reflective tutoring system built for the virtual humans [7] walks the student through three questions: (1) What happened? (2) Why did these events occur the way they did? and (3) How can good performance be sustained and poor performance be improved? A promising approach here is to leverage explanation facilities of virtual humans to uncover their thinking via explanation and discover what other actions may
Promoting Metacognition in Immersive Cultural Learning Environments
137
have produced better outcomes. An advanced tactic is to restart the simulation to give the learner a second chance (a “mulligan”). Reflection at this point may enhance selfassessment skills and intercultural growth.
4 Conclusions and Suggestions for Future Research This paper has argued that achieving intercultural competence requires strong metacognitive abilities. Although cross-cultural training programs frequently adopt metacognitive approaches to teaching intercultural competence, the connection is rarely made explicit. The Peace Corps describes this growth as a continuum from unconscious incompetence (not knowing anything and being blissfully unaware of differences) to unconscious competence (full awareness of differences and appropriate behavior is second nature). By describing these stages, the learner put in a position to self-monitor their advancement. This then requires application of other metacognitive skills, such as self-assessment, self-explanation and self-regulation, to progress through the stages. The Developmental Model of Intercultural Sensitivity (DMIS) is an empirically derived and validated model of intercultural development based on notion of how cultural differences are “construed” by a learner [3,11]. To develop intercultural competence, it is necessary to construe cultural difference in progressively more complex ways, such as from different cultural worldview perspectives. Growth here also requires mature metacognitive abilities and it may be possible to promote these skills in modern immersive learning environments through a combination of experience manipulation and explicit guidance. The DMIS suggests cultural difference as the pivot point for intercultural growth, and so careful direction of virtual humans and delivery of feedback that targets self-assessment, predictive, and reflection skills has the potential to speed the growth to intercultural competence. Most of the computer simulations built for cultural education have not undergone rigorous experimental evaluations for learning or for intercultural development. Hammer and Bennett’s Intercultural Development Inventory (IDI) [8] has been used to validate the DMIS and may provide a suitable metric for determining the valueadded, if any, that comes from augmenting cultural training programs with immersive learning environments – especially if the IDI can be administered repeatedly and rates of change can be tracked. Other more general questions about feedback are suggested for further study. For example, • • • •
Does implicit pedagogical feedback break immersion? If so, what is the cost (if any) to breaking immersion with respect to learning? What is the interplay between implicit and explicit feedback? What situations merit the use of explicit feedback?
The use of implicit feedback and experience manipulation is perhaps one of the most important open questions to address. The instructor interface to the ATL serious game [14] provides the ability to throw “curve balls” to teams during their missions, such as helicopter fly-overs. These are intended to support the development of adaptive thinking skills under stress. This is related to research in the ITS community on finding the appropriate level of challenge for a learner. These connections need further exploration, as does the reasoning behind expert instructors’ decisions to throw curve balls:
138
H.C. Lane
What are the triggers? How do instructors decide which curve ball to throw? The answers may not always involve metacognitive skills, but as this paper has attempted to argue, manipulations of this sort in an intercultural context may be ideal to highlight cultural differences to give learners practice in dealing with them and in developing intercultural competence.
Acknowledgments The project described here has been sponsored by the U.S. Army Research, Development, and Engineering Command (RDECOM). Statements and opinions expressed do not necessarily reflect the position or the policy of the United States Government, and no official endorsement should be inferred.
References 1. Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward meta-cognitive tutoring: A model of help-seeking with a cognitive tutor. Int. J. of Art. Int. in Ed. 16, 101–130 (2006) 2. Bennett, J.M.: Intercultural communication training in cultural self-awareness for study abroad. PhD thesis, University of Minnesota (October 1985) 3. Bennett, M.J.: Towards ethnorelativism: A developmental model of intercultural sensitivity. In: Paige, R.M. (ed.) Education for the Intercultural Experience, pp. 21–71. Intercultural Press, Yarmouth (1993) 4. Bransford, J.D., Brown, A.L., Cocking, R.R. (eds.): How People Learn: Brain, Mind, Experience, and School. National Academy Press, Washington (2000) 5. Brown, A.L.: Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In: Weinert, F.E., Kluwe, R.H. (eds.) Metacognition, motivation, and understanding, pp. 65–116. Lawrence Erlbaum Associates, Hillsdale, New Jersey (1987) 6. Chi, M.T.H., de Leeuw, N., Chiu, M.H., LaVancher, C.: Eliciting self-explanations improves understanding. Cognitive Science 18(3), 439–477 (1994) 7. Core, M.G., Traum, D., Lane, H.C., Swartout, W., Gratch, J., van Lent, M., Marsella, S.: Teaching negotiation skills through practice and reflection with virtual humans. Simulation 82(11), 685–701 (2006) 8. Hammer, M.R., Bennett, M.J., Wiseman, R.: Measuring intercultural sensitivity: The intercultural development inventory.I. J. of Intercultural Relations 27, 421–443 (2003) 9. Hoopes, D.S.: Intercultural communication concepts and the psychology of intercultural experience. In: Pusch, M.D. (ed.) Multicultural education: A cross cultural training approach, pp. 10–38. Intercultural Network, Inc., La Grange Park (1979) 10. Johnson, W.L., Vilhjalmsson, H., Marsella, S.: Serious games for language learning: How much game, how much AI? In: Looi, C.-K., McCalla, G., Bredeweg, B., Breuker, J. (eds.) Proc. of 12th Int. Conf. on Artificial Intelligence, pp. 306–313. IOS Press, Amsterdam (2005) 11. Kelly, G.A.: Theory of personality: The psychology of personal constructs. Norton, New York (1963) 12. Kirschner, P.A., Sweller, J., Clark, R.E.: Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist 41(2), 75–86 (2006)
Promoting Metacognition in Immersive Cultural Learning Environments
139
13. Ogan, A., Aleven, V., Jones, C.: Culture in the classroom: Challenges for assessment in illdefined domains. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053. Springer, Heidelberg (2006) 14. Raybourn, E.M.: Applying simulation experience design methods to creating serious game-based adaptive training systems. Interacting with Computers 19, 206–214 (2007) 15. Riedl, M.O., Stern, A., Dini, D., Alderman, J.: Dynamic experience management in virtual worlds for entertainment, education, and training. Int. Tran. on Systems Science and Applications, Sp. Iss. on Agent Based Systems for Human Learning 4(2), 23–42 (2008) 16. Swartout, W., Gratch, J., Hill, R., Hovy, E., Marsella, S., Rickel, J.: Toward virtual humans. AI Magazine 27(2), 96–108 (2006)
The Application of the Flexilevel Approach for the Assessment of Computer Science Undergraduates Mariana Lilley and Andrew Pyper University of Hertfordshire, School of Computer Science, College Lane, Hertfordshire AL10 9AB, United Kingdom {m.lilley,a.r.pyper}@herts.ac.uk
Abstract. This paper reports on the use of the flexilevel approach for the formative assessment of Computer Science undergraduates. A computerized version of the flexilevel was designed and developed, and its scores were compared with those of a traditional computer-based test. The results showed that the flexilevel and traditional scores were highly and significantly correlated (p<0.01). Findings from this study suggest that the flexilevel approach is a viable adaptive testing strategy, and may be a good candidate for smaller applications where IRT-based CATs may be too demanding in terms of resources. Keywords: e-assessment, flexilevel, adaptive testing strategies.
The Application of the Flexilevel Approach for the Assessment of Computer Science
141
Other studies on the use of adaptive testing in a higher education environment include research previously conducted by one of the authors [7]. In this work, it was shown that the application of an IRT-based approach to adaptive testing supported proficiency level estimates comparable to those obtained using traditional (i.e. static) tests. The IRT-based approach to adaptive testing was also shown to be effective at tailoring the level of difficulty of a test to the proficiency level of individual students. By tailoring the level of difficulty of the question or task to individual proficiency levels, students were challenged by test items at an appropriate level, rather than demotivated by items that were above or below their proficiency level. Despite these positive findings, procedural factors impeded a greater uptake of the adaptive approach as proposed in the research [7]. The implementation of IRT-based adaptive testing applications is a complex task. Most importantly, the IRT model requires a large and calibrated database of questions. The calibration process usually requires a large item pool and specialist calibration programs, such as Xcalibre [1, 5]. This paper reports on a pilot study that investigated the implementation of computerized adaptive testing based on the flexilevel approach [8, 11] as an alternative to IRT-based algorithms. An overview of the flexilevel approach is presented in the next section of this paper.
2 The Flexilevel Approach Like IRT, flexilevel algorithms attempt to match the level of difficulty of questions to the proficiency level of individual students. The Flexilevel approach, however, is based on a fixed branching strategy that is less complex to implement than IRT-based algorithms. Database calibration is also simpler; indeed Lord [8, p. 117] suggests that “any rough approximation” of the difficulty, of the questions will be adequate. The difficulty of a question is determined by the following formula (adapted from Ward [12]):
⎛ n × 100 ⎞ ⎟⎟ D = 1 − ⎜⎜ p n r ⎝ ⎠
(1)
In Equation 1, np is the number of students who answered the question correctly, and nr is the total number of students who answered the question. In addition to simpler calibration, smaller question databases are required. The flexilevel approach requires a database of 2n-1 question (where n is the number of questions to be administered during the test). A flexilevel test typically starts with a question of medium difficulty. If the student answers the question correctly, a more difficult question is presented. Conversely, if the question is answered incorrectly, an easier question follows. The flexilevel approach was initially devised as a self-scoring paper-and-pencil test [8]. As an example, consider a paper-based test with 19 questions; the difficulty of the questions ranging from 0.05 (easiest) to 0.95 (hardest).
142
M. Lilley and A. Pyper [Q10, difficulty = 0.50] Red 1 [Q9, difficulty = 0.45] Blue 1 [Q11, difficulty = 0.55] Red 2 [Q8, difficulty = 0.40] Blue 2 [Q12, difficulty = 0.60] Red 3 [Q7, difficulty = 0.35] Blue 3 [Q13, difficulty = 0.65] Red 4 [Q6, difficulty = 0.30] Blue 4 [Q14, difficulty = 0.70] Red 5 [Q5, difficulty = 0.25] Blue 5 [Q15, difficulty = 0.75] Red 6 [Q4, difficulty = 0.20] Blue 6 [Q16, difficulty = 0.80] Red 7 [Q3, difficulty = 0.15] Blue 7 [Q17, difficulty = 0.85] Red 8 [Q2, difficulty = 0.10] Blue 8 [Q18, difficulty = 0.90] Red 9 [Q1, difficulty = 0.05] Blue 9 [Q19, difficulty = 0.95]
Fig. 1. Adapted from Lord [8]. Layout of a paper-based flexilevel test, in which Q1 is the easiest question and Q19 is the hardest question
In the example shown in Fig. 1, the student will start the test by answering Q10 (i.e. question of medium difficulty). The answer sheet will inform the student whether each response was correct or incorrect; for example, a red spot appears where the student has selected an incorrect answer and a blue spot appears where the student’s response was correct. Each time a correct answer is given, the question to be answered next is the lowest numbered “blue” question not previously answered. In the event of an incorrect response, the next question to be answered is the lowest numbered “red” question not previously answered. A student who answers all questions correctly will answer the following sequence of questions: Q10, Q11, Q12, Q13, Q14, Q15, Q16, Q17, Q18, and Q19. A student who answers all questions incorrectly will answer the following sequence of questions: Q10, Q9, Q8, Q7, Q6, Q5, Q4, Q3, Q2, and Q1. A student who answers the first question incorrectly, and all the following questions correctly will answer the following sequence of questions: Q10, Q9, Q11, Q12, Q13, Q14, Q15, Q16, Q17, and Q18. Following the directions for a paper-based flexilevel test can be an onerous and disengaging task. This issue can be avoided by developing a software application to select, administer and score the questions, and this is the focus of the study reported here.
3 Methodology The experiment was designed to model a real formative assessment situation as far as possible, given the constraints of the experimental design. So, whilst participants were free to leave without consequence and they were not the subject of the test, they were encouraged to engage with the test as if it were a real formative assessment. 3.1 Participants Twenty-four final year Computer Science undergraduates participated in the experiment. They had previously studied the topic that was the subject of the test, but were briefed that the experiment was a test of the software application and not of them.
The Application of the Flexilevel Approach for the Assessment of Computer Science
143
3.2 Software The software prototype was developed using VB.NET with an Access database to store the questions and responses. There were 29 questions; these were separated into two pools, one to populate the flexilevel test (19 questions), and the other to populate the statically selected test (10 questions). The questions had previously been used in standard testing and calibrated based on the calibration formula described previously. The order of presentation of the two tests was randomised. The two different sets of questions had different light background colours on the question screens. This was done in order to enable the experimenters and participants to refer to the different tests without naming them. The colours used were selected because they maintained a clear contrast between the text of the questions and the background of the screen. Fig. 1 shows the layout of the question screen.
Fig. 2. Screenshot of the flexilevel software application
Finally, the software allows for the timing of a test, and displays time elapsed and number of questions answered throughout the test. 3.3 Questionnaire The questionnaire document consisted of a briefing section setting out information about the flexilevel approach, test and questionnaire and 15 statements that participants could rate using a 5 point Likert scale from 1: Strongly Disagree to 5: Strongly Agree. The statements are shown in Tables 3 (ease of use) and 4 (perceived usefulness). The statements were designed to elicit participants’ views on the ease of use of the software application and the perceived usefulness of the software application.
144
M. Lilley and A. Pyper
A further 3 questions that asked for free text responses were included, and are shown in Table 1 below. Table 1. Open questions included in the questionnaire
16
Statement What problems, if any, can you see to the uptake of this adaptive testing approach?
17
Is there a question that you would like to have been asked? If so, what is it and how you would answer it?
18
Can you see any benefits of this adaptive testing approach?
3.4 Experimental Procedure The participants were briefed that they were about to engage in an assessment using a new form of software assessment. They were told that the test was of the software and not of them. However, in order to encourage them to answer the questions to the best of their ability, participants were informed that the participant with the highest overall score would be given £20.00. They were also told that they would be asked to fill in a questionnaire. All participants were offered £10.00 as a thank you for participating, but it was made clear that they could leave at any time and did not have to complete the test or the questionnaire in order to receive the money. The participants took the test in a computer lab in test conditions. They started the test in their own time and had 25 minutes to complete the whole test; this was monitored by the software application. This enabled the collection of the performance of participants in the two tests and the reporting of their results at the end of the test. Once the participants had completed the tests they were asked to fill in the questionnaires that had been placed by their computers. This enabled the collection of data associated with the views of participants. 3.5 Hypothesis The hypothesis of the experiment was that there would be a significant correlation between students’ results on the statically generated test and the flexilevel generated test.
4 Results A summary of students’ performance is presented in Table 2. Table 2 shows that the range of scores achieved by participants was relatively large for both test conditions and that there is little apparent difference between the conditions.
The Application of the Flexilevel Approach for the Assessment of Computer Science
145
Table 2. Summary of student scores for each section of the test (N=24) Test Flexilevel Static
Minimum
Maximum
1.0 2.0
9.0 8.5
Mean 4.62 4.89
Std. Deviation 2.10 1.91
A Pearson product-moment correlation coefficient was computed to assess the relationship between the flexilevel and static scores shown in Table 2, r=0.764, n=24, p<0.01, Sig. (2-tailed)=.000. A paired-samples t-test showed no significant difference between the flexilevel and the static scores, t(23) = -.954, p=.350. The scatterplot shown in Fig. 2 illustrates the results.
Fig. 2. Scatterplot diagram showing the scores achieved by participants in the two test conditions
Student responses to the questionnaires were also analysed. Table 3 below shows that the majority of participants agreed with statements concerning the ease of use of the software application. As can be seen from Table 4, participants were also positive when responding to statements about the perceived usefulness of the software application. This was particularly the case in terms of using the flexilevel for formative assessment (statement 4), areas that the participants might need to work on (statement 6), or not (statement 11) and that if flexilevel tests were made widely available that they would use them (statement 13).
146
M. Lilley and A. Pyper Table 3. Ease of use. Mode and median for the responses. Statement Median Mode 1. Learning to use the Flexilevel software application would be 4 4 easy for me. 2. I would find it easy to remember how to perform tasks (e.g. 4 4 how to answer a question) using the Flexilevel software application. 3. I would find the Flexilevel software application easy to use. 4 4 Table 4. Perceived usefulness. Mode and median for the responses. Statement Median 4. I would find the Flexilevel approach useful for practice 4 tests. 5. I would find the Flexilevel approach useful in summative 3.5 tests (i.e. the test score counts towards my final grade). 6. The adaptivity supported by the Flexilevel approach 4 would help me to identify the areas in which I need to work harder more quickly. 7 The adaptivity supported by the Flexilevel approach would 4 enhance my overall assessment experience. 8 For practice tests, I would prefer using the Flexilevel 4 approach to other forms of objective testing. 9. For summative tests, I would prefer using the Flexilevel 3 approach to other forms of objective testing. 10. The system used to score a Flexilevel test makes sense to 4 me. 11. I would find the score provided by the Flexilevel 4 approach useful at identifying how much I have learned. 12. I would find it useful if the level of difficulty of a test is 4 tailored to my level of understanding. 13. Assuming the Flexilevel software was available to me for 5 practice tests, I predict that I would use it on a regular basis. 14. In practice tests, test questions that are too easy are less 4 engaging than those questions that are tailored to my level of understanding. 15. In practice tests, test questions that are too difficult are 3 less engaging than those questions that are tailored to my level of understanding.
Mode 5 3 5
4 4 3 4 4 4 4 4
3
Participants were less positive in their responses to the use of the flexilevel approach in summative assessments (statement 5) and tended not to agree that more difficult questions were less engaging (statement 15). The open questions were included to enhance the richness of the data collected, but were too few to be subjected to content analysis. As such, they will be used to inform the discussion of the results that follows, rather than being reported in this section.
The Application of the Flexilevel Approach for the Assessment of Computer Science
147
5 Discussion The results of this pilot study are very encouraging, there is a high and significant correlation between participants’ performance on the two different types of test and participants were positive about their use of the software. Additionally, participants were positive about the use of the flexilevel approach in formative assessment. Moreover, there was a reasonably wide distribution of scores. So, it seems that performance of participants in the two tests is consistent across a range of attainment. Nonetheless, this is a pilot study, and as such there are aspects of this study that future experiments will need to replicate. The participants are all final year undergraduate students. It may be expected that such a sample would have no problems with using the software application. The measures of ease of use are important in controlling for potential extraneous variables, and also to identify any usability issues in these early stages of development. In this sense a sample of Computer Science undergraduates was a good choice for the pilot study. Expert analysis of the interface indicated no serious usability issues that might be expected to trip up even users with a high degree of technical literacy. The results of the study bear this out. Clearly the downside to using Computer Science students as participants is that this sample of participants may not be representative of a wider student population. Whilst the participants in this study reported no problems in using the software application, it seems likely that extrapolating this to a wider population would be unwise. Clearly an important part of future experiments would be to involve participants from a wider population of students. One reservation raised about the software application was that “You can't go back and correct a previous answer”. This is a necessary feature of the system, however, because if participants were able to go back and review the questions, they would soon be able to enhance their marks simply by going back and attempting the questions until they found the correct response. In terms of the perceived usefulness of the application, there seems to be a difference in the way participants perceived the flexilevel approach as a tool for formative and summative assessment. Participants were positive about the formative use of the flexilevel approach. One participant commented that “providing multilevel questions after one another is good”. As noted previously, participants believed it could support them in their academic progress – “...would really help to outline where problems in understanding lie and help students to address those areas.” Also “...good approach to learning giving better students harder questions.” This attitude did not extend to the use of the flexilevel approach in summative contexts: “all students may not be tested equally” and “I think that there would be a smaller gap in marks between good and bad students than in normal tests which would not be good for assessment”. Participants did suggest that the flexilevel approach would enhance their overall assessment experience (statements 7 and 12), but it seems that this enhancement would mostly be realised in the formative assessment they engage in. An informal observation has been that students were uncomfortable using a system in which the scoring system was not clear to them; for example, maximum likelihood estimates, used in IRT-based adaptive testing may not be obvious to all. In this study, participants indicated that they did indeed understand how the scoring system worked
148
M. Lilley and A. Pyper
(statement 10). This could be taken to be support for the use of a simpler adaptive system than those that had been previously employed. However, this bears further scrutiny in future work given that participants felt that high-performing students may be disadvantaged by this scoring in summative assessments.
6 Conclusion This pilot study was concerned with the efficacy of the flexilevel approach to adaptive testing when compared to standard computer based testing. The results show a highly significant correlation between scores achieved by participants on the two tests. This provides a basis upon which to replicate and extend this research to include larger groups of students and those in studying in different academic disciplines. This will be the subject of future work.
References 1. Assessment Systems Corporation, XCALIBRE Marginal Maximum-Likelihood Estimation, http://assess.com/xcart/product.php?productid=270&cat=0& page=1 2. Challis, D.: Committing to quality learning through adaptive online assessment. Assessment & Evaluation in Higher Education 30(5), 519–527 (2005) 3. Conejo, R., Guzmán, E., Millán, E., Trella, M., Pérez-de-la-Cruz, J.L., Ríos, A.: SIETTE: A Web–Based Tool for Adaptive Testing. International Journal of Artificial Intelligence in Education 14, 1–33 (2004) 4. Conejo, R., Millán, E., Pérez-de-la-Cruz, J.-L., Trella, M.: An empirical approach to online learning in SIETTE. In: Gauthier, G., VanLehn, K., Frasson, C. (eds.) ITS 2000. LNCS, vol. 1839, pp. 604–614. Springer, Heidelberg (2000) 5. Gierl, M.J., Ackerman, T.: Software Review: XCALIBRE — Marginal MaximumLikelihood Estimation Program, Windows Version 1.10. Applied Psychological Measurement 20(3), 303–307 (1996) 6. Joy, M., Muzykantskii, B., Evans, M.: An Infrastructure for Web-Based ComputerAssisted Learning. ACM Journal of Educational Resources 2(4), 1–19 (2002) 7. Lilley, M.: The Development and Application of Computer-Adaptive Testing in a Higher Education Environment, Unpublished PhD thesis, School of Computer Science, University of Hertfordshire, Hertfordshire (2007) 8. Lord, F.M.: Applications of Item Response Theory to Practical Testing. Lawrence Erlbaum Associates Inc., Mahwah (1980) 9. Ríos, A., Conejo, R., Trella, M., Millán, E., Pérez-de-la-Cruz, J.L.: Aprendizaje automático de las curvas características de las preguntas en un sistema de generación automática de tests. In: Actas de la Conferencia Española para la Inteligencia ArtificialCAEPIA 1999 (1999) 10. Ríos, A., Millán, E., Trella, M., Pérez-de-la-Cruz, J.L., Conejo, R.: Internet Based Evaluation System. In Open Learning Environments: New Computational Technologies to Support Learning, Exploration and Collaboration. In: Proceedings of the 9th World Conference of Artificial Intelligence and Education AIED 1999, pp. 387–394. IOS Press, Amsterdam (1999) 11. Wainer, H.: Computerized Adaptive Testing: A Primer. Lawrence Erlbaum Associates Inc., Mahwah (2000) 12. Ward, C.: Preparing and Using Objective Questions. Nelson Thornes Ltd., Cheltenham (1980)
Development of Ubiquitous On-Demand Study Support Environment for Nursing Students Yukie Majima1, Yumiko Nakamura2, Yasuko Maekawa2, and Yoichiro So3 1
Department of Liberal Arts and Science, Osaka Prefecture University 1-1 Gakuencho, Naka, Sakai, Osaka 599-8531, Japan [email protected] 2 School of Nursing, Osaka Prefecture University 3-7-30 Habikino, Habikino, Osaka 583-8555, Japan [email protected], [email protected] 3 Production Systems Research Lab. KOBELCO 1-5-5 Takatsukadai, Nishi, Kobe, Hyogo 651-2271, Japan [email protected]
Abstract. We were selected to work with “Development of e-Learning Program to help enhance human resource ability based on needs,” which is a government supported program of fiscal year 2005 to address modern education needs (Modern GP). The program has therefore been underway since 2005. The objectives of the program are to use e-Learning to further improve education and teaching practices qualitatively for nursing education, to supply high-level nursing practice capabilities, and provide a new environment in which students can study independently in an efficient manner. The main targets to achieve the program goals are: i) to produce e-Learning training materials that include examples of nursing to use for the study of nursing practice and support students’ acquisition of problem-solving abilities during nursing operations; and ii) to construct an environment to support ubiquitous on-demand studies in which students can study for themselves easily, at any time, and anywhere on campus or on the actual practice site using this e-Learning methodology. This paper presents a report of an actual lecture class conducted in using such training materials. Keywords: Education of nursing, Nursing practice ability, E-learning, Ubiquitous on-demand study.
and instruction of nursing more qualitatively, and to provide students with a new environment in which they can study efficiently. Among all educational activities, the most effective way for students to acquire actual nursing capabilities is to have the learner join on-site practice training, which is arequired part of the nursing education curriculum. Through on-site practice, students can learn comprehensively by communicating directly to patients or those who require nursing care. However, such practicing facilities are spread out in various locations. For that reason, the situation available on site is not necessarily as good a study environment as that available on campus. The campus might provide facilities such as libraries, Internet capability, and IT equipment. By making e-Learning available at every on-site practice location, we believe that we can not only improve the study environment situation, but also help to raise the quality level of the study itself. Under these circumstances, the School of Nursing at the Osaka Prefecture University was selected to work with “Development of e-Learning Program to help enhance human resource ability based on needs,” which is a government-supported program of fiscal year 2005 to address modern educational needs (Modern GP). The actual title is “e-Learning to help acquire nursing practice ability –Construction of a ubiquitous on-demand study supporting environment–.” The program began activities last year as the “CanGo” project. “CanGo” stands for “Communication”, “Art”, “Nursing”, “Good practice”, “Osaka Prefecture University”. This paper reports the actual classes taught in 2006-2007 using materials that were developed specifically for this project based on project activities undertaken in 2005, and of the results obtained from on-site training support we gave to students.
2 CanGo Project Summary The CanGo project objective is to fill the gaps that are readily noticeable from one study method to another, to improve the efficiency and usefulness of every study activity on-campus or off-campus, and to improve students’ attitudes toward study more aggressively. This objective must be achieved so that students can acquire the ability to solve study-related problems that might often occur in lectures, or during practice sessions made on-campus or on-site, all of which are conducted in the regular curriculum of nursing education. The backbones of our activities are: i) to produce e-Learning materials for the study of nursing practice and examples for use by students to acquire problem-solving ability; and ii) to construct a ubiquitous on-demand study-supporting environment in which students can study independently through e-Learning at any time, anywhere, and without any difficulty whatsoever. Through these four-year studying opportunities on and off campus, we are hoping to help students acquire capabilities to solve all kinds of nursing problems efficiently and aggressively so that, eventually, we can produce in our society good human resources with full proficiency in advanced nursing practice capabilities. Specifically, the following three unique environments are available to support students’ study.
Development of Ubiquitous On-Demand Study Support Environment
151
Utilizing USB memory that stores the recorded study history, anyone can easily download examples of nursing from the nursing training materials server machine. Each student is allowed to compile a digital nursing dictionary by downloading the necessary sub-training materials before the on-site practice starts. We believe that any student can study independently while compiling such a “personal digital nursing dictionary.” During that study process, a learner would be able not only to review and summarize the contents of past study, but also to proceed with meta-recognition that would point to further study or related activities, and what knowledge and nursing technology must be acquired before the on-site practice starts. Finally, to support selfstudy, a bi-directional study support environment would be prepared in which a
Fig. 1. Overall structure of CanGo project Table 1. CanGo project targets and its accomplishments Targets 1) Develop 100 examples of nursing by re-structuring the existing nursing examples documents, and construct a training-material database 2) Construct a ubiquitous on-demand study support environment that is openly available to students Accomplishments 1) Development of 107 examples in four specialized nursing areas, and a total of 1901 references for the digital nursing dictionary 2) Supplying equipment for study support environment 272 portable multimedia terminals 15 laptop PCs to support lecture classes 1 wireless attachment for a projector
152
Y. Majima et al.
student can receive guidance from the instructor through e-mail that might be available using a cell phone or other media equipment. Figure 1 depicts the overall structure of the CanGo project. Table 1 shows its targets.
3 e-Learning Materials for Nursing Examples Simulation We produced nursing training materials with examples for each specialized nursing area based on prototype training frames for e-Learning of nursing, developed by Majima et. al. and Seta et. Al.[1,2,4,5] to improve nursing students’ problem-solving abilities.
Fig. 2. Training frames on nursing examples
Fig. 3. Digital nursing dictionary
Development of Ubiquitous On-Demand Study Support Environment
153
Training frames (Figure 2) consist of two study modes. In study mode 1, a series of nursing processes can be studied through “Understanding patients’ examples”, “Problem studies on nursing”, and “Evaluation of analyzed results”. Study mode 2 is available for each example of a patient, in which the student can self-study, when necessary, through the contents of basic and required knowledge of specialized nursing practices, nursing techniques, and national-examination-related materials. Training materials were produced using multimedia techniques incorporating sound effects, animation, and illustrations. Contents that are organizing the training materials can be reorganized and reused for other digital sub-training materials, allowing the creation or addition of any new training materials without difficulty. Moreover, after downloading, they can be used to create a new digital nursing dictionary (Figure 3). In the other mobile method, Maag[3] introduced an emerging technology using by podcasting in nursing education.
4 Practice and Example in Nursing Education From 2006 to 2007, we hosted a series of lectures using training materials developed in from 2005 to 2007, and used portable multimedia terminals for on-site practice. Following are the ideas and opinions given to us after that on how to improve practices, as well as other useful advice. 4.1 Lectures Using Training Materials with Nursing Examples Instructors who joined in the creation of training materials actually used them in their classes. We asked them to report about how they used the training materials in their classes in addition to the responses they received from their students. To have them conduct such training in their classes, the study-support environment development team of the project organization supplied the training materials, computers and assemblies, and other materials to instructors who requested them. The execution evaluation team helped create questionnaires for students and the reporting formats to be used by the instructors. The actual lectures were intended mainly for third-year nursing department students (about 120–140) studying for work in five different nursing fields. Following are some examples of how training materials were used: i) Starting with the introduction of the patient starring in the training materials of examples on nursing (giving self-introduction narration by the patient), the patient subsequently presented scenes in which nursing problems are described; then the students were separated into work groups and each was expected to make a nursing care plan, and to present it before others, which were finally evaluated by the instructor with appropriate comments provided. ii) Introduction of some nursing examples and scenes of nursing care problems were posed. iii) Providing the students with some nursing care problems beforehand to use them in later work to be done by every group. Therefore, different approaches seem to have been taken by each instructor. Some opinions from the students are as follows: “It was easy to grasp the images of the examples,” “It was easy to read the patient’s emotions shown in the examples,” “It was realistic, so it was easy to understand,” “It was realistic and practical,” “It
154
Y. Majima et al.
was good from a visual perspective with the image pictures provided,” “It was easy to know the necessary information,” “It was good to hear the information given by voice,” and “I was able to understand the priority of the nursing care problems.” Additionally some students pointed out not only about how to collect information, but also about the methodology of how nursing-care assistants should recognize situations to move forward to create opportunities to educate patients. 4.2 On-Site Practice Support by Portable Multimedia Terminals We prepared portable multimedia terminals (PSP, Play Station Portable; Sony Corp.) loaded with training material contents prepared for the associated nursing practice areas, and lent them each to students who agreed to monitor them. Subsequently, we requested that they use them during the entire practice period. We requested them to give us opinions regarding the operability of the training materials and other information when they had to return the terminal. Following are some of the comments. Portable Multimedia Terminal 1. It was good to be able to find and view the necessary document without taking much time. 2. Because it is basically a game machine, its operability is good. 3. It was easy to use the training materials in this way. 4. The images in the training materials are useful, as if we were playing a game. 5. It is easy to carry because it is small and light. 6. The screen was easier to see than I thought. 7. The battery power duration time is too short. 8. It would be nice if there were a search function available everywhere in the training materials. Training Materials’ Contents 1. They were helpful in the practice class. It would have been nice if they had been available before. 2. What we wanted to see out of the training materials was restricted for our practice class. 3. It was very convenient to be able to see technically related pictures close at hand. 4. It was helpful to understand things using images. 5. We can see it anytime anywhere. Therefore, it is helpful when we study at home after the nursing technique practice is over.
5 Discussions After having had the training materials and the contents we produced used in the actual classes, we found that the students’ evaluations were good partially because it is something new to them. Nevertheless, their comments and opinions accurately reflected their experience, and helped us to determine the issues clearly and work continuously to correct and improve them in future versions of the system. Some
Development of Ubiquitous On-Demand Study Support Environment
155
instructors were not very happy with manipulation of information equipment, which underscores the importance of support them during use in actual classes. Through the process of having the training materials used in actual classes, we can continue to search for the most appropriate training materials. In so doing, we can review previous teaching methods and adapt and reconstruct them, if necessary. Additionally, using such training materials in actual classes, we can review the class proceedings and teaching methods the classes with the materials combined. Above all, the fact that the instructors felt that teaching a class was fun should create a positive impact on future educational activities resulting in leading to develop of our faculty.
6 Conclusions This report described our introduction of e-Learning in the School of Nursing at the Osaka Prefecture University (as the CanGo project), and reported comments and opinions given when we had the nursing practice example materials and sub-material contents we produced from 2005 to 2007 used for the actual classes from the first half of 2006 to 2007. In addition, we reported those comments and opinions given from the experiences we had when supporting the actual on-site practices. Future studies will further introduce e-Learning techniques based on the plan, along with regular and constructive review. We would be happy if the knowledge and experience we received were referenced and used anywhere else.
Acknowledgments This project was conducted through the aid of the Modern Education Needs Supporting Program of the year 2005 offered from the Ministry of Education, Culture, Sports, Science and Technology.
References 1. Kazuhisa, S., Yukie, M., Yoichiro, S.: Nursing task ontology based learning system redesign for enabling adaptive on-demand learning environment. In: Proceedings of the 3rd WSEAS/IASME International Conference on Engineering Education, Greece, pp. 114–119 (2006) 2. Kazuhisa, S., Yukie, M., Yoichiro, S.: Towards Building Adaptive On-Demand Learning Environment Based on Nursing Ontology. WSEAS Transaction on Advantages in Engineering Education, 563–570 (2006) 3. Margaret, M.: Podcasting: An emerging technology in nursing education. Studies in Health Technology and Informatics 122, 835–836 (2006) 4. Yukie, M., Yoichiro, S.: Development of E-learning for Problem solving Approach of Nursing Students. Studies in Health Technology and Informatics 122, 919 (2006) 5. Yukie, M., Yoichiro, S., Kazuhisa, S.: Framework for Problem-Solving Based Learning in Nursing Domain – An Experimental Study. Learning by Effective Utilization of Technologies: Facilitating Intercultural Understanding, pp. 625–628 (2006)
The Effects of Prior Knowledge on the Use of Adaptive Hypermedia Learning Systems Freddy Mampadi, Sherry Chen, and Gheorghita Ghinea School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK {Fred.Mampadi,Sherry.Chen,George.Ghinea}@brunel.ac.uk
Abstract. Prior knowledge and cognitive styles are considered important determinants in adaptive hypermedia learning systems (AHLSs) as they influence how learners select information to put into memory. However, there is a need to investigate how they influence learner performance and perceptions prior to comparing them and establishing if they can be used together to maximise learning in AHLSs. To this end, this study investigated the effects of prior knowledge on the use of AHLSs to set the foundation for the comparison. 60 students participated in this study. The results showed that, in general, adapting to prior knowledge improves learner performance and perceptions, especially for users with low prior knowledge. However, the results also indicated that the relative improvement in learner performance is significantly higher than that of perceptions when using the AHLS. The implications of the design of AHLSs are discussed by the paper. Keywords: Learning performance, Perceptions, Novice, Experts, Evaluation, Cognitive styles, Learning styles.
knowledge improves learner performance and/or perceptions, as per the previous empirical studies, prior to making comparison with systems that adapt to cognitive styles. The objective of this study is to, therefore, develop the adaptive hypermedia learning system that adapts to individuals’ prior knowledge and then compare it with an ordinary hypermedia system as an evaluation process to determine whether the adaptive version has a different effect from the ordinary version.
2 Related Research There have been a number of adaptive hypermedia systems that have been developed that takes into consideration, the user’s level of prior knowledge. These systems are normally grouped under Adaptive Educational Hypermedia Systems (AEHS), which are also known as Adaptive Hypermedia Learning Systems (AHLS). Examples of such systems are ELM-ART II [22], and InterBook [2]. The evaluation of the developed AHLSs mostly analysed one of the adaptive techniques without taking the totality of the system. That is, adaptation in an adaptive hypermedia system is driven by several adaptive techniques that work in conjunction to improve the learning of an individual. However, previous research (e.g. [2]) investigated adaptive navigation support alone while Boyle and Encarnacion [1] investigated adaptive presentation. Our research, however, combines the previously research adaptive techniques, also basing on the proposed frameworks (e.g. [7]) to develop an AHLS that could be tested against an ordinary hypermedia system that does not offer adaptivity. Also, according to the review of the literature, previous research focused on the improvement of learning performance as a determinant for the importance of prior knowledge in driving adaptation in AHLSs. A little research targeted the perceptions of use for such systems. Higher test scores were, in effect, transcribed as a representation for positive perceptions as several studies have indicated that students performed better in learning environments of which they had more positive perceptions [6,13]. To this end, there is a need to establish whether learning performance or perceptions (or both) are influenced by adapting to prior knowledge in AHLSs. We conducted a controlled experiment investigating the effects of prior knowledge on the use of AHLSs. Our experiment was based on AHLS developed with previous Table 1. Summary of preferences for low prior knowledge and high prior knowledge users (From [7])
• • • •
Low Prior Knowledge Perform better in hierarchical structure Need advance organizers and advisement Prefer guided navigation Prefer concept maps
• •
High Prior Knowledge Perform better in network structure Prefer free navigation
158
F. Mampadi, S. Chen, and G. Ghinea
research findings that combined the conceptual framework for designing AHLSs [7] and preferences of novices and experts in hypermedia environments (Table 1 and 2). The experiment was aimed at answering two research questions: (1) whether adapting a hypermedia learning system to an individual’s level of prior knowledge has an effect on learning performance; (2) whether adapting a hypermedia learning system to an individual’s level of prior knowledge has an effect on perceptions.
3 Experiment 3.1 Method Sixty participants took part in this experiment. In order to determine whether or not the AHLS was better than the ordinary hypermedia system, a between-subjects design was used. In other words, this meant that each student used either of the systems once but not both. The experiment was controlled. The same content was used for both systems without incurring the practice and fatigue effects in the experiment. Furthermore, each participant went through the same procedures in order to minimise bias. 3.2 Procedure Participants started with a pre-test which consisted of 19 questions before perusing the learning content (either Ordinary or AHLS). The AHLS used adaptive techniques in Table 2 to cater for novices and experts. When the participants felt they had completed the material, they followed a link to post-test before finishing the experiment by answering an exit questionnaire. The questionnaire gathered perceptions and attitudes of use. Table 2. Differences between novices’ and experts’ interfaces Adaptive Hypermedia Link hiding Adaptive layout Additional support Annotated Links
Rich links Alphabetic Index No advisements No annotations
4 Discussion of Results The sample mean scores for novices and experts using the ordinary hypermedia system were determined and treated as the expected means for the respective populations. The means for novices and experts using the adaptive hypermedia system were also calculated and then compared to the expected means of the population. In other words, the calculated means for the ordinary hypermedia system were compared to the calculated means for adaptive hypermedia system. For learning performance, an independent one-sample t-test was used. The onesample t-test compares the mean score of a sample to a known value. It measures
The Effects of Prior Knowledge on the Use of AHLS
159
whether a sample value significantly differs from a hypothesized value [18]. In our case the hypothesized value is the mean calculated scores resulting from the use of the ordinary hypermedia system. The sample mean is calculated from results drawn using the adaptive hypermedia system. The resultant two means can, therefore, be compared. The means of gain scores and post-test scores were used to determine the differences in learning performances. To make a comparison between the perceptions of those students that used the ordinary hypermedia system with the ones that used the adaptive hypermedia system that adapts to their levels of prior knowledge, we employed the independent onesample t-test as used in the analysis of learning performance. The means of certain questions from the exit questionnaires tailored to novices and experts using the ordinary interface were calculated, and treated as the hypothesised mean for the population, which was then compared to the means for the adaptive hypermedia version using the one-sample t-test. The statistical analysis, to make all the comparisons on the data, was conducted using Statistical Package for the Social Sciences (SPSS) for Windows version (release 15.0). A significance level of p < .05 was adopted for the study. 4.1 Learning Performance Analysis of learning performance was measured using the post-test scores and the gain scores (post-test score minus the pre-test score) for novice students using the ordinary hypermedia and the adaptive hypermedia versions. The ordinary version was used to hypothesise the scores in order for the mean to be compared with that of the adaptive version. The results from the one-sample t-test are summarised in Table 3. With respects to post-test score means, the results show that there was a significant difference in learning performance between users of the ordinary hypermedia version and the adaptive version (t (21) = 3.465, p = .002). In other words, the average posttest score of novices using the ordinary hypermedia version (M = 62.65, SD = 11.875) is significantly different from the novice using the adaptive hypermedia version (M = 72.23, SD = 12.965). The result shows that there was significant improvement in posttest scores when novice students used the adaptive hypermedia that was adapted to their levels of prior knowledge. The results of the gain scores indicate that there was a significant difference in learning performance between novices of the ordinary hypermedia and adaptive hypermedia versions (t (21) = 2.590, p = .017). In other words, the average gain score Table 3. Summary of the learning performance results (one-sample t-test) Novice Variable Po st-test Score Ga in Score
Mean (SD) M ean (SD)
Expert
Ordinary Hypermedia
Adaptive Hypermedia
Ordinary Hypermedia
Adaptive Hypermedia
62.65 (11.875)
72.23 (12.965)
73.40 (7.542)
74.61 (7.678)
35.42 (14.396)
43.09 (13.894)
10.32 (11.235)
15.33 (8.485)
160
F. Mampadi, S. Chen, and G. Ghinea
of novices using the ordinary hypermedia version (M = 35.42, SD = 14.396) was significantly different from that of novice using adaptive hypermedia version (M = 43.09, SD = 13.894). The result shows that there was significant improvement of gain score when novice students used the adaptive hypermedia system that was adapted to their levels of prior knowledge. With respects to expert users, the one-sample t-test indicate that there was no significant difference in mean post-test scores between users of the ordinary hypermedia and adaptive hypermedia versions (t(17)=.625, p = .540). This means that the average performance of experts, with respect to mean post-test scores, using the adaptive hypermedia version (M = 74.61, SD = 7.678) was not significantly different from that of experts using the ordinary version (M = 73.40, SD = 7.542). The results, however, showed that with respect to gain scores, there was slightly significant difference between users of the ordinary hypermedia and adaptive hypermedia versions (t(17)=2.507, p = .023). This means that the average performance of experts, with respect to mean gain scores, using the adaptive hypermedia version (M = 15.33, SD = 8.485) was significantly higher than that of experts using the ordinary hypermedia version (M = 10.32, SD = 11.235). In this respect, this result shows that the adaptive hypermedia system that adapt to individuals’ prior knowledge can improve students’ learning performance. These results are consistent with the majority of reported studies (e.g. [9,17]). 4.2 Perceptions and Attitudes To analyse the perceptions and attitudes of novice and expert users towards adaptive hypermedia that adapts to their levels of prior knowledge, we analyses users’ responses from the questionnaires. More specifically, a comparison is conducted between the results of the questionnaire from the ordinary hypermedia system and those from the adaptive hypermedia system. A one-sample t-test was used to determine if there was any positive improvement in perceptions and attitudes when students used the adaptive hypermedia version. The summary of results is shown in Table 4. Table 4. Summary of results for perceptions (one-sample t-test) Novices Statements
Experts
Ordinary Hypermedia
Adaptive Hypermedia
Ordinary Hypermedia
Adaptive Hypermedia
I felt the structure of this tutorial is not clear
Mean (SD)
2.57 (1.025)
1.77 (0.922)
2.16 (0.879)
1.72 (0.904)
I found the sequence of topics was logical It is hard to find a route for a specific topic in this tutorial Sometimes I found it hard to keep track which bits I had learnt.
Mean (SD)
3.01 (0.950)
4.41 (0.596)
3.42 (0.982)
4.44 (0.602)
Mean (SD)
3.68 (0.875)
1.77 (0.923)
3.07 (0.987)
2.50 (0.930)
Mean (SD)
3.64 (0.822)
2.77 (0.922)
2.46 (0.932)
2.06 (0.994)
The Effects of Prior Knowledge on the Use of AHLS
161
Students’ views on content structure. The adaptive hypermedia version was designed such that it offered overall a topic map for students with low prior knowledge and a topic index for students with high prior knowledge of XML. In the map version (used by novices), the content was organised into three levels, with topics, subtopics and pages, presenting a global picture of the entire hierarchical structure. In the index version (used by experts), the content was organised alphabetically with all of the nodes covering the topics, subtopics and pages in a network structure. The ordinary hypermedia system did not employ adaptive hypermedia techniques. In comparing the ordinary version and the adaptive version basing on the structural differences, the results indicated that there were significant differences in attitudes and perceptions between the groups. Two questions were extracted from the questionnaire to determine the views of students using the adaptive hypermedia system. The response was in comparison to those students that used the ordinary hypermedia version. With respect to novices, the results of the independent on-sample t-test (Table 4) indicate that there was a significant difference in perceptions and attitudes towards structure of the tutorials between users of ordinary hypermedia system and adaptive hypermedia version, t(21)=-4.055, p = .001. This means that students that used the adaptive hypermedia version (M = 1.77, SD = 0.922) found the structure of the tutorial to be clearer as compared to those that used the ordinary hypermedia version (M = 2.57, SD = 1.025). The results of the one-sample t-test also indicate that there was a significant difference in perceptions and attitudes towards sequence of topics between users of the ordinary hypermedia XML tutorial and adaptive hypermedia version, t(21) = 11.116, p = .000. This means that the students that used the adaptive hypermedia version (M = 4.41, SD = 0.596) found the sequence of topics to be more logical than those that used the ordinary hypermedia version (M = 3.01, SD = 0.950). The same questions were posed to experts to determine their views on structure of their respective tutorials. There were significant differences in attitudes and perceptions between students that used the adaptive hypermedia version and those that used the ordinary hypermedia version. The one sample t-test results, t(17) = 2.470, p = .024 shows that in terms of clarity of structure experts users using the adaptive version (M = 1.72, SD = 0.904) found it slightly clearer than those using the ordinary hypermedia version (M = 2.16, SD = 0.879). A significant difference was also found when experts were asked about how they felt about the logical sequence of topics. As showed in Table 4, the results (t(17) = 8.500, p = .000), show that the experts students that used the adaptive hypermedia version (M = 4.44, SD = 0.602) found the sequence of topics to be more logical as compared to those that used the ordinary hypermedia version (M = 3.42, SD = 0.982). In summary, with respect to structure, the results of this investigation support prior research [15,19] that indicate that adaptive hypermedia systems that adapt the structure of the content to learners’ prior knowledge improve learning performance and are perceived as useful and enjoyable. Students’ views in terms of navigational structure. In terms of getting lost using the tutorial, novice users using the adaptive hypermedia version enjoyed using their version while their counterparts using the ordinary hypermedia version felt frustrated probably because they did not have a suggested route (implemented through the
162
F. Mampadi, S. Chen, and G. Ghinea
pre-requisite structure of the tutorial) to follow through the learning system (Table 4), hence prone to getting lost. The results of the independent one-sample t-test indicate that there was a significant difference in perceptions and attitudes towards getting lost using the tutorials between users of the ordinary hypermedia and adaptive hypermedia versions (t(21)=-11.010, p = .000). This means that students that used the adaptive hypermedia version (M = 1.77, SD = 0.923) found it easy to find a route for a specific topic in the tutorial as compared to those that used the ordinary hypermedia version (M = 3.68, SD = 0.875). The problem of navigation and getting lost was not as significant for experts as was for novices. However, experts using the adaptive hypermedia version felt that the navigational structure was better as compared to those that used the ordinary hypermedia system which did not have an index. As showed in Table 4, (t(17) = 2.202 , p = .042), experts found it slightly easier to find a specific topic when using the adaptive hypermedia version (M = 2.50, SD = 0.930) than the ordinary hypermedia version (M = 3.07, SD = 0.987). The results of the experiment support the findings of previous studies that matching individuals with their preferred navigational features improves the likeability of a hypermedia system. The studies have evaluated the effectiveness of different navigation tools for high and low prior knowledge users. They found that these tools influence users’ achievement and attitudes (Chen et al., 2006; Farrell and Moore, 2001). The adaptive hypermedia version of the tutorial used in this experiment used the same principles to adapt navigational structure to novice and experts. It used a map for novice students and an index for experts. Students’ view in terms of additional support. To provide additional support for novices the adaptive hypermedia system uses different colours for hyperlinks and the list of pages read and those that still need reading. Novice students using the ordinary hypermedia version (M =3.64, SD = 0.822) found it harder to track which bits of information they have learnt as compared to novice students that used the adaptive hypermedia version (M = 2.77, SD = 0.922) of the tutorial as shown in Table 4 (t(21) = -4.411, p = .000). For the experts, there was no significant difference between those who used the ordinary hypermedia version (M = 2.46, SD = 0.932) as compared to the ones that used the adaptive hypermedia version (M = 2.06, SD = 0.994). The results show that for t(17) = -1.966, p = .066. This may be due to the fact that both experts using the two systems did not find it hard to keep track of what bits the have learnt. A possible reason for this finding may be that they have the ability to restructure the material because they are already familiar with the basic concepts [14]. The results of the experiment are in support of previous research which shows that additional support when provided can help learners with a low level of prior knowledge in hypermedia learning, particularly in free navigation conditions [7,14]. Advisement, which provides learners with visual aids and recommended navigation paths, is helpful in preventing disorientation in non-linear hypermedia learning. As novice learners cannot rely on their prior knowledge to help them structure the text,
The Effects of Prior Knowledge on the Use of AHLS
163
graphical overviews and structural cues are powerful and beneficial in providing navigation guidance so as to ease disorientation problems[7].
5 Conclusions Two research questions were examined in this study: (1) whether adapting a hypermedia learning system to an individual’s level of prior knowledge has an effect on learning performance; (2) whether adapting a hypermedia learning system to an individual’s level of prior knowledge has an effect on perceptions. The answer to the first research question is that an adaptive hypermedia learning system that adapts to individuals’ levels of prior knowledge improves their learning performance and therefore affects it. The answer to the second research question is that an adaptive hypermedia learning system that adapts to individuals’ levels of prior knowledge affects perception of use. This study demonstrated that the adaptive hypermedia learning system is more beneficial than ordinary hypermedia learning system, especially for users with low prior knowledge. The study also supports the results of previous research which also emphasises on the usefulness of adaptive hypermedia learning systems tailored to learners’ prior knowledge. The study revealed, however, that not everybody benefits from the adaptive hypermedia learning systems, especially users with a high level of prior knowledge. A possible reason for this finding may be that they have the ability to restructure the material because they are already familiar with the basic concepts. Therefore, the major challenge lies with improving the learning performance and perceptions of high prior-knowledge users with adaptive hypermedia systems that adapt to their levels of knowledge. With respect to perceptions, the study has demonstrated that although students had more positive perceptions using the adaptive version over the ordinary version, there was a slight difference between experts using the adaptive version and those using the ordinary version. The study also revealed that, in comparing the results of learning performance and perceptions on the adaptive version, learning performance scores seemed to be slightly more significant than those of perceptions. That is, students seem to do well in terms of test scores but not showing as much positive attitudes towards the interfaces provided to them. Hence, this study recommends that interfaces design incorporate mechanisms that are dedicated to improving learner performance like pre-requisite structure driven by a learner model. However, there should be an investigation on how to improve perceptions of use for such systems, by analyzing the importance of cognitive styles. This type of design could maximize performance of learners using adaptive hypermedia systems. There is also a need to incorporate both prior knowledge and cognitive styles and determination of the right blend to maximise learning, in adaptive hypermedia systems design could then be established, with its associated pros and cons. This was a small-scale study, hence we recommend that further studies be undertaken with a larger sample to provide additional evidence. We believe that such evidence will not only help to improve the design of adaptive hypermedia learning systems, but also will also be useful for the development of other personalized applications, such as e-commerce systems and virtual training environments.
164
F. Mampadi, S. Chen, and G. Ghinea
References 1. Boyle, C., Encarnacion, A.O.: MetaDoc: An Adaptive Hypertext Reading System. User Modeling and User-Adapted Interaction 4(1), 1–19 (1994) 2. Brusilovsky, P., Pesin, L.: Adaptive navigation support in educational hypermedia: An evaluation of the ISIS-Tutor. Journal of Computing and Information Technology (1998) 3. Brusilovsky, P.: WebEx: Learning from examples in a programming course. In: Fowler, W., Hasebrook, J. (eds.) Proceedings of WebNet 2001, World Conference of the WWW and Internet, Orlando, FL, AACE, October 23-27, 2001, pp. 124–129 (2001) 4. Brusilovsky, P., Eklund, J., Schwarz, E.: Web-based education for all: A tool for developing adaptive courseware. Computer Networks and ISDN Systems 30(1-7), 291–300 (1998) 5. Caramel, E., Crawford, S., Chen, H.: Browsing in Hypertext: A Cognitive Study. IEEE Transactions on Systems, Man, and Cybernetics 22(5), 865–883 (1992) 6. Chen, S.Y.: A cognitive model for non-linear learning in hypermedia programmes. British Journal of Educational Technology 33, 449–460; Ghinea, G., Chen, S.Y.: The impact of cognitive styles on perceptual distributed multimedia quality. British Journal of Educational Technology 34, 393–406 (2003) 7. Chen, S.Y., Fan, J., Macredie, R.D.: Navigation in Hypermedia Learning Systems: Experts vs Novices. Computers in Human Behavior 22(2), 251–266 8. Cleary, C., Bareiss, R.: Practical Methods for Automatically Generating Typed Links. In: The Proceedings of the Seventh ACM Conference on Hypertext, Washington, DC (1996) 9. Dochy, F.: Assessment of prior knowledge as a determinant for future learning. Lemma BV/Kingsley Publishers, Ultrecht (1992) 10. Farrell, I.H., Moore, D.M.: The effect of navigation tools on learners’ achievement and attitude in a hypermedia environment. Journal of Educational Technology Systems 29, 169–181 (2001) 11. Ferguson, W., Bareiss, R., Bimbaum, L., Osgood, R.: ASK systems: An approach to the realization of story-based teachers. The Journal of the Learning Sciences 2, 95–134 (1992) 12. Ford, N., Miller, D.: Gender differences in Internet perception and use. In: Electronic Library and Visual Information Research. Papers from the third ELVIRA conference, April 30, 1996, pp. 87–202. ASLIB, London (1996) 13. Fullerton, K.: The interactive effects of field dependence- independence and Internet document manipulation style on student achievement from computer-based Instruction. Ed.D Dissertation. University of Pittsburgh (2000) 14. Last, D.A., O’Donnell, A.M., Kelly, A.E.: Using hypermedia: Effects of prior knowledge and goal strength. In: The annual meeting of the Society for Information Technology & Teacher Education, Washington, D.C (1998) 15. McDonald, S., Stevenson, R.J.: The effects of text structure and prior knowledge of the learner on navigation in hypertext. Human Factors 40, 18–27 (1998) 16. Mitchell, T.J.F., Chen, S.Y., Macredie, R.D.: Hypermedia learning and prior knowledge: domain expertise vs. system expertise. Journal of Computer Assisted Learning 21, 53–64 (2005) 17. Moerkerke, G.: Assessment for flexible learning. Lemma, Utrecht (1996) 18. O’Mahony, M.: Sensory adaptation. J. Sensory Studies I, 237–258 (1986) 19. Shin, E., Schallert, D., Savenye, W.: Effects of learner control, advisement, and prior knowledge on young students’ learning in a hypertext environment. Educational Technology, Research and Development 42(1), 33–46 (1994)
The Effects of Prior Knowledge on the Use of AHLS
165
20. Spyridakis, J.H., Isakson, C.S.: Hypertext: a new tool and its effect on audience comprehension. In: IEEE-IPCC 1991, pp. 37–44 (1991) 21. Weber, G., Specht, M.: User modeling and adaptive navigation support in WWWbased tutoring systems. In: Jameson, A., Paris, C., Tasso, C. (eds.) User Modeling: Proceedings of the Sixth International Conference, UM 1997, pp. 289–300. Springer, Vienna (1997) 22. Weber, G., Brusilovsky, P.: ELM-ART an adaptive versatile system for web-based instruction. International Journal of Artificial Intelligence and Education, this volume (2001)
Supporting Learners in Adaptive Learning Environments through the Enhancement of the Student Model Luca Mazzola and Riccardo Mazza University of Lugano, Faculty of Communication Sciences Institute for Communication Technology Via Buffi 13, CH-6904 Lugano, Switzerland Ph.: +41 58 666 4674; Fax: +41 58 666 4647 {luca.mazzola,riccardo.mazza}@lu.unisi.ch
Abstract. This positional paper presents our research aimed at finding some possible research directions towards the enhancement of the use of open student models in the field of Technology Enhanced Learning and Adaptive Systems. Starting from the historical evolution of the learner model, we will describe some possible uses of learner models and propose some possible directions of enhancement. We will present 6 possible directions of research, and 11 dimensions on analysis. The 6 directions have been evaluated against the dimensions, and tentative ranking has been proposed. The result of this analysis will guide the work on open learner models which will be undertaken in the context of the European Union funded project GRAPPLE [1] aimed at building an infrastructure for adaptive learning systems that will adopt the strategy of opening learner models to the course learners and instructors Keywords: Technology Enhanced Learning, independent Open Learner Model, Human Computer Interaction, adaptation.
Supporting Learners in Adaptive Learning Environments
167
This paper aims to explore the historical evolution of the student model, to identify the different approaches to model the level of knowledge of the learners and, based on this analysis, we will propose some possible directions of research on student models to support learners and encourage reflection in adaptive learning environments. The motivation behind this research is that adaptive learning systems have been found to be useful in engaging the learner more in the educational experience [6]. The necessity to offer a higher level of commitment with the learning environment, especially when the most engaging human aspects inside these platforms are not so evident, became one of the key success factors for the wide diffusion of computerbased learning. A large part of the research in the field of adaptive learning environments focuses mainly on providing the learner with a customized learning environment that adapts the content of the course to the user's knowledge and preferences [7]. These systems don’t usually allow for the fact that the learner, besides being a user of the learning platform, is part of a “community of learners” (i.e. the class). A number of researches on social aspects of learning [8] stress the importance of the social interactions that occur during the learning processes. Our proposal aims to contribute to the research on adaptive systems by exploring some possible ideas to extend the adoption of user models. In this view, the open student model is not only an internal component of an adaptive system, but is also a useful source of information that can be stressed to enhance the user’s commitment to the online learning experience. The next section begins with an historical analysis of the state of the art in the field of student models. We have identified 6 directions of research that could be explored in the research of learner models with the aim of encouraging the learner to inspect his/her model, and promote reflection as learning. Then an analysis, driven by a rating system based on 11 characteristics, proposes the most promising idea which will be investigated in the context of the EU-funded project GRAPPLE [1].
2 Modeling Learners in Technology Enhanced Learning Environments Our attention is drawn to the field of Technology Enhanced Learning environments. We believe that it is important to improve the user commitment and the effectiveness along the whole learning process undertaken by using these learning environments, while also considering the importance of education for humanity and the progress of civilization and science. We consider the application of adaptive features in learning systems particularly well-suited to achieving this goal. From the literature, we have identified four different approaches to the use of student models in educational systems, which are listed in the following sections. 2.1 Internal Models This approach, also known as “close models”, was developed to build an internal representation of the learners’ progress with the course, mainly to implement adaptive features. Research has been conducted on internal users and students models for a long time, with the aim of profiling the user in different domains and applications.
168
L. Mazzola and R. Mazza
Internal modeling procedures collect data from the users' actions and feedback, store it in an internal database, and provide the input data that can be used by the adaptive functionalities for user data reasoning. The modeling process is quite complex and involves different steps: deciding and structuring the information to be collected, collecting raw data, scrubbing, extracting new information with reasoning rules, and, if necessary, updating existing data. Internal models are built for system internal use only, as a black-box component: neither students nor instructors have the possibility of exploring the contents of the student model. 2.2 Open Models The idea of either partially or totally opening learner models to the learner for inspection was developed mainly to promote student self-reflection and awareness of the adaptive features of the learning environment. The term “scrutable user models” was introduced by Judy Kay in 1999 [9]. The notion of scrutability is related to the possibility of the learners to scrutinize the model to see, not only what information the system holds about them, but also the process used by the system to collect the data about the learners and the inferences based on that data. It must be noted that scrutability concerns inherently convey a complementary but different view on personalization, which stresses upon the learner's awareness of the personalization process he/she is committed to. Recent insistence on scrutability or “inspectable open learner models” [10] advocate for explicit communication to learners of the pedagogical aspects framing the personalized learning experience designed for them by an adaptive learning technology. Student models can also be opened to peers, but this raises new problems, such as privacy, control over personal data, and trustworthiness of the system. One possible approach to deal with some of these problems presented is to distinguish between friendship networks and peers' groups. Users can decide which part of their profile to release in a named (or anonymous) form, and who is authorized to access this data representation (peer models). It could be observed that opening the models to peers can foster collaboration (with friends) and competition (with peers) [11]. 2.3 Group Models The recent achievement of the social network centrality into the social constructivist theory provides reasons to investigate models that take social and group aspects into account. Group modeling is a recent field of research, in which the learners are modeled as a group instead of a set of single individuals. These model the characteristics of an identified group of learners, and aim also to present the position and the relative distance of profiles, in order to allow learners to compare and understand their own situations. Opening group models to the users may offer some advantages. It can help learners to reflect on their progress in the group context and understand the problems of other group members [12]. Group models have been used to support the collaboration between learners of the same group, and to foster competition in a group of learners [13]. Right now, only simple methods have been used to mine the group models. The most common is using the average individual values representing a particular aspect considered in the model.
Supporting Learners in Adaptive Learning Environments
169
2.4 Interactive Models Interactive models introduce some levels of interactivity that can be expressed in two ways: the first, called inspection methods, is the capability to manipulate the graphical representation of the model, change the appearance, apply filters to the visualization and thus achieve a more in-depth understanding of the model. The other option, called interactive methods, is the possibility to influence the model, both by changing the data used by the system to represent the user itself, or by challenging and convincing the system that the current profile doesn't represent the learner status well. In this case, one problematic aspect could be the verification of the real understanding that the user has of the model, and which data support the claim for changing the internal profile according to the user request. Some systems tackle this problem by using a challenging process: they ask learners to solve a problem related to the particular aspect that should be changed and decide if this modification will be done based on the performance reached by the learner in this process. In the next section we will take the most interesting aspects from each of the previously described modeling approaches into account, and will investigate how they could be extended with input from other fields to provide the user with a better experience.
3 Extending the Use of Open Learner Models Traditionally, research on Open Learner Models (OLM) has been carried out by scholars active in adaptive systems and educational technologies. This is evident as the main OLM applications are in adaptive systems. However, we believe that contributions from other fields of research could be beneficial to extend and improve the adoption of OLM. For instance, one of the key issues in OLM is how to graphically represent the model; techniques from Information Visualization could provide indications on how to select and encode data in a graphical format suited to the representation task of the user. Another aspect is how to aggregate information and find correlations between data; techniques from artificial intelligence could help in this aspect. Other useful works carried out in different educational research could be adopted in student models. For example, Glahn et al. [14] propose the adoption of “smart indicators” as a way to aggregate user model information in a compact and intuitive way through a visual indicator that draws attention to ongoing relevant events only when really necessary. Erickson & Kellogg [15] at IBM T.J. Watson Research Center propose the “social translucence” idea as an approach to designing systems that support social processes: socially translucent systems are digital systems that “support coherent behaviour by making participants and their activities visible to one another and they have three main characteristics—visibility, awareness, and accountability—which enable people to draw upon their experience and expertise to structure their interactions with one another” [15, p.59]. More specifically, works from other disciplines have led us to identify some possible directions of research for OLM: P1
Positioning the learner with respect to the class or to a group of learners. This aims to use the OLM as a tool for supporting socialization, by making the student aware of the social context in which the learning experience is
170
L. Mazzola and R. Mazza
taking place, as suggested by the social translucence theory of Erickson & Kellogg [15]. P2
The introduction of innovative graphical interfaces. Contributions for Information Visualization [16] may help in investigating how to design innovative graphical interfaces, in order to reduce their complexity and the cognitive load required to interpret the underlying data, and adopt interaction techniques that enable the exploration of different aspects of user data.
P3
The representation of the temporal evolution of the model. The student model is not static, it changes as the student performs any action in the learning environment. A research work could explore whether opening this dynamic evolution of the learner model could be beneficial to the students (or to the instructors).
P4
The use of adaptive representations of OLM. An initial attempt to introduce some of the ideas of adapting the graphical representation of OLM comes from the work on “smart indicators” by Glahn et al. [17]. The idea is that the introduction of adaptive features in the graphical representation could help students to perceive the OLM in a more meaningful way. This allows the production of more engaging and effective external representations of OLM, and could provide a better experience to the users of the learning applications.
P5
The exploration of techniques that allow the definition of a global student model, by integrating different independent student models from different courses. Traditionally, student models keep track of the learner's knowledge and the skills acquired during the learning process on a course basis. Although a learning platform may run different courses, and students can enroll in several courses, each course has an independent student model. This is primarily due to practical, as opposed to theoretical, reasons: it is very hard to define a global ontology that models the concepts for several courses.
P6
The introduction of a metric, a function that defines a “distance” between students, based on the data stored in the learner model. This also allows the identification of groups of students having similar profiles.
Proposals P1, P2, P3, and P4 deal with an externalization of one or more aspects of the learner model, where the information is related to a specific course. P5 and P6 deal, instead, with more complex subjects: the creation of a global model of the user (P5) and the definition of a metric to measure the distance on learner models (P6) are very complex tasks for which research is still in the early stages. P5 and P6 are possible through data mining techniques, that work mainly with student tracking data (logs) to extract relations between data and can work even if a global model of the domains of the courses is missing [18]. In order to investigate how the research directions specified above could be effective in the learning process, we have identified a number of dimensions of analysis. These dimensions take into account the level of difficulty in implementing
Supporting Learners in Adaptive Learning Environments
171
the idea, the difficulty in managing the new functionalities proposed, the expected impact at the system level, and the estimated benefits offered to the student in terms of the metacognitive skills that can be acquired. The dimensions that we have considered are: D1 To what extend does this research enhance socialization among students? D2 How much effort is required for the knowledge extraction and for reasoning over the data? D3 The computational complexity needed to maintain the model. D4 The difficulties in identifying one or more metrics. D5 The granularity of representation of the problem space (continuous, stepped or discrete) [16]. D6 The amount of data required to have a reliable model. D7 The difficulty in identifying the most useful data to collect and the level of aggregation. We also propose other dimensions of analysis, which are more related to the experience, interaction, and mental model of the learner: D8 The novelties and the benefits introduced by the new graphical interface. D9 The impact on the cognitive load of the learner caused by the new information presented and the new way of presentation. D10 The complexity of the rules that drive the creation of the model and the speed of their convergence into a stable state. D11 The level of interactivity and the interaction type (continuous, stepped, passive or composed) [16].
4 Analysis of Proposed Dimensions To investigate which dimension seems useful for exploration in our EU project Grapple, we have created an evaluation space and divided it into 4 areas. Each area represents an aspect that we would like to stress: the social aspects that can be influenced, the innovation in the graphical user interfaces that a research approach may bring, a representation of the temporal evolution, and the clustering of data that comes from learner models. The first step was an identification on the 4 areas of the 6 proposed directions of research (see Fig. 1). In order to identify which proposals among P1 ... P6 are the most promising and worth investigating, we decided to set up a ranking system based on the 11 dimensions (D1 ... D11) defined previously. These dimensions were used to rate every proposal according to a 5 level scale (from a to e). Since some dimensions describe positive aspects (such as enhancing socialization, D1) and others denote problematic aspects (such as the computational complexity needed to maintain the model, D3), we divided them into two groups (D1, D5, D8, D11 and D2, D3, D4, D6, D7, D9, D10) and rated them using a direct scale (e is the highest value in the scale)
172
L. Mazzola and R. Mazza
Fig. 1. The projection of the 6 proposals on a 4-area evaluation-space
for the first group and a negative one (a is the highest value in the scale) in the second. Then, each proposal P1, ..., P6 has been matched with each dimension D1, ..., D11 and the match has been empirically assigned with a value in the scale (a ... e) by a team of experts in educational technology at the laboratory of eLearning at the University of Lugano. The result of this exercise is illustrated in Table 1. This table is also used to derive a sort of ranking among the different proposals (columns Pos). Table 1. Matching proposals against dimensions D1
D5
D8
D11
Pos
D2
D3
D4
D6
D7
D9
D10
Pos
Tot
P1
a
c
d
e
3
b
c
a
b
c
c
d
2
2
P2
e
a
e
e
6
a
b
a
a
d
c
e
1
3
P3
d
d
c
e
5
d
d
d
d
e
e
e
6
6
P4
e
a
a
b
1
d
b
a
c
c
b
d
2
1
P5
c
c
d
c
3
d
e
d
e
c
e
d
4
3
P6
b
a
d
c
2
e
d
e
e
c
e
c
5
3
The data shown on Table 1 has been visually encoded into a star plot graphical representation [19]. From the visualization depicted in Fig. 2, according to the findings on dimensions defined in Table 1, the proposal P4 (The use of adaptive representations of OLM) and P1 (Positioning the learner with respect to the class or to a group of learners) appears to be the most promising. After this analysis, we decided to concentrate our work on the direction of introducing graphical interfaces adapted to the learners' characteristics stored in the learner model. Moreover, we want to integrate some support for social aspects in our work, such as the positioning of learners in the class or group (P1). The final two proposals (P5 and P6) are very interesting from the point of view of possible
Supporting Learners in Adaptive Learning Environments
173
Fig. 2. Star plot of analysis dimensions for every proposal
outcomes in the user experience, pose a great number of open issues and will be classified as possible research extensions of this work.
5 Conclusions We have analyzed some possible research directions towards the enhancement of the use of open learner models in the field of Technology Enhanced Learning and Adaptive Systems. The aim of this study was to identify, among a list of possible research directions, the most promising according to a list of 11 dimensions. The proposals of research have been evaluated against the dimensions, and a tentative ranking has been proposed. We located the most promising and plan to explore them in the context of the EU-funded project GRAPPLE. A further aim of this paper was to stimulate the cooperation of researchers from different disciplines. The research directions proposed in this paper had the specific purpose of collecting feedback and asking other researchers to share their experiences and to foster collaboration.
174
L. Mazzola and R. Mazza
We believe that this work could help, as an initial seed, to lead to a self-analysis by researchers into the improvements that could bring Personal Learning Environments to a new level of commitment and awareness in the users' experience.
Acknowledgments This work was supported by the 7th Framework Program European project GRAPPLE ('Generic Responsive Adaptive Personalized Learning Environment'). The first author wishes to thank all of his colleagues and the coordinators of the Ph.D. School Red-Ink at the University of Lugano, as well as the researchers involved in the NewMinE lab and the eLab at USI for the interesting and useful discussions and ideas provided.
References 1. GRAPPLE project, http://grapple-project.org 2. Esposito, F., Licchelli, O., Semeraro, G.: Discovering Student Models in e-learning Systems. Journal of Universal Computer Science 10(1), 47–57 (2004), http://dx.doi.org/10.3217/jucs-010-01-0047 3. Brusilovsky, P., Peylo, C. (eds.): Adaptive and intelligent Web-based educational systems. International Journal of Artificial Intelligence in Education 13(2-4) 4. Graf, S., Kinshuk: Learner Modelling Through Analyzing Cognitive Skills and Learning Styles. In: Adelsberger, H., Kinshuk, P., Pawlowski, J., Sampson, D. (eds.) Handbook on Information Technologies for Education and Training, 2nd edn., pp. 179–194. Springer Publishing Company, Incorporated, Heidelberg (2008) 5. Bull, S.: A Simple Student Model to Make Students Think. In: Jameson, A., Paris, C., Tasso, C. (eds.) User Modelling Proceedings from 6th International Conference, UM, pp. 315–326 (1997) 6. Conlan, O., O’Keeffe, I., Brady, A., Wade, V.: Principles for Designing Activity-based Personalized eLearning. In: IEEE International Conference on Advanced Learning Technologies (ICALT 2007), pp. 642–644 (2007) 7. Dolog, P., Henze, N., Nejdl, W., Sintek, M.: Personalization in distributed e-learning environments. In: Proceedings of the 13th international World Wide Web Conference on Alternate Track Papers &Amp; Posters, WWW Alt. 2004, pp. 170–179. ACM, New York (2004) 8. Dillenbourg, P., Fischer, F.: Basics of Computer-Supported Collaborative Learning. Zeitschrift für Berufs- und Wirtschaftspädagogik. 21, pp. 111–130 (2007) 9. Kay, J.: A scrutable user modelling shell for user-adapted interaction, PhD Thesis, Basser Department of Computer Science, University of Sydney, Australia (1999) 10. Bull, S., Nghiem, T.: Helping Learners to Understand Themselves with a Learner Model Open to Students, Peers and Instructors. In: International Conference on Intelligent Tutoring Systems 2002 - Workshop on Individual and Group Modelling Methods that Help Learners Understand Themselves (2002) 11. Bull, S., Mabbott, A., Abu-Issa, A.: UMPTEEN: Named and Anonymous Learner Model Access for Instructors and Peers. International Journal of Artificial Intelligence in Education 17(3), 227–253 (2007)
Supporting Learners in Adaptive Learning Environments
175
12. Vassileva, J.I., Greer, J.E., McCalla, G.I.: Opennes and Disclosure in Multi-agent Learner Models. In: Proceedings of the Workshop on Open, Interactive, and Other Overt Approaches to Learner Modelling at AIED 1999, Lemans, France (1999) 13. Vassileva, J.: Open Group Learner Modeling, Interaction Analysis and Social Visualization. In: Dimitrova, V., Tzagarakis, M., Vassileva, J. (eds.) Proceedings of Workshop on Adaptation and Personalisation in Social Systems: Groups, Teams, Communities. Held in conjunction with UM 2007, Crete (2007) 14. Glahn, C., Specht, M., Koper, R.: Reflecting on Web-readings with Tag Clouds. In: Conference Paper, Computer-based Knowledge & Skill Assessment and Feedback in Learning Settings (CAF) Special Track at the 11th International Conference on Interactive Computer aided Learning (ICL 2008), September, 24-26, 2008, Villach, Austria (2008) 15. Erickson, T., Kellogg, W.A.: Social translucence: using minimalist visualizations of social activity to support collective interaction. In: Höök, K., et al. (eds.) Designing information Spaces: the Social Navigation Approach, pp. 17–41. Springer, Berlin (2003) 16. Spence, R.: Information Visualization: design for interaction, 2nd edn. Pearson Education/Prentice Hall, Harlow (2007) 17. Glahn, C., Specht, M., Koper, R.: Smart indicators to support the learning interaction cycle. International Journal of Continuing Engineering Education and Lifelong Learning 18(1), 98–117 (2008) 18. Minaei-Bidgoli, B.: Data Mining for a Web-Based Educational System. Doctoral Thesis. Michigan State University (2005) 19. Mazza, R.: Introduction to Information Visualization. Springer, London (2009)
The Concept of IMPRESSION: An Interactive Instruction System and Its Practice for Real-Time Distance Lessons between U.S. and Japan Takashi Mitsuishi1, Fumiko Konno1, Yuki Higuchi2, and Kentaro Go3 1
Graduate School of Educational Informatics, Tohoku University Kawauchi 27-1, Aoba-ku, Sendai, Japan {takashi,fumiko}@ei.tohoku.ac.jp 2 PRO & BSC Corporation Honcho 2-1-8, Aoba-ku, Sendai, Japan [email protected] 3 Interdisciplinary Graduate School of Medicine and Engineering,University of Yamanashi Takeda 4-4-37, Kofu, Japan [email protected]
Abstract. In order to perform flexible and effective lesson, we proposed “Double Loop Instructional Design Process Model” and developed an interactive instruction system; IMPRESSION based on the proposed model. In this paper, we show the concept of IMPRESSION and also describe the realtime distance lessons we conducted continuously in the first semester of 2007 with IMPRESSION and Skype by connecting between Mountain View, CA, U.S. and Tohoku University, Sendai, Japan via the Internet. As a result of these lessons, we confirmed that although we had a little time lag and some QoS problems of video stream caused by narrow bandwidth and very long distance, we could perform flexible and effective real-time distance lessons by using IMPRESSION and a videoconference system like ordinary lessons such as chalk and talk lessons in a classroom. However, we also found some points to be improved on for continuous use of IMPRESSION. Keywords: real-time distance lesson, interactive electronic whiteboard system, multimedia materials, instructional design process model.
The Concept of IMPRESSION: An Interactive Instruction System
177
modify lesson plan immediately based on formative evaluations in a lesson, and developed an interactive instruction system; IMPRESSION[6]. In this paper, we show the concept of IMPRESSION and also describe the result of the real-time distance lessons we performed continuously for half a year by connecting between U.S. and Japan via the Internet with IMPRESSION and a videoconference system.
2 Double Loop Model and IMPRESSION IMPRESSION is an instruction system for flexible and effective lesson with multimedia materials. It is a sort of sharable and interactive electronic whiteboard system based on Double Loop Instructional Design Process Model (Double Loop model) we proposed [7]. 2.1 Double Loop Model Double Loop model is an instructional design process model for designing interactive lessons by using information technologies and it forms double loop shown in Fig 1. The outer loop defines the activities of designing a lesson, and is consists of (1) Plan, (2) Apply, and (3) Evaluate phases. The inner loop of Apply phase defines the activities in a lesson, and is consists of (1) Implement, (2) Check, and (3) Modify phases.
Plan Apply Implement
Modify
Evaluate
Check
Fig. 1. Double Loop Instructional Design Process Model
As mentioned above, Double Loop model, differ from existing models, defines modification process of instructional design in a lesson in addition to instructional design process of a lesson. Thus, it clarifies interaction between a teacher and students, and allows us to cope with unexpected learners’ reactions flexibly in a lesson. It improves effectiveness of a lesson. And we are also able to review processes of performed lessons after them and confirm the modifications from plans. It facilitates us to redesign next class with the results of actually performed lessons.
178
T. Mitsuishi et al.
2.2 Facilities and Features of IMPRESSION IMPRESSION was designed and implemented as an instruction system based on our Double Loop model and allows us to perform activities in Apply phase. It consists of two types of terminals; a terminal for instructor and terminals for student, which are equipped with pen-based input device such as Tablet PC, and a relay server as shown in Fig. 2. We are able to apply this system to both in-class lesson and distance lesson. And it also allows us to utilize various multimedia data provided by web servers over the Internet as educational materials. A teacher registers a set of multimedia data to use as educational materials in advance. During a lesson, he/she select ones at will from them, present them on a
Terminal for Instructor
Terminal for Student Lesson Records < ?x m l v er s i on= " 1. 0" en cod n i g= " euc - j p" ? > < e l c ur t e> < dr aw it m e= " 18 37" >
operations
g= " euc - j p" ?> < e l c ur t e> < st ar t da t a="2 005 0 / 5/ 04/ 1 6: 13 : 16" /> < /d r aw > < op er at e ti m e= " 11 62"> < end d at e=" 200 5/ <pr05/ es0en 4/ 17: t i d="I 1 6:mag 27" /e1 > - 1" /> < /l ect ur e> 0 " e nco di ng =" e uc- p j " ?>
operations
< end d e=" at e=" 200" 5/ 05/ 0 4/ 17: 1 6: 27" / > < /l ect ur e> < m ov i e d i ="M o vi e1- 1"> < see k t i m e= " 36. 21 79 83 2"/ > < /m o vi e> <en d dat e= " 20 05 0 / 5/ 04/ 1 7: 16: 2 7" / >
operations
Relay Server Students
Teacher Multimedia Materials Internet Web Servers
Fig. 2. System Construction of IMPRESSION
Fig. 3. A Screenshot of IMPRESSION terminal for instructor
The Concept of IMPRESSION: An Interactive Instruction System
179
whiteboard, handle them (e.g. move, resize), and draw annotations on them by handwriting with an IMPRESSION terminal for instructor. Fig 3 shows a screenshot of the terminal. With these operations, it facilitates us to conduct interactive lessons according to reaction of students with various multimedia materials not only in a class but also a distance class. Thus, it makes possible to conduct flexible lessons based on formative evaluation which was defined as Apply phase of Double Loop model.
3 Practice of Real-Time Distance Lessons between U.S. and Japan This time, we conducted a series of lessons as real-time distance lessons continuously every week in the first semester of 2007 with IMPRESSION and Skype by connecting between U.S. and Japan via the Internet. 3.1 Content and Class Format of Practiced Lessons The series of lessons we conducted this time is a course named “Fundamental of IT Education B.” This course has been opened at Graduate School of Educational Informatics, Tohoku University from 2002 and provided to graduate students who have not always studied computer science or engineering. The first author lectures on fundamental computer engineering and information technologies for education (e.g. logical operation, database, computer network, and so on) in this course. This time, the first author stayed at California for a business in 2007, so we decided to practice distance lessons. We conducted fourteen lessons through 13:00 to 14:30 on every Tuesday of Japan Standard Time (through 21:00 to 22:30 on Monday of Pacific Daylight Time) from April to July. Participated students were five, and the second author, who is a doctoral candidate, also participated as a teaching assistant in order to help students and operate computer systems for IMPRESSION and a videoconference system. So far, the teacher had been preparing a lecture note and providing it before each lesson, assigning homework after every lesson, marking it and giving feedback to it, and returning it with comment at a next lesson. Therefore, this time, we provide lecture notes, receive reports for assignment, and return marked reports through a LMS of ISTU[8]. But, TA marked reports and gave feedback to them at this moment. 3.2 Implemented Environment and System Configuration Fig. 4 shows an implementation environment and a system configuration for the practiced distance lessons. This time, we connected between a room in an apartment house at Mountain View, CA, U.S., where the first author lived in, and a classroom at Tohoku University, Sendai, Japan, where students participated to lessons, via the Internet, and used Skype[9] for videoconference in addition to IMPRESSION as shared whiteboard. At the teacher’s room, we prepared a Table PC (Toshiba PORTEGE M400) for both IMPRESSION and Skype and connect it to the Internet via DSL. At Tohoku University, we prepared two Tablet PCs, one (Toshiba PORTEGE M400) with video projector is for IMPRESSION, and another (Toshiba Dynabook SS 3500) with 50in.
180
T. Mitsuishi et al.
Student Student side side (Tohoku (Tohoku University, University, Japan) Japan) Screen for Skype
Screen for IMPRESSION
Teacher Teacher side side (U.S.) (U.S.)
Terminal for IMPRESSION
TA
U.S. → Japan: about 70Kbps
USB Camera teacher
Terminal for Skype Internet Internet
TAINS/G
students
USB Camera
Japan → U.S.: about 110Kbps
DSL Terminal for IMPRESSION and Skype
Fig. 4. Implemented Environment and System Configuration of Practiced Distance Lessons
wide screen plasma monitor is for videoconference by Skype. In order to capture the video, we prepared USB camera on both site. We also prepared speakers in the classroom. The teacher used Bluetooth wireless headset in order to prevent howling. 3.3 Results of the Practices Fig. 5. (a) shows the front of the classroom where students participated and Fig. 5. (b) shows the terminals which TA operated in Japan. Fig. 5. (c) shows the teacher conducting a lesson by operating the terminal. Fig. 6 shows a screenshot of IMPRESSION in a practiced lesson.
Video from Teacher (Skype)
Shared Whiteboard Screen (IMPRESSION)
Terminal for Skype
Terminal for IMPRESSION
Terminal for IMPRESSION and Skype
(a) Front of a Classroom in Japan (b) Terminals in Japan (c) A Terminal in U.S. Fig. 5. Scenes of Practiced Distance Lessons
As results, although there were some problems that were necessary to reboot Skype for a few times because video streams froze and some audio communication failure occurred, IMPRESSION worked with no problem and we were able to conduct whole planned lessons almost well. However it was not communication failure, there exited about 1 second time lag in audio and video streams which may be caused very long distance communication and encoding and decoding of streams. By restricted bandwidth, quality of video images was also not so clear. Therefore, we could not distinguish expression of each student very much even if he/she was closer to the camera when we wanted to see whole students.
The Concept of IMPRESSION: An Interactive Instruction System
181
Fig. 6. A Screenshot of the Shared Whiteboard Screen in a Conducted lesson
4 Evaluation Here we describe subjective evaluation by the teacher and also evaluation by participated students. 4.1 Subjective Evaluation by the Teacher This time, although we had some problem on videoconference by Skype, we confirmed that we were able to conduct distance lessons with a videoconference system such as Skype and IMPRESSION as an interactive instruction system in combination. And it seems that they almost stand comparison with ordinary in-class chalk & talk lessons from result of assignments and a term-end exam. As we mentioned in the results, there exists time some lag in video streams and resolution of video images are restricted. We confirmed that these lacks of quality were affecting communication between the teacher and students. For example, even though sounds are clear enough, time lag upset timing to speak and prevent smooth communications. And the teacher sometimes felt difficulty to decide to put forward or explain again with different words because it was difficult to grasp whether students certainly understood or not from vague video images. On the other hand, IMPRESSION worked well with its features although there are some restriction such as screen size and resolution in comparison with ordinary blackboards. For example, we are able to present any materials at will by registered them in advance, it reduced the load to write complex figures and tables in a lesson and the teacher could concentrate on explanations. IMPRESSION is able to go back to the screen in the past. With this feature, the teacher showed again the drawn screen after putting forwarding explanations. Such a progress is impossible in ordinary lesson with blackboard. And the more, by giving the right of operation to students, the teacher and students were able to interact with each other through the numbered cards on the screen of IMPRESSION when we discussed sorting algorithm. Therefore it facilitated the teacher to conduct not only flexible lessons which were the original objective of IMPRESSION but also smooth and effective lessons.
182
T. Mitsuishi et al.
However, we also found some problems to be solved as will be discussed later in this paper. 4.2 Evaluation by Students In order to investigate the effectiveness of distance lessons with IMPRESSION and Skype, we conducted the survey with questionnaires to students [10]. Fig. 7 shows part of the result of questionnaires; from Q1 through Q7 on the systems and Q8 and Q9 on help and feedback by TA.
22
Q1: Could you see the screen of videoconference enough? ……….……….……….…………..
11
22
11
22
11
Q2: Could hear the voice of videoconference enough? ……….……….……….…………………
11
Q3: Did you think you could communicate with the teacher by videoconference? ………........
11
33
11
Q4: Could you see the screen of whiteboard system enough? ……….……….……….………..
11
33
11
Q5: Did you think materials and annotations help your understanding? ………………………..
22
22
11
Q6: Did you think you could interact with the teacher via whiteboard system? ……….…….....
22
22
11
Q7: Did you think these distance lessons were enough in comparison with ordinary lessons?
22
22
11
44
Q8: Did you think consultation with TA helped your study? ……….……….…….…………........
33
Q9: Did you think feedback to assignments from TA help your study? ……….……….….…….
0 Yes
Relatively Yes
Neutral
11
1
11 2
Relatively No
3
11 4
5
No
Fig. 7. Results of Questionnaire to Students Table 1. Reasons why they felt not enough with these distance lessons
- Face to face lessons seem better than distance lessons. - It seems a little difficult to communicate with the teacher. - Sometimes voice had disconnected. - Much better communication line seems necessary. Although it doesn’t have statistical meaning because the amount of participants were a very few at this moment, we can grasp a rough tendency. As a result, students felt that the systems were not enough but not too bad. About a question; “when you felt the systems are not enough, which are the problems of them or how to solve them?” they answered to this question as shown in Table 1. From these opinions, most of the reasons why they did not feel enough or satisfied with the distance lessons are lack of stability of Skype causing freeze of video streams and audio communication failure, lack of quality of video images caused restricted bandwidth and time lags caused very long distance.
The Concept of IMPRESSION: An Interactive Instruction System
183
As for drawing and presentation with IMPRESSION, although we had some negative answers to the question Q6: “Can you interact with the teacher via shared whiteboard screen?” we had relatively positive answers to the question Q5: “Is the explanation by drawing annotations and presenting materials on shared whiteboard screen helpful for your understanding?” It shows the effectiveness of features of IMPESSION for distance lessons. The reason of negative answers to Q6 might be assumed that we could prepare only one Table PC for IMPRESSION in a classroom, students should move to the terminal in order to operate it, and we could not interact at will via IMPRESSION. And these are not about the systems such as IMPRESSION and Skype, the second author helped students, checked reports for assigned homework and gave feedback as TA. Q8 and Q9 were questions asking how affect these help and feedback by TA and the answers were almost very well. Thus, we could assume these help and feedback by TA made up for restrictions and problems in communication between the teacher and students. 4.3 Confirmed Problems of IMPRESSION to Be Solved Although we find some restrictions of bandwidth and lack of stability of the videoconference system we used, these matters will be solved in the near future and we confirmed the effectiveness of IMPRESSION to sum up. However we also found some problems of IMPRESSION to be solved. The load to prepare materials and manage them is especially one of big problem of them. This time was the first time to use IMPRESSION in this course and the teacher should prepare materials. So, the load to prepare was of course heavy. However, in addition to the load to prepare materials, we found following two problems; 1) it was burdensome to register materials to IMPRESSION or to delete registered materials from IMPRESSION, 2) it was difficult to find an aimed material when so many materials were registered. Current IMPRESSION is just prototype to demonstrate interactive presentation of multimedia materials and has features for this purpose. Therefore it has only simple features and user interfaces to handle and manage materials. For example, it allows us to register materials one by one, but we are not able to register a set of materials at once. And it does not have a feature to sort out registered materials nor retrieve an aimed material by query. So, when we select a material to present in a lesson, we need to search the aimed material from all registered materials. This time we conducted lessons continuously every week and materials for each lesson were different from others. So we should register a new set of materials every week. And, we assumed it was difficult to find an aimed material to present when so many materials were registered, we deleted materials which were registered for previous lessons. These were very burdensome tasks. Furthermore, even when we regard some materials were not necessary and deleted them, sometimes they became necessary in order to review previous lessons. It could be said as lack of reusability. In order to solve these problems and facilitate to select an aimed material from many materials and to reuse registered materials in previous lessons, it is necessary not only to improve user interfaces but also to develop some features to sort and manage materials based on lesson plans.
184
T. Mitsuishi et al.
5 Conclusion This time, we conducted real-time distance lessons continuously for half a year by connected between U.S. and Japan with Skype and IMPRESSION via the Internet. As a result, we confirmed that although there were several restrictions on videoconference via the Internet such as time lags and resolution of video images, we were able to conduct real-time distance lessons with a videoconference system and our IMPRESSION, which allows interactive presentation of multimedia materials and drawing annotations on a shared whiteboard screen among a teacher and students, and the effectiveness of such lessons in comparison with ordinary lessons in a classroom. However, we also found the problems of IMPRESSION to be solved in order to conduct lessons continuously such as usability of user interfaces and features to handle and manage many materials. We are developing such features[11]. On the other hand, we will be able to record conducted lessons when we conduct them with a system like IMPRESSION, and it could be expected to create contents for review by students or to prepare data for reflection by the teacher[12][13][14][15][16]. We will solve the confirmed problems, improve our system, and propose novel educational environments as our future work. Acknowledgments. A part of this work was supported by KAKENHI (16700550 and 20700631).
References 1. Yoshino, T., Inoue, Y., Yuizono, T., Munemori, J., Ito, S., Nagasawa, Y.: Development and Application of a Supporting System for Distance Learning Classroom Using Personal Computers via Internet. IPSJ Journal 39(10), 2788–2801 (1998) 2. Murakami, M., Yagi, K., Kakusho, K., Minoh, M.: Analysis of Evaluation Factor of Distance Learning System Focusing on Effect of Experience and Custom in Lecture (in Japanese). IEICE Transactions on Information and Systems J84-D-I(9), 1421–1430 (2001) 3. He, A., Kara, A., Cheng, Z., Go, K., Koyama, A., Huang, T., Imamiya, A.: RIDEE-SPS: A Presentation System for Realtime Interactive Distance Education Environment (in Japanese). IPSJ Journal 44(3), 700–708 (2003) 4. Hayashi, T., Watanabe, K., Otani, M., Tanaka, H., Okazaki, Y., Hayashida, Y., Kondo, H.: Design and Implementation of a Remote Lecture Based on Instruction with Blackboard Using High-Quality Media Devices and High Speed Information Network (in Japanese). Journal of Japanese Society for Information and Systems in Education 22(1), 3–14 (2005) 5. Matsuura, T., Maeda, K., Kohno, E., Kishida, T.: A Telepresentation System to Assist Presenting Rich Multimedia to Audiences in Remote Locations (in Japanese), Technical Report of JSiSE Chugoku branch 5(4), pp.16-19 (2005) 6. Higuchi, Y., Mitsuishi, T., Go, K.: An Interactive Multimedia Instructional System: IMPRESSION for Double Loop Instructional Design. IEICE Transaction on Information and Systems E89(6), 1870–1884 (2006) 7. Higuchi, Y., Konno, F., Mitsuishi, T., Go, K.: A Double Loop Instructional Design Process Model for Interactive Instructions. Japan journal of educational technology 3(4), 457–468 (2008)
The Concept of IMPRESSION: An Interactive Instruction System
185
8. Mitsuishi, T., Kumai, M.: ISTU: Internet School of Tohoku University (in Japanese). Journal of IEICE 86(11), 816–820 (2003) 9. Skype web site, http://www.skype.com/ 10. Mitsuishi, T., Konno, F.: A Report on Practice of Distance Class between Japan and U.S. with ISTU and Real-time Distance Education System (in Japanese). Annual report of ISTU 4, 9–21 (2006) 11. Suzuki, T., Konno, F., Ohkawa, Y., Mitsuishi, T.: Design and Implementation of a Lesson Plan Based Data Management Mechanism for Educational Materials in IMPRESSION. In: Proc. of 33th JSiSE Annual Conference, pp. 296–297 (2008) 12. Konno, F., Higuchi, Y., Mitsuishi, T.: A proposal of a methodology of supporting teacher reflection by presenting differences between planned and implemented actions. In: Proc. of AACE SITE 2007, pp. 1059–1064 (2007) 13. Konno, F., Higuchi, Y., Mitsuishi, T.: Teacher’s Awareness in New Reflection Methodology Using Highlighted Process Displays. In: Proc. of AACE SIT 2008, pp. 2665–2673 (2008) 14. Kanno, Y., Konno, F., Ohkawa, Y., Hashimoto, K., Mitsuishi, T.: A Proposal of Teacher Reflection Supporting System Based on Observation of Reviewing Activities with Handouts and Video Records (in Japanese). Educational Informatics Research (7), 1–8 (2008) 15. Konno, F., Higuchi, Y., Mitsuishi, T.: A Study on a Teacher Reflection Method by Presenting Differences between Lesson Plan and Actual Instructions (in Japanese). Japan journal of educational technology, 32(4) (printing) 16. Konno, F., Kanno, Y., Mitsuishi, T.: Effects of Teacher Reflection with Highlighted Level Process Displays: Presenting Differences between Lesson Plan and Implemented Actions. In: Proc. of AACE SITE 2009, pp. 1236–1243 (2009)
Improving Children’s Writing Ability Joana Pereira, Luís Carriço, and Carlos Duarte LaSIGE, Faculty of Sciences of the University of Lisbon, Portugal [email protected], {lmc,cad}@di.fc.ul.pt
Abstract. This paper presents IWA, a platform to aid children when learning how to write. The proposed system offers both tutor and child a certain degree of autonomy. IWA provides tutor and child with different interfaces. The features available to the tutor allow the definition and configuration of repetition exercises comprising letters, numerals and freeform gestures. The child interface supports the child in the task of solving those exercises. The system has been evaluated in two sessions with children. From the evaluation results and the feedback provided by a school teacher we conclude this to be a very promising system towards optimizing the repetition process required for perfecting hand-writing. Keywords: Hand-writing, Children, Education, Multimedia, Evaluation.
to support interaction mechanisms similar to the ones used in traditional learning techniques by having the child write directly on the device, in the same way she would do with paper. This will allow lessening the child’s adaptation efforts while offering a rich and motivating interaction experience. The paper reports on the development of such platform, named IWA (Improving Writing Ability), which applies gesture evaluation techniques in teaching to write scenarios. The platform strives to perfect some of the writing and calligraphy characteristics trough contextualized challenges and task execution, while making use of appropriate feedback mechanisms capable of stimulating children of target age ranges to a healthy and constructive learning experience. The next section presents a summary of related work. Afterwards, we present the IWA system supporting concepts, the tutor and pupil interfaces, the data storage and logging mechanisms and the gesture grading algorithm. This is followed by a description of the preliminary evaluation conducted and a presentation of the results obtained. Finally we discuss those results and finish with some concluding remarks and future work directions.
2 Related Work One very often applied technique in learning to write support methods based on recent technologies is gesture recognition and evaluation, from gestures performed with the assistance of haptic systems [1, 2, 3, 4]. Telemaque [1] is a system which employs a force feedback ready pen to help children learning how to write. According to its authors this system achieved positive results regarding the adoption of a proactive strategy to control handwriting movements by the children who participated in the evaluation [1]. However, the force feedback system limits the children’s movements to a specified area around the letter that has to be drawn, accompanying the movement. This system, aimed at reducing the errors made, also limits the child’s independence and, potentially, the child’s confidence to operate without the system’s assistance. Other systems targeted at improving the performance of people perfecting their writing skills have also been explored in the context of learning a second language. ITOUCH [2] is one such a system. It was conceived to help learning Japanese characters, with an animation completed by voice instructions teaching how the character should be drawn. This system also showed benefits, with a significant improvement in the classifications obtained by the persons who used the system over the ones who didn’t. However, the classification process had to be made by an independent teacher. IWA supports automatic classification and additionally is capable of providing feedback during the shape’s execution. Other studies [5] show how technology can help children to develop their reading skill and improve their knowledge. Systems using technology together with children appealing content, like games or animations, show great potential when correctly conceived. Given that children are one of the most complex target audiences, some guidelines to assist in the development of systems targeting them have been defined [6, 7]. One of those studies concerns the capability of a system or application to engage children [6]. The author presents some fundamental factors to achieve this
188
J. Pereira, L. Carriço, and C. Duarte
capacity, like supporting rich interactions and always provide adequate feedback, amongst others, while pointing out a set of precautions that should be considered when designing such systems. IWA follows these guidelines, thus becoming an engaging system, providing the sufficient independence to the child and the tutor, ensuring appropriate feedback is given to the child, and, additionally, being able to present important information regarding the child’s evolution to the tutor.
3 The IWA System The IWA system aims to provide the required autonomy for both tutors and pupils during the calligraphy learning, repetition and perfecting process. The system was developed targeting both user groups: teachers and pre-school and first school years children. For the teachers, the system provides a set of tools allowing them to define and configure the exercises to present to the pupils. For the pupils, the system offers a rich graphical interface for presenting the teacher defined exercises, developed taking into account their literacy level, and taking care to provide immediate feedback according to the child’s performance. Each exercise is presented to the child as a challenge. A challenge is based in two concepts: a template and a task. A template is a set of sequentially ordered points defining the shape of a gesture. The task defines a set of presentation and interaction settings. Current settings allow defining how the template is to be presented (with a solid or dashed line), what scale is to be used (thus varying the size of the presented shape) and how many repetitions are required for the challenge to be considered successful. A challenge is thus defined as an association of a task to a template. It is the responsibility of the tutor to define and configure each challenge before it being presented to the pupil. A set of challenges is named learning unit. In this fashion, the teacher can prepare a set of learning units to use in her classes. After being presented with a challenge, the pupil executes the assigned tasks and waits for the system attributed score. Each challenge overcome by the child provides an immediate reward and the enabling of new challenges. In the following sections we will present in greater detail both profiles available in the IWA system, the logging mechanisms offered and the algorithm employed to grade the child’s performance and on which is based the presented feedback. 3.1 The Tutor Interface The IWA user profile dedicated to the teachers’ population has as main goal supporting the definition of the challenges to present to the children. Prior to presenting the challenges to the children, these must be defined and appropriately configured by the teacher. To this end, the IWA system offers two main groups of features: one allowing the definition of templates and the other supporting the association of tasks to templates in order to create challenges. When defining templates the tutor is responsible for identifying the class of gesture to ask the children to do. Currently, IWA supports for classes of gestures: upper case letters, lower case letters, numerals and freeform. When defining a letter or a numeral the tutor can select the desired symbol by pressing the corresponding button on the
Improving Children’s Writing Ability
189
IWA tutor interface. Figure 1 presents this process for the letter “o”. The template is stored on the local file system, using the symbol as an identifier. When defining a free form gesture the tutor is requested a symbol name to act as its identifier. After defining the identifier the tutor draws the gesture in the drawing space on the right side of the interface (see figure 1). The tutor can define as many templates as whished for each letter, numeral or freeform gesture.
Fig. 1. Creating a template for the letter “o” in the IWA tutor interface
Once the templates have been defined the tutor can proceed with the challenges creation. All the previously defined templates are available to the tutor during the challenge creation process. After selecting the template, the tutor must configure the task the child will have to perform. As described above the task consists on a line following assignment, and the tutor can configure how the template is presented (line type and shape size) and how many repetitions will be requested. Additional details also have to be selected, like the voice instructions (which can be recorded at this point if not already available) or the difficulty settings for the scoring algorithm. Figure 2 illustrates this process.
Fig. 2. Configuring the challenge in the IWA tutor interface
190
J. Pereira, L. Carriço, and C. Duarte
The tutor interface also supports browsing of the challenges completed by pupils. The teacher can load the results of the challenge that has been submitted by the pupil and, besides seeing the score achieved and the time it took for the child to draw the shape, she can play an animation of how the child draw the shape, which will assist in identifying possible problems, like the child drawing the shape in the wrong direction, for instance. 3.2 The Pupil Interface Figure 3 shows the pupil graphical interface where the challenges will be solved by the children. The interface is organized in three functional areas: the top area consists of a set of buttons corresponding to the challenges; in one of the sides a colored progress bar is used to present the child’s gesture score; and a workspace with a horizontal guide line where the challenges are presented and the gestures executed.
Fig. 3. The IWA pupil interface
The challenges are presented and executed in a precise sequence. The challenge execution process starts with the child pressing the first button in the top area. In response to the button press, the challenge is presented in the workspace area through an animation exemplifying how the gesture is to be made. This animation can be replayed anytime the child wishes to, by pressing the corresponding button. After the gesture animation stops the shape’s line is drawn according to the teacher definition and a sound cue is played signaling the child can start the gesture drawing. Once the drawing task is concluded the child requests the grade by pressing the colored progress bar. In response the system slides the progress bar to the obtained score and follows up with one of two behaviors: plays a sound associated with a less achieved performance and then displays the same exercise for the child to repeat and perfect; plays a successful sound and unlocks the next challenge by enabling the related button and sharpening its image. This is expected to captivate the child, motivating him to reach good scores and unlocking new challenges. The described process is repeated until all the challenges are overcome. However, the child is able to repeat a challenge whenever he desires it.
Improving Children’s Writing Ability
191
The drawing process is supported by some assistance mechanisms. After the shape has been presented, or whenever the child lifts the pen from the drawing surface during the challenge execution, a graphical animation presents a hint of where the child should continue drawing the shape, and in what direction. Additionally, when the difference between the shape being drawn by the child and the template rises above a teacher defined threshold the line drawn by the child gets thicker and changes color as can be seen in figure 4.
Fig. 4. The drawing line thickens and changes color when the pupil strays away from the template
The images and sounds used in the pupil interface are tailored to child’s age and gender in order to try to motivate them as most as possible. 3.3 Data Storage and Logging Mechanism Every data related to challenge creation and their execution are stored in XML files. These files comprise a set of metadata with the challenge configuration and the set of sequentially ordered points defining the template used in the challenge, or the shape drawn by the child. The metadata pertaining to a challenge is the information characterizing both the template and the tasks defining the challenge. The metadata pertaining to a challenge execution consist of not only the challenge’s full characterization, but also of every point drawn with temporal information and the score achieved. For each attempt at solving a challenge a complete independent file is generated. The files storing the challenge solving attempts can be browsed and visualized later in the tutor’s interface to assist in the identification of difficulties the child may be going through. 3.4 The Grading Algorithm The grading of the child’s executed gestures is based on two main criteria. The first criterion is the average absolute distance between the drawn points and the template points. The identification of what is the closest point in the template for each point in the drawn shape is not a trivial problem when lines cross in the shape. This happens in
192
J. Pereira, L. Carriço, and C. Duarte
many of the alphabet letters. To prevent the algorithm to identify as the closest point, a point that is not in the same stroke, the comparison between the current point and the template points does not include all the template points. Instead, it includes just the next thirty points in sequential order from the last matched point. This prevents the matching with incorrect point, while supporting situations where there is a need to lift the pen to continue drawing in another part of the workspace. The second criterion is the percentage of the template that has been covered in the child’s shape. This prevents the situations where the child only draws part of the shape, even if they were drawn close to the template, which would result in a good grade if only the first criterion was used. Based on these two criteria, and the internal parameters configured by the teacher, the shape’s score is computed. The algorithm also works in a proactive manner, keeping an internal count of the consecutive failed attempts in the same challenge. This allows the mechanism that adjusts the line thickness and color according to the distance between drawn point and template point, to operate after the child has failed the challenge the number of times specified by the teacher. This provides an immediate feedback to the child on his performance, as presented on figure 4.
4 Preliminary Evaluation To assess the proposed system two preliminary evaluation sessions were conducted. Our goal was to assess the usability of the pupil interaction platform while a child is executing the challenges, and also our assumptions regarding the usefulness of a system like the one described towards an increase in motor skills related to handwriting tasks. To this end we set up two evaluation sessions, with a three and a half years old girl participating in one session and a four and a half years old boy participating in the other session. For each session twelve distinct challenges were prepared, totaling twenty four challenges distributed over two learning units. All challenges were presented in dashed lines, keeping the same scale that was used in the template definition. Each evaluation session took approximately twenty minutes. The application was executed in a LG P100 tablet PC. From the first to the second session, some usability improvements were made. This will not impact our conclusions since we will not try to establish any comparison between both sessions’ results. The sessions were conducted at the children’s homes, in order to stress them as little as possible. The children’s parents were present during the evaluation. They were briefed prior to the sessions, and agree on having the sessions recorded on tape. The tape was later used to assess the stress level felt by each child during the evaluation. During the first evaluation a school teacher, who is the child’s grandparent, was also present. Instructions on what was expected of the children were explained to them at the beginning of the trial, and every time they requested it, or was felt needed by one the evaluation team members. During the evaluations, both children provided very positive reactions regarding the system’s feedback mechanism, and, on their own initiative, wanted to repeat the
Improving Children’s Writing Ability
193
challenges after completing them successfully once. This allowed us to record results on two rounds per learning unit. After the session’s completion, results and information gathered were processed and analyzed, resulting in three metrics regarding the executed challenges: best score, average number of attempts until successfully completing the challenge, and average time taken per attempt. The score is assessed in a 0 to 100 scale, with 100 being the perfect score. Table 1 presents these results for both children. Table 1. Evaluation sessions‘ results
Best Score
Girl Boy
Round 1 72.13 69.16
Average # of tries Round 2 73.48 72.83
Round 1 1.75 1.5
Round 2 1.5 1.2
Average time per try (s) Round 1 Round 2 10.32 10.66 8.79 9.93
A quick analysis of the presented results shows that from the first to the second round in both sessions there was an increase in the best score, a reduction in the number of attempts per challenge and a slight increase in the time taken per attempt. Due to the small number of evaluation sessions conducted we feel it is not adequate to perform a detailed quantitative analysis of the results. A qualitative discussion of the evaluation sessions conducted is presented in the next section.
5 Discussion During the first evaluation session we identified some usability problems in the pupil interface. The school teacher present in the session also offered some constructive criticism regarding some pedagogical aspects employed in the IWA system. The scoring progress bar was placed on the right side of the workspace. For righthanded children, as was the case of both children involved in the evaluation session, this placement meant their arm covered the score bar. We noticed that after completing the shape the children did not remove the arm immediately, and failed to watch the progress bar animation. For the second session, the bar was placed on the left side of the workspace, in a way that was always visible to the child. During the first session it was noticed the child failed to follow closely the animation that exemplifies how the shape should be executed. We introduced a different cursor, with a pencil shape and a bigger size, which proved better at drawing the child’s attention. When the challenge used a template requiring multiple strokes (e.g. “E”), we noticed that, many times, the child did not know where to proceed after finishing a stroke and lifting the pen. We introduced a mechanism to hint at where the child should continue the shape’s execution, and in what direction, every time the pen is lifted. The way the score is presented was also changed. In the initial version the score indicator began its animation in the middle of the progress bar. This meant that when the achieved score was low, the indicator dropped. For the second session, this
194
J. Pereira, L. Carriço, and C. Duarte
behavior was altered. The score indicator starts at the bottom of the progress bar, which means that whatever the score is, the indicator always rises, improving the child’s motivation. Although only two evaluation sessions have been performed, it can be noticed a slight increase in the scores obtained and in the time taken to perform the shapes from the first to the second round. This can be a sign that the children were trying to perfect their execution to improve the classification achieved. Regarding the boy who participated in the second session it was perceivable that the degree of difficulty demanded (a score of 50 points or better to unlock the next challenge) proved to be a demotivating factor, due to the child’s proficiency level for the requested tasks. This effect can be avoided if the difficulty can be adjusted in execution time, based on the scores that have been achieved in the previously executed challenges.
6 Conclusion and Future Work This paper presented IWA, a system to assist both teachers and pupils in the handwriting learning process. The system employs pen-based interaction techniques in order to mitigate the effort required of the children to adapt to a technological solution. The paper also presented the results of a preliminary evaluation of the proposed system. The results were positive and extremely encouraging, regarding both the improvements perceived in the evaluation sessions and the enthusiasm shown by the children. Also the feedback provided by the school teacher was very positive and her suggestions were used to improve the system for the second evaluation session. However, from the small number of evaluations performed, we can envision several evolutions to the current system in order to improve both pupil and tutor interface and the grading algorithm. Perhaps the biggest evolution we could make to the system is to endow it with adaptive capabilities to adjust to difficulty level used in the grading algorithm in accordance to the acuity revealed by the child in the previous challenges.
References 1. Palluel-Germain, R., Bara, F., Hillairet de Boisferon, A., Hennion, B., Gouagout, P., Gentaz, E.: A visuo-haptic device - Telemaque - increases kindergarten children’s handwriting acquisition. In: Proceedings of the Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC 2007), pp. 72–77. IEEE Computer Society, Los Alamitos (2007) 2. Eid, M., Mansour, M., El Saddik, A., Iglesias, R.: A Haptic Multimedia Handwriting Learning System. In: Proceedings of the international workshop on Educational multimedia and multimedia education (EMME 2007), pp. 103–108. ACM Press, New York (2007) 3. Bara, F., Gentaz, E., Colé, P.: The visuo-haptic and haptic exploration of letters increases the kindergarten-children’s reading acquisition. Cognitive Development 19, 433–449 (2004)
Improving Children’s Writing Ability
195
4. Henmi, K., Yoshikawa, T.: Virtual lesson and its application to virtual calligraphy system. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1275–1280. IEEE Press, Los Alamitos (1998) 5. Liang, P., Johnson, J.: Using technology to enhance early literacy through play. Computers in Schools 15(1), 55–64 (1999) 6. Said, N.: Towards a ‘model of engagement’ Designing Multimedia Application for Children. Digital Learning 3, 1 (2007) 7. Wyeth, P., Purchase, P.H.: Using Developmental Theories to Inform the Design of Technology of Children. In: Proceeding of the 2003 conference on Interaction Design and Children, pp. 93–100. ACM Press, New York (2003)
From Paper to Module – An Integrated Environment for Generating SCORM Compliant Moodle Courses Out of Text and Multimedia Elements Hans-Martin Pohl, Benedikt Deicke, and Jan-Torsten Milde Fulda University of Applied Science HCI Research Center Heinrich-von-Bibra-Platz 3, 36037 Fulda, Germany [email protected], [email protected], [email protected]
Abstract. ECampus is a project spanning all departments at the University of Fulda. It has been started to create a uniform learning environment at the university. The objective is to research and develop a user-friendly easy-to-use editor to generate SCORM 2004 conform E-Learning modules. This editor is based on Open Source software and new technologies such as XSL transformations and the Google web toolkit. The whole system is provided as a web application and embedded within the E-Learning environment of the university. Some E-Learning modules can be developed with the system immediately. These modules are now being used during the lessons with great success. Keywords: SCORM 2004, XSLT, transformation, creation of content, E-Learning, modules, lesson, user friendly, style sheet, LOM, moodle, LMS.
2 State of the Art While there are currently a lot of different Learning Management Systems, the use is limited to but a few in Germany. Hereby Moodle (see http://www.moodle.org) and ‘Ilias’ (see http://www.ilias.de/ios/) play the greatest role. In contrast to the LMS there are only a limited number of editors for creating an ELearning module. HTML editors are widely-use but don’t provide adequate functionality. Furthermore E-Learning modules generated by an editor can mostly only be used within a proprietary LMS. Additionally, a lot of effort is required to learn how to use these editors. Such a system fails entirely if there has to be consistent development and presentation of tests and learning success. The advantage of these systems is the functionality for communication between users or between tutor and student. In our case this can be neglected because this functionality is captured by the LMS ‘System2Teach’ (see also http://www.system2teach.de) which is a proprietary development of the University of Applied Sciences Fulda. This paper focuses on easy and comfortable development of E-Learning modules based on textual content by using the eCampus framework. Concerning the technology for state of the art web applications there are many choices. Many different systems support the web based approach. Besides PHP (see http://www.php.net) or JSP (see http://java.sun.com/products/jsp/index.jsp) it is also possible to use Ruby (see http://www.ruby-lang.org) in combination with the Ruby on Rails framework (see http://www.rubyonrails.com).
3 The eCampus Project This research focuses on the development of a user-friendly system for writing E-Learning units, allowing authors to concentrate on the content and the didactic concept of the unit, instead of worrying about the underlying technology. It is part of the eCampus project at the University of Applied Sciences Fulda. In this interdisciplinary project, content is being created for the faculties Nutritional Sciences, Food and Consumer Sciences, Applied Computer Science, Nursing & Health Care as well as Food Technology. Within the first years of the project, many E-Learning modules were produced. All modules address prominent introductory courses of the particular program of study. This was only possible by using the eCampus framework which is described below. The central target of the project is the implementation of a tool that decreases the complexity of the process of learning module production. The central objectives of the learning module production are an increase in the interactivity of the course, as well as an increase in the quality and quantity of self learning. Currently LMS with SCORM 2004 support are not commonly available. Until Moodle is SCORM 2004 capable the format of transformed E-Learning modules will be SCORM 1.2.
198
H.-M. Pohl, B. Deicke, and J.-T. Milde
Fig. 1. System Architecture
3.1 Overview System Architecture The learning module production takes place in three phases (see Figure 1): • the structural annotation of the textual content • the automatic generation of the E-Learning unit • the automatic transmission into the LMS and allocation as SCORM compliant E-Learning module In phase one, most of the (textual) content is created in a standard word processor (e.g. Open Office, [10]). Here the text is structurally annotated using a predefined formatting template. The text is stored and automatically converted into a simple XML file, which provides the basis for subsequent transformation steps. This approach works very well for static content and is used in a number of systems (e.g. eLAIX (see http://www.boldt-media.de/)). In the second phase, the file prepared in phase one is transformed into a SCORM 2004 compliant E-Learning module automatically. For this, the document is unpacked and analyzed. To get by on cascaded XSL transformations (EXtensible Stylesheet Language) each element is identified and transformed with specific instructions (see Figure 2). Finally all contents are packed into an E-Learning module and multiplexed with the necessary auxiliary files. The available metadata is separated and attached to the module in a LOM (Learning Objects Metadata) compliant form. The complete process was formerly controlled by the Apache ant build tool ([4]) but is now ported to Rake (see http://rake.rubyforge.org). It uses saxon8 (see http://saxon.sourceforge.net/) as a transformation engine.
From Paper to Module – An Integrated Environment
199
Fig. 2. Transformation Tree of eCampus Framework
In the last phase the author is able to either download the education module or to transfer it to a Moodle server, where a new course is created automatically. Unfortunately Moodle currently lacks a stable web service API. Due to this the automatic transfer and creation of courses is not yet supported until Moodle fully implements this. In addition the transformation server itself also provides a RESTful Web service API. These interfaces make it possible to integrate the transformation server into other applications. 3.2 User Front End The user front end is very simple and easy to use. All main functions can be controlled by the main screen (see Figure 3). The menu bar on top contains the administration functions and an online help. The main part is the dark green beam with the input box for the source file. If the path is unknown the file can be selected using the browser's file chooser. Afterwards the transformation can be started with a single click. In advanced mode there is the possibility to set the name of the target file or assign a style for the presentation as single HTML files within a browser. Additionally it is possible to add media files like images, flash animations, audio or video. The lower part of the screen lists all transformations and displays their current state. It is possible to get a detail view and to download the results of completed transformations. The detail view (see Figure 4) displays the state of the transformation and contains the included source files, the resulting learning units and some functions to reannotate or retransform the source file. 3.3 Transformation Process The complete process is controlled by the Rake (see http://rake.rubyforge.org) build tool and uses saxon8 (see http://saxon.sourceforge.net/) as a transformation engine supported by a graphical user interface (GUI).
200
H.-M. Pohl, B. Deicke, and J.-T. Milde
Fig. 3. User Front End Main Screen
Fig. 4. Detail View of a Transformation
After starting transformation, the written Open Office document is unpacked. After that the file ‘content.xml’, which contains the content, is accessible and can now be transformed (see Figure 4). The first step of the transformation is to transform this ‘content.xml’ into a new file called ‘puretext.xml’. This new file is in an intermediate format. Thereby the
From Paper to Module – An Integrated Environment
201
annotated elements are assigned an explicit format template. Furthermore all elements without annotation are deleted. The result of this transformation is a universal intermediate format without direct correlation to the producer application. The transformation is controlled by the XSLT stylesheets ‘content2pure_1.xsl’ and ‘content2pure.xsl’. The advantage of this intermediate format is that the resulting transformations are totally independent of the content collecting process or application. As a result a content collecting process using other applications is also possible in the future, e.g. Microsoft Word. Only the transformation into the intermediate format has to be adapted.
Fig. 5. Abstract of the unpacked *.odt
The file in intermediate format can be transformed into the output format (here HTML) by applying the XSLT stylesheets ‘generateHTML.xsl’ and ‘html.xsl’ For this purpose each element from ‘puretext.xml’ is analyzed and the corresponding transformation instruction is searched. If the correct instruction is found, it is applied to the specific element. The transformation result is stored into the HTML file. To print out the content easily a further version is generated. In contrast to the first form all learning objects, sessions and sections, are stored within one page. This page can easily be printed by using the print functionality of the browser. An overall glossary is generated by applying the stylesheet ‘glossary.xsl’. Hereby all elements which are annotated as glossary entries are transformed into a separate glossary file. To use the learning module without an LMS there must be extra navigation. In this case all headlines from sections and sessions are transformed into the file ‘menu.xml’ supported by ‘menu.xsl’ The metadata which is stored within the ‘content.xml’ is read and transformed into a further intermediate format ‘intermediateMD.xml’. From this the ‘generate LOMMetadata.xsl’ generates the ‘modulID.xml’ and ‘parenttitle.xml’ which
202
H.-M. Pohl, B. Deicke, and J.-T. Milde
conforms to the LOM standard. Subsequently the complete content and all metadata is transformed into XML files. 3.4 Integration in LMS It is possible to manually download and import the resulting ZIP archive into any SCORM compliant LMS. Additionally, as the University of Applied Sciences Fulda is currently using Moodle, it is planned to implement an automatic transfer and import of generated learning modules into this LMS. Unfortunately Moodle's SOAP based web service API is currently under development and scheduled for its 2.0 release. As soon as the required functionality is implemented, the eCampus web application will be extended accordingly. By clicking a single button the user will be able to automatically create a new course in the Moodle LMS based on the previously generated learning unit.
4 Discussion and Prospects As the transformation process is run on a single central server, it becomes relatively simple to update and extend the system functionality. An expensive deployment onto multiple computers is not needed anymore and it is ensured that all authors are always able to use the up-to-date transformation process. The transformation server is based on the dynamic scripting language Ruby and the programming language Java. Both are using virtual machines which are available for various operating systems. Although the transformation server is currently running on Linux it is possible to run it on other platforms, too. Future works will be concentrating on the development of a web based editor making it possible to replace OpenOffice. This editor should be able to import documents in different formats, automatically analyse the document structure and provide functions to annotate these documents online.
References 1. Crockford, D.: RFC 4627. The application/json Media Type for JavaScript Object Notation (JSON), http://www.json.org/ (accessed, 13.01.2009) 2. David Eisenberg, J.: OASIS OpenDocument Essentials. LuLu.com (2004) 3. Hodgins, W., Duval, E., et al.: Draft Standard for Learning Object Metadata. Report, Learning Technology Standards Committee. IEEE, Los Alamitos (2002) 4. Holzner, S.: Ant - The Definitive Guide. O’Reilly Media, Sebastopol (2005) 5. Google Inc. GWT: the Google Web Toolkit, http://code.google.com/webtoolkit/ (accessed 13.01.2009) 6. Kay, M.: XSLT 2.0 programmer’s reference. Wiley, Indianapolis (2004) 7. Koper, R.: Modeling units of study from a pedagogical perspective (2001), http://eml.ou.nl/introduction/articles.html 8. Advanced Distributed Learning. SCORM 2004 (2004), http://www.adlnet.gov/scorm (accessed, 13.01 2009)
From Paper to Module – An Integrated Environment
203
9. Schulmeister, R.: Grundlagen hypermedialer Lernsysteme: Theorie - Didaktik - Design. Oldenbourg Verlag, München (2002) 10. OpenOffice.org team. OpenOffice, http://www.openoffice.org/ (accessed 13.01.2009) 11. W3C. XML, http://www.w3.org/MarkUp/ (accessed, 13.01.2006) 12. Wilde, E., Lowe, D.: XPath, XLink,XPointer, and XML. Addison-Wesley Professional, Reading (2002)
Development of a Simulator of Abacus: Ancient Analog Calculator on a Mobile Phone as a Teaching Material Kenta Saito, Yuki Makita, Vu Quang, and Hitoshi Sasaki Faculty of Engineering, Takushoku University, 815-1 Tatemachi, Hachioji, Tokyo 193-0985, Japan {myrte,makita}@eitl.cs.takushoku-u.ac.jp, [email protected] Hitachi Software Engineering Co., Ltd. 4-12-7, Higashishinagawa, Shinagawa, Tokyo 140-0002, Japan
Abstract. Portable electric devices such as mobile phones and video game consoles are becoming high performance and they are becoming ubiquitous. And a lot of people are in situations that allow for use of these applications. On one hand, it is useful for students to study out of class if they can use teaching materials at school with a portable device. And Japanese people had widely been using an abacus until the electronic digital calculator appeared recently. Even today, the abacus is used to teach math in Japanese primary schools teaching. We are developing a simulator of abacus as one of the computerizations of the teaching materials in this research. Keywords: abacus, android, education, teaching material, mobile phone.
developed functions which display the number to understand expression of peculiar number for abacus and set simple addition problem to practice the calculation for using this application.
2 Overview of Japan Style Abacus The arrangement of an abacus is five beads per row as in Figure 1. A single row shows one digit. The top five beads are shown of 5. Four beads under the top show one respectively. And as shown in Figure 2, number from 0 to 9 is shown by sequence of the beads. Students using an abacus learn skills that not only the calculation using an abacus is quick but also a mental calculation that the student images the abacus their mind. If a student can use the mental calculation, a student can come out calculations quickly without the abacus [4].
Fig. 1. Sharpe of a Japanese abacus
An abacus is used to understand a basic calculation at classes of primary school. However, they don’t have enough time to study it. Therefore almost students can’t master the mental calculation using abacuses. A lot of students who can make the mental calculation have been to the abacus classroom besides school classes.
Fig. 2. How to show number from 0 to 9
3 The Android Platform 3.1 Overview The Android is a platform that it was developed Google and Open Handset Alliance (OHA) [5]. OHA is a group of forty seven technology and mobile companies. They
206
K. Saito et al.
released Android as an open source. We can use Android and a development environment for free. T-Mobile releases G1 [6] as a mobile phone equipped Android. A million units had sold from October 2008 to December 2008 in USA. In addition, Google have sold Android Dev phone that was removed to SIM lock and Hardware lock from G1 since December 2008. There are some features in Android. One of features is using the touch sensor and the acceleration sensor. So, a user can sensuously operate Android with there sensors. And the Android implements OpenGL ES as a 3D graphic function and a database corresponding to SQLite. Moreover, Android can synchronize Google account. Therefore, a user can take the schedule the user made and message of Gmail. 3.2 Execution Environment of Android Applications The Android runs on a Linux operating system. Applications for the Android operate by using Dalvik Virtual Machine (DVM) of Java Virtual Machine (VM) implemented on Linux. DVM is what Google developed originally for the Android, it is suitable architecture for operation on a mobile phone. DVM is different from normally Java VM. Google develops originally for Android. And this is suitable shape to operate on mobile phone.
4 Development of Application for the Android 4.1 Background and Purpose Various teaching materials like the abacus are used in schools. However, it is costly to the purchase of a lot of teaching tools and student can't freely take these teaching materials out of classroom. We developed such a teaching material as application. Students can use a teaching material anywhere by using the teaching material application. Then in this research, we paid attention to the Android platform for mobile phones. The mobile phone is small. So, students can use the application any time and everywhere. Students easily can get the mobile phone for mass-production. The used mobile phone is cheaper than other portable electronic instruments. Therefore, there is a possibility of spreading because people in the poor segment of the population and the developing country can buy it. In addition, Android is free that license fee as for use regardless of the efficient platform. Therefore, there is a good possibility that famous companies sell Android mobile phone and people using other mobile phone use Android. 4.2 Operation Method and the Main Function We develop the simulator of abacus using a touch sensor function. The start screen of a simulator of abacus is shown in Figure 3. It uses like an actual abacus. Students can move a bead to touch it as shown Figure 4. When a leaner touches it, the application emits the sound in which the abacus moves. Also, if the student moves their finger like the arrow shown in Figure 5, all beads are initialized as shown Figure 3. Of course, this operation is the same as an actual abacus. Thus, we expect that the effect of the calculation power improvement as well as an actual abacus to reproduce the operation feeling similar to an actual abacus.
Development of a Simulator of Abacus
207
Fig. 3. Start screen of a simulator of abacus
Fig. 4. An example of beads operations
Fig. 5. Initialization operation
And we developed the function to display number shown with the abacus by numbers to understand expression of peculiar number to abacus as shown Figure 6. Therefore, the inexperienced student to handling the abacus becomes easy to understand expression of numerical value that uses abacus. In addition, we developed the computational problem mode. This mode set questions of the addition. Students can see a problem and current answer at the lower side of screen as shown Figure 7. Students operate numerical values using the abacus. It is possible to become accustomed to the abacus by solving the problem of addition in this function with an abacus, and the calculation ability improvement can expect it.
Fig. 6. Display of numerical value
208
K. Saito et al.
Fig. 7. An example of the computational problem mode
5 Conclusion We developed of the simulator of abacus using Android. This simulator achieved similar to an actual abacus by using Android, and students came to be able to use the abacus anytime anywhere. We plan to add the function that students can practice the abacus and the calculation test to use the abacus application. As a practice function of the abacus, we will develop the calculation function that the student listen to the sound reading number and add them and the correction function when students mistake the operation of the abacus. And we will develop an application of Android not only abacus but also other teaching materials. Moreover, we opened the source file of this simulator of abacus to the public because we would like for this application to be used by many students. The application can be improved in the future by taking into account student’s opinions.
References 1. Android, G., http://code.google.com/android/ 2. Hikaru, A.: Mental Arithmetics by Soroban Method and Teachers of Soroban. Annual report of researches in environmental education 11, 97–110 (2003) (in Japanese) 3. Japan Ministry of Public Management: The survey of telecommunications trends in 2007 (2009) (in Japanese) 4. Oskar, Y.M.K., Jesse, C.Y.W., Blake, C.J.Y., Chi-Jen, L., Tak-Wai, C.: Maintaining Student Engagement in Extensive Practice by Implanting Gaming Factor. In: The 16th International Conference on Computers in Education, pp. 721–728 (2008) 5. Open Handset Alliance, http://www.openhandsetalliance.com/ 6. T-Mobile G1 with Google, http://www.t-mobileg1.com/
A Proposal for a Framework for an e-Alumni Program Using SNS Hiroshi Sano Faculty of Foreign Studies, Tokyo University of Foreign Studies 3-11-1 Asahi-cho, Fuchu-shi, Tokyo, 183-8534 Japan [email protected]
Abstract. In Japan, there has been some argument that many academic programs today are often inadequate from the viewpoint of practical education. However, it is difficult for universities to secure enough human resources to satisfy their students’ needs with sufficient service. TUFS, or Tokyo University of Foreign Studies, also faces the same problem as other universities do in Japan. One proposed idea to help solve this problem is to exploit the collective intelligence of alumni who have considerable expertise and experience in the real world. This paper introduces TUFS’s new development of a SNS application on academic education for the purpose of improving its services. A framework of the knowledge management of participants’ collective intelligence is also suggested here. This attempt proposes a general framework of SNS application on practical education of universities. Keywords: SNS, Implicit knowledge, Education.
resource to bridge the gap between the current situation at TUFS and students’ needs. In order to put this idea into action, a project was initiated to make a supportive alumni network using SNS (Social Networking Service applications for the students’ practical needs. The plan, titled “Student overseas support through an e-alumni program”, has been adopted as a MEXT (the Ministry of Education, Culture, Sports, Science and Technology) financially assisted program of 2008 under the category (or heading?), “Support program for contemporary educational needs” [1]. Though the initial purpose of the e-alumni program was to support students studying abroad, this paper is not limited to presenting TUFS’s experience of creating an alumni knowledge network with SNS. The experience can be fully extended to propose a general framework of SNS applications on practical education of universities. 1.1 Background Since TUFS’s degree programs feature diverse language and culture studies, many of its students study abroad for at least a short term. Since there is a wide variety of destinations for the students’ overseas studies, assistance for the students is acutely needed in addition to offering detailed information on daily life in the countries that they choose. However, it is hard to meet the needs with the limited budget and human resources TUFS has. In other words, the university’s organization alone cannot cover all of the needs of its students. In this dilemma, TUFS has started the “Student overseas support through an e-alumni program”. The project aims at organizing TUFS alumni’s collective knowledge through Information and Communication Technology, or more specifically SNS. The system will facilitate active communication among TUFS, its alumni and students, and enable the exchange of necessary, practical information. 1.2 Overview of the Project The e-alumni program can be described as an experiment to help solve some of the issues that TUFS and most other Japanese universities have. TUFS expects SNS to be
Fig. 1. E-alumni project outline
A Proposal for a Framework for an e-Alumni Program Using SNS
211
applicable to the project because of the following reasons. SNS sites have been extremely popular as a means of sharing information and contents on the Internet. The idea of SNS is often associated with one of the most popular websites in Japan called Mixi. Although SNS sites are considered a kind of circle or community, they are thought to evolve into a virtual world for people who can share their expertise with people in similar fields. There are two points that are discussed in the following sections describing our attempts to make this a successful project. Section 2: A framework of SNS settings for an e-alumni program Section 3: A methodology for e-alumni knowledge management
2 A Framework of SNS Settings for an e-Alumni Program In the course of planning the e-alumni program, we found that an effective alumni network has to meet two needs: 1. Form identifiable specialist groups in a virtual world 2. Provide a usable ubiquitous network environment for users to share technical knowledge in their groups These terms can be met technically, but need to give more consideration to facilitate the use of the network. The points are: 1. Organize user groups, paying careful attention to their expertise, and operate the groups successfully 2. Profile users need to contribute to intended educational purposes so that users can accept and send information effectively 2.1 Organizing Users To organize users, an administrator of TUFS will be in charge of forming an SNS community. When alumni wish to make a new community, the administrator must permit and create the requested community. This is to prevent the SNS from forming Table 1. User permissions
permission community forming
invitation community
to
University
alumni users
Students
(under the community guidelines for forming specialists groups)
(partially allowed for the purpose of alumni communications)
(allowed to make a profile for himself )
(under the purpose of committing educational plans for the students)
(partially allowed for the purpose of alumni communications)
(partially allowed for the purpose of user communications)
212
H. Sano
unexpected communities which do not suit the university’s purpose of offering its students practical applications. Once the community is prepared, the applicant will take over its management. By making a closed society shown in Table 1, users’ privacy is protected and therefore it facilitates the entry of alumni with expertise and keeps the use of the SNS secure.
Fig. 2. Administrated community
2.2 User Administration User profile information is divided into two categories, “basic information” and “registered information”. When users log in, their home will be a community selected through the use of the basic information. The registered information allows users to search any expert groups that they wish to access. The profiles are used mainly for three types of purposes as follows: 1. To confirm the identification of the community members by each member 2. To confirm the identification of the alumni by the university 3. To confirm the identification of students by the university The following tables are suggested settings for the profile information and the range of disclosure (Tables 2 and 3). Table 2.
classification
administrator read write
person himself read write
other members read write
1 2
optional
3
optional
A Proposal for a Framework for an e-Alumni Program Using SNS
213
Table 3.
classification 1 2 3
range of disclosure person himself and administrator person himself and MY members All
Fig. 3. Three forms of access to resources
The detailed user profiles allow users to accept and send messages efficiently among special interest groups. Once the network has started its service, the contents that the users create themselves will become part of TUFS’s valuable educational resources. The resources will be accessed in three forms: (1) sharing information in the community, (2) exchanging information between members, and (3) browsing diaries of members mutually. 2.3 Operation of SNS Alumni Network Since the network includes a large amount of personal data, a stringent operation manual will be developed. To secure adequate operation, a server will be placed in the TUFS Information Collaboration Center. A full-time overseas study coordinator will also be posted at the center to be in charge of office matters. TUFS plans to make a test installation of the network in mid-2009.
3 A Methodology for e-Alumni Knowledge Management Along with the SNS network development, it is important to build a methodology for e-alumni knowledge management. Information provided by the alumni has to be effectively stored and has to be ready to be searched through and referenced in order to maximize the use of the expertise. To develop a usable method, two elements have to be considered: 1) to identify the features of the communities, 2) to establish the best way to categorize the knowledge so that it is suitable for the specialized communities.
214
H. Sano
3.1 Features of e-Alumni Students at a specialized college like TUFS receive professional education even when they are undergraduate students. Graduates usually engage in highly-specialized work where they contribute to society in various fields. To take TUFS for instance, the Faculty of Foreign Languages has seven (or six?) courses and 26 majors where language study is the main focus. Its purpose is to train language specialists who will work in the international arena. For example, many graduates are involved in various global intellectual professions such as import-export, global sales and marketing, or supervising an overseas subsidiary with their language expertise. 3.2 Organizing Knowledge Once the characteristics of the communities are defined, the next question is how to compile appropriate databases for the prospective knowledge exchanged on the SNS.
Fig. 4. Knowledge management process
A Proposal for a Framework for an e-Alumni Program Using SNS
215
The goal is to give a proper categorization of the specialized BOK, or Body of Knowledge. In the initial stage of usage of the SNS, the users exchange their knowledge based on the provided communities and concepts as described in Section 2. Database design should be started after some period of monitoring and storing the knowledge of the users. Since any gathered body of knowledge will exhibit some tendencies, appropriate categorization should be attempted after monitoring the stored data thoroughly by university professors. After the categorization is carried out, accordant indexes are released to the users through the top of the blog menu. Then the users will be able to follow the indexes when they post new information. The users can give the university feedback on the usability of the indexes, and the university will reform them accordingly. Through the continuous repetition of this process, the indexes will become effective enough in a few years. Actually, these indexes themselves will be a source of valuable knowledge that the university should use.
4 Summary of the Process in Designing the Framework for the e-Alumni Project As described, this project suggests the general framework of a SNS to be used for a university alumni network. The process is summarized here again, so that the concept will be easily understood and reproducible by other universities. 1. A university forms specialized communities that connect alumni and current students following and pursuing its educational policies and goals. 2. The users exchange information through SNS features such as chat, blogging, or Q&A. 3. The university monitors and stores the knowledge for a certain period of time, and its professors categorize the data with appropriate specialized indexes. By using constant feedback and maintenance of the categorization, the indexes will gradually become more sophisticated as time advances. These indexes and the indexing process themselves will serve as unique and valuable knowledge for the university. The indexing itself forms one part of the knowledge management system that the university should value. TUFS will execute this project this year and will experience the creative process involved in this knowledge management system. Then we can expect a specific report of the results that will reveal just how effective our proposed database will be.
5 Conclusion This paper proposed a framework of an alumni network through SNS technologies as a means of practical education. TUFS will inspect the framework thoroughly to prove its efficiency through its actual effort in supporting students with the e-alumni program. With more elaborate observation and development of the framework, it is hoped that practical applications in university education will be improved greatly.
216
H. Sano
References 1. Application for 2008 Ministry of Education, Culture, Sports, Science and Technology (MEXT) grant, Support Program for Contemporary Educational Needs ‘Student overseas support through an e-alumni Program’ application form, Tokyo University of Foreign Studies (2008) (in Japanese) 2. The Central Council for Education, daigaku bunkakai seido kyoiku department, gakushi katei kyouiku no kouchiku ni mukete (2008) (in Japanese), http://www.meti.go.jp/press/20080627007/20080627007-4.pdf 3. The Ministry of Economy, Trade and Industry, syakaijin kisoryoku ni kansuru kennkyuukai interim report (2008) (in Japanese), http://www.meti.go.jp/press/20060208001/20060208001.html
Supporting End-User Development of Personalized Mobile Learning Tools Marco de Sá and Luís Carriço LaSIGE & University of Lisbon Campo-Grande, 1749-016, Lisboa, Portugal {marcosa,lmc}@di.fc.ul.pt
Abstract. Mobile devices present great features for the support of pervasive learning and content personalization. This paper presents a framework which takes advantage of these features and supports end-users while creating their customized tools for their students. Additionally, the framework comprises means for teachers to include persuasive and motivational mechanisms and hints, promoting student engagement while pursuing their learning activities. We describe the framework’s architecture, its features, including the supporting guidelines and development process, and detail some of the already developed material and the results that emerged during initial trials and case studies, also stressing their contributions to the field of m-learning. Keywords: Mobile devices, personalization, multimedia content.
to students on a dynamic manner, reacting and behaving according to their needs. Globally, despite the great benefits that some of these experiences demonstrated [10,11], there is still the need to provide personalized content and, most importantly, personalized attention and access to the educational material. This paper describes a development framework which supports teachers and educators while creating and defining personalized educational artifacts and tools (e.g., books, tutorials, questionnaires, tests, videos) which can be accessed by their students through personal mobile devices. It aims at tackling the abovementioned shortcomings of existing software by providing means for developers (e.g., teachers) and users (e.g., students) to interact with the same tools but achieving different goals. In general, the framework encompasses a set of modules that support the creation and utilization of different learning tools which range from simple homework forms, to elaborate tests, annotations and multimedia books. The framework also offers the possibility of exchanging information and supports collaboration between teachers and students, including analysis tools and synchronization mechanisms. All the tools take into account users’, devices’ and content requirements, allowing teachers to adjust the final user interface to various dimensions, including content and domain, accessibility features and multimodal capabilities. We address the framework’s various functionalities and features, stressing its architecture and extensibility choices also focusing the inclusion of multimodalities to support universal access to the learning material and to the inclusion of behavior in order to extend the teacher’s presence outside classes. The goal is to arrange persuasive user interfaces that compel and request students to engage in certain endeavors throughout their daily lives or whenever accomplishing their educational tasks. We explain the requirements that emerged throughout the design stage and present the solutions that were created in order to overcome existing challenges. Afterwards, we detail the framework’s functionalities that pertain directly to accessibility, adaptation and persuasion, addressing the issues that are focused and the support that is provided while end-users develop their own material and tools. Finally, as a validation we present two case studies where teachers defined their own content and created the adequate containers, including presentation and interaction dimensions, to support a better learning process for their students.
2 Framework’s Architecture The end-user development framework is divided into several tools, available for two main platforms. The first and main tool is the interface/artifact builder. It allows endusers/power-users (e.g., teachers or tutors) to design and create the educational artifacts, selecting content, arranging the user interface layout and personalizing the overall artifact to several dimensions (e.g., student’s needs, location, content). This tool is available for desktop computers. As a result from the utilization of this tool, and as depicted in figure 1, XML files containing the artifact’s specifications are created and can be later interacted with on the runtime environment. The runtime environment is the tool that allows students to engage on their learning activities while using mobile devices. Globally, it reconstructs the specification of the artifact, contained within the XML file and materializes it into an interactive
Supporting End-User Development of Personalized Mobile Learning Tools
219
Fig. 1. Overall framework architecture
application which contains the selected content (e.g., images, textual questions, videos, e-books) and allows students to browse the content, respond to questions, take annotations and transfer data to other students as required. Currently, there are runtime environments for Windows Mobile Devices and PalmOS devices. A Windows version was also developed in order to allow students to use their learning material on TabletPCs or laptops. During the utilization of the artifacts on the runtime environment, logs and results are generated, containing information about usage patterns, the student’s selections and answers [3]. These can be exported to a desktop computer and analyzed on the framework’s last tool, the result analyzer. The result analyzer includes means to review logs through a video like mode or to browse results screen by screen or element by element. On a lower level, the framework is divided into several components which support standardization of formats, reutilization of software elements and, most importantly, the framework’s scalability. As shown in figure 2, User Interface guidelines and artifact templates feed the developer tool allowing end-users to utilize them and compose the educational artifacts accordingly. In addition, different element libraries, created for specific contexts/domains (e.g., drawing elements for arts classes; pictorial books for small children; multiple-choice questions for homework) can be used. These are used during development and can be easily arranged and dragged into the educational artifact and are later used on the actual artifacts. Each artifact is composed by a set of screens and each screen is composed by a set of elements (e.g., question and image; video and audio clip). The UI design guidelines and artifact templates can be added to the developing tool as XML files. New element libraries need to be programmatically created but can also be imported into the developing environment.
220
M. de Sá and L. Carriço
Fig. 2. Component Architecture
3 Development Support Given the highly dynamic content, structures and domains that the developed framework targets; its main contribution over previous work is the ability that it provides, for end-users, to create their own educational artifacts. Besides allowing teachers/educators to customize content and the way it is presented and used by students, it also copes with the need to adjust the artifacts whenever required without any intervention by software programmers or designers. However, taking into account that most teachers/tutors do not have any design background or programming skills, the design and development of these artifacts has to be supported, wizard-based and must not require previous programming/UI design knowledge. Accordingly, the developing environment is composed by a wizard-based interface which allows teachers to create screens sequentially, dragging and dropping the selected elements that compose the screen. Figure 3 shows the overall layout for the artifact developing tool. On the top, teachers can select items and elements from the available pool and libraries. Beneath the selection pane, there are the two configuration options. The first allows teachers to configure the artifact’s and elements’ presentation while the second displays the various interaction possibilities. On the right side of the tool, an automatic preview of the resulting screen is presented along with the artifact’s main data. In addition to the various configuration options that pertain to interaction, presentation and content, the developing tool is also able to enforce usability guidelines, which ensure that the resulting artifact will be usable by the end-users. To achieve so, it loads the XML file with the design guidelines as previously detailed. Usability guidelines that pertain directly to the used device and its characteristics (e.g., screen resolution; touch screen or physical keyboard) can also be used (Figure 4).
Supporting End-User Development of Personalized Mobile Learning Tools
221
Fig. 3. Artifact development wizard. Configuration options and available elements on the left and screen preview on the right.
The guidelines automatically arrange screen elements according to specific values that can be configured and updated according to the targeted domain. Currently integrated guidelines allow the automatic docking and adjustment of elements to screen sizes and layouts. Moreover, the amount of elements per screen, as well as the amount of content per element (e.g., number of sentences within a label, size of an image, number of options within a list) can be limited according to various aspects (e.g., screen resolution, orientation). As a complement for these guidelines, when available, the system is also able to provide suggestions and alternative approaches to the current design. For instance, combo-boxes can be automatically replaced by radio buttons whenever options are longer than 25 characters or vice-versa when the list contains more than 5 items.
Fig. 4. Artifact development wizard. Device related usability guidelines.
222
M. de Sá and L. Carriço
Overall, the entire design and artifact construction process is straightforward and supported by the design guidelines and offered features which help end-users while creating and personalizing the educational material.
4 Accessibility and Persuasion To provide access to different students with different dis/abilities, the framework includes elements that allow teachers to create accessible artifacts. Besides the various configurations and common interaction options that can be used to interact (e.g., text input, multiple-choices) or to visualize the content (e.g., images, video), multimodal elements are also available. These include text-to-speech features which recreate the content through audio output. For instance, a question that was provided through a text-label element can be automatically recreated as an audio file that can be listened to instead of read. Other type of content can also be transformed into audio. For instance, images can be tagged with a description which will be transformed into audio or can contain an audio-only description directly recorded by the developer/teacher. The same principle applies to all other elements that can be used to create the artifact. Gesture-based navigation was also included for situations where eyes-free interaction is necessary. Furthermore, to facilitate eyes and hands-free interaction, the main basic navigation options (e.g., next, previous, play, pause, stop) can also be issued by voice, allowing the artifact to be utilized by, for instance, visually impaired users. Additionally, as a complement to the accessibility features, and in order to create proactive digital learning documents/artifacts that offer hints and aids to users, manuals that omit or show new information according to the user's performance while completing it, and overall motivate students whenever needed, the design framework also includes the customization of the artifacts' behavior. Behavior can be configured through the definition of rules. Rules are composed by time, interaction or content triggers and a consequent action. They can be attached to a specific element or to the entire artifact. On the former, depending on the interaction or usage within a specific element, a certain action can take place. On the latter, the interaction within various elements of the artifact (e.g., sequence of navigation or time to browse through various elements) defines the entire artifact's behavior. Time-based triggers can be configured to prompt warnings or change elements according to the time the user is taking to review/complete them. They can also be used to define alarms that alert users to use or complete an artifact at a given time. Interaction triggers analyze the user's interaction with the device by counting clicks or detecting where on the screen the user has interacted. Content based triggers activate actions depending on the content of the elements. For example, when the user chooses an option from a list or a value within a defined threshold, a certain action can be triggered (e.g., selecting "YES" displays a warning). In concert with the rules that are defined for each item or artifact, four different types of actions can be selected. The first one prompts a message that is composed by the user while the second jumps from the current screen or element to another within the artifact (e.g., first screen, end of the artifact, drawing, previous element). The two remaining actions hide or show elements (e.g., if a user checked a radio button a textbox is shown).
Supporting End-User Development of Personalized Mobile Learning Tools
223
5 Case Study and Examples The design of the developing framework and the experimentation of the resulting artifacts followed a user centered design approach. Accordingly, throughout the entire process, end-users were involved and participated by experimenting and providing their opinions on both the features and usability of the resulting tools. However, to fully validate its potential and features, the framework was also used to create educational artifacts for young students, for a variety of subjects. The main goal was to provide students with evaluation tests that could be answered on personal mobile devices (e.g., PocketPCs), regardless of their whereabouts. The framework was provided to two teachers in order to create the evaluation tests. Tests included several questions with different input and output elements (figure 5) and were distributed to a group of students. Two different sessions took place. One took place inside the students’ class room and the second outside the class-room, simulating a mobile scenario. All the results were gathered after the class and later reviewed by the teachers. Initial tests, composed by 20 screens/pages each, with two elements per page (e.g., one for output and one for input) took roughly one hour to create. On the second iteration (the tests that were completed outside the class), teachers composed the artifacts with video and audio elements and took between 20 and 40 minutes (figure 5, left). Students were given one hour to complete each test. Log revision showed that most students completed the tests in less than 30 minutes and revised their questions an average number of two to three times before locking their results. On a qualitative analysis, students responded to questionnaires and results showed that they appreciated the fact of conveying within one device their annotations, exams and manuals provided by the teachers. Hints, the varied pictorial content and the visual appeal were also very well received. The ability to respond to their questions through different modalities and access different content through different means was also one of the most highlighted benefits.
Fig. 5. Artifacts on the runtime environment - created with the development environment
224
M. de Sá and L. Carriço
For teachers the major contributions were the ability to configure artifacts according to various dimensions, adjusting them to the students’ needs, subject and content. Additionally, the ground breaking analysis that the log player features provided for class and homework and the inclusion of the persuasion/motivation mechanisms were deemed very important and useful.
6 Conclusions and Future Work The advances within mobile technology allow their utilization on a daily basis for a wide set of activities. As a result, they are now strongly integrated within the lives of a large set of the population and are especially appreciated and used with ease by the young adults, teenagers and children. Moreover, as they features and processing capabilities are further extended, they are becoming multimedia and multimodal devices which offer better support to highly interactive and demanding tasks. Educational activities and the m-learning concept are gradually making use of mobile devices and capitalizing these potentialities to new levels, enhancing both learning and teaching processes. However, the absence of personalized tools which simultaneously cope with the large variety of domains and subjects that are currently taught, added with the limited persuasion and motivational features, still hinder the widespread of m-learning tools. In this paper we presented a tool which supports end-user development, endowing teachers and tutors with the ability to create their own m-learning material. Additionally, the framework includes features which allow users to overcome previous limitations by personalizing content, taking into account accessibility concerns and introducing motivational and persuasive mechanisms to maintain student engagement even while away from classes and their teachers. To facilitate this process, the tool integrates design guidelines and offers a guided development process which, through the case studies has proved to be easy to utilize. Initial results have demonstrated the framework’s potentialities regarding customization and personalization and have inspired teachers to create multimedia and highly interactive educational artifacts that promote cooperation between students. For students, initial trials have also shown promise, especially the ability to receive aid and help from their teachers through a deferred modality and while away from classes. Current and future work is directed toward the definition of new element libraries and the inclusion of new interaction mechanisms (e.g., gesture recognition) and the integration of the mobile device artifacts with large displays, commonly used during classes. The enhancement of the developing and of the analysis tools is also envisioned. In particular, a mobile version of each tool is being designed, allowing the on-the-spot analysis of student’s results and the adjustment of the artifacts on-the-fly, which will be useful for mobile scenarios such as field trips to museums or other locations.
Acknowledgements This work was supported by LaSIGE and FCT, through the Multiannual Funding Programme and individual scholarship SFRH/BD/28165/2006.
Supporting End-User Development of Personalized Mobile Learning Tools
225
References 1. Virvou, M., Alepis, E.: Mobile educational features in authoring tools for personalised tutoring. Computers & Education 44, 53–68 (2005) 2. Sá, M., Carriço, L.: Handheld devices for cooperative educational activities. In: Proceedings of SAC 2006, Dijon, France (2006) 3. Sá, M., Carriço, L.: Detecting Learning Difficulties on Ubiquitous Scenarios. HumanComputer Interaction. In: Proceedings of HCI Applications and Services, 12th International Conference, HCI International 2007, Part IV, Beijing, China, July 22-27, 2007, pp. 235–244 (2007) 4. Theng, et al.: Mobile G-Portal Supporting Collaborative Sharing and Learning in Geography Fieldwork: An Empirical Study. In: Proceedings of JCDL 2007, Vancouver, Canada, pp. 462–471. ACM, New York (2007) 5. Costabile, et al.: Explore! Possibilities and Challenges of Mobile Learning. In: Proceedings of CHI 2008, Florence, Italy, pp. 145–154. ACM, New York (2008) 6. Sánchez, et al.: Mobile Science Learning for the Blind. In: Extended Abstracts, CHI 2008, Florence, Italy, pp. 3201–3206. ACM, New York (2008) 7. Lindquist, D., et al.: Exploring the Potential of Mobile Phones for Active Learning in the Classroom. In: Proceedings of SIGCSE 2007, Kentucky, USA, pp. 384–388. ACM, New York (2007) 8. Pownell, D., Bailey, G.D.: The next small thing – handheld computing for educational leaders. In: Learning and Leading with Technology, vol. 27(8), International Society for Technology in Education (2000) 9. Kortenkamp, U., Materlik, D.: Geometry teaching in wireless classroom environments using Java and J2ME. Science of Computer Programming 53 (2004) 10. Inkpen, K.M.: Designing handheld technologies for kids. Personal technologies 3, 81–89 (1999) 11. Alford, K.L., Ruocco, A.: Integrating Personal Digital Assistants (PDAs) in a Computer Science Curriculum. Frontiers in Education, Reno (October 2001)
Didactic Models as Design Representations Chris Stary University of Linz, Department for Business Information Systems, Austria [email protected]
Abstract. The contribution focuses on the role of didactic knowledge when designing interactive e-learning environments. Several representational approaches for the preparation of domain content and learning support have been developed. However, for the context-sensitive design of interactive artifacts not only the representation of particular aspects of learning is essential, but rather the propagation of didactic knowledge to functional services and interaction facilities. Such an endeavor requires the explicit representation of relationships between structure and behavior elements. Model-driven design supports the distinctive representation of multiple perspectives while allowing the mutually tuned refinement of design elements. In this paper a model-based approach for self-organized e-learning is presented. It supports the design of learnercentered knowledge acquisition by specifying user roles and learning tasks. We discuss the required enrichments of traditional model-based design approaches, due to the consistent tuning of high-level design elements, and the coherent propagation of task and user information to interaction services. Keywords: model-based design, e-learning, learning management, coherence, consistency, integrated specification.
Although several techniques for guiding the preparation and representation of content and transfer of knowledge have been developed, the benefits of model-based design have barely been explored. However, for context-sensitive design both, the representation and propagation of didactic knowledge to functional services, and the intertwining of model elements and relations, such as between the user and the domain model, are essential. They are discussed revisiting existing model-based approaches and reflecting recent e-learning platform developments.
2 Constructivist e-Learning: The Scholion Project Constructivist approaches to e-learning provide environments to explore information and guide learners actively to build individual mental processes that occur during the construction of mental representations. Active (re-)construction is seen as particularly beneficial for learners as they can pursue their individual interests, while they are motivated to communicate their understanding to others. As we know from studies in constructionism, the situated and public nature of any construction activity is identified as important [3]. Both, the individualized content, and the social aspect of learning processes have to be tackled by learning management. Such an understanding of e-learning is grounded in mathetics, a special science of (scientific) teaching centered on the operational processes of the learning, and dealing in particular with the principal aspects of conceptualization and cognitive organization. Didactics, defined as the strict scientific core of pedagogy, are thought of as inconceivable from any viewpoint other than a mathetic one [13]. Computer technologies for learning have opened up new avenues for designing content. In the sense of mathetics they can trigger active learner participation along knowledge provision and acquisition processes. To meet the requirements for computer-mediated context-sensitive and collaborative learning, it must be possible for learners to explore different categories of information in virtual environments and to communicate, so that meaningful learning of a domain can proceed in tandem with establishing communities of practice. Still, the ultimate goal is to create personally meaningful mental representations [6]. The significance of being able to treat learning as a socially valid exploratory activity, rather than a linear, pre-planned activity in detail, is recognized by coaches and developers step-by-step. One appreciation can be gained through looking at deeper issues than domain-specific structures of knowledge, web-design of user interfaces, or domain-specific methods. It addresses context from different perspectives: (i) the mathetic knowledge that drives the provision and acquisition of knowledge – developers have to look for a proper preparation process, (ii) communication channels utilized for learning and guidance – developers have to look for links of communication entries to content items, and (iii) information beyond the core of domain content, such as cultural issues like ethno-computing in computer science education – developers have to look for additional information to facilitate comprehensive understanding a topic. Our research so far has not only been targeting at mathetically effective content development, but also towards context-rich guidance and situation-sensitive learning ([12], scholion.ce.jku.at, www.blikk.it).
228
C. Stary
The frame of reference developed in the Scholion project initially requires the didactically relevant decomposition of learning material into so-called blocks, such as ‘definition’. They serve as focal point in the learning process and might be encoded into different media (text streams, images, videos etc.). Hence, multiple (re)presentations of content (polymorph content) may exist. Hyperlinks between blocks and media are likely, once linear learning material (information) is decomposed and de-linearized. In addition, different levels of detail (LoD) for each block can be specified, e.g., providing a slide for a definition on the top level (LoD 1) based on the full text of the definition on LoD 2, e.g., representing a textbook. Annotations constitute individual views on content items by commenting, linking, or highlighting items, block elements, or enriching content blocks. Some of those annotations can be links to communication entries of the Scholion communication components (chat, forum, infoboard etc.). In this way communication elements are directly linked to content blocks and vice versa. Communication can be established among peers for learning, as well as between learners and coaches. The latter, as quality managers, are responsible for improving content and structures based on learner input and feedback. In Scholion the content is arranged according to the aforementioned didactically relevant information types, conforming to current e-learning standards (cf. www.imsproject.org). They represent leaves in the hierarchy courseÆ moduleÆ learning unit. Currently, 15 generic types of this sort are available as part of an XML schema. They comprise definition, motivation, background, directive, example, self test, and other domain-independent content structures. Domain-specific block types can be added to support dedicated applications, such as proof when handling content from mathematics. Each block type can be visualized in Scholion through a certain layout, e.g., green background for proof. Block types allow learners and coaches to browse content along specific categories using a filter function. The workspace then contains only filtered block types. In this way learners might follow their interests and habits, such as starting to learn with studying background information. In order to establish self-organized learning processes Intelligibility Catchers (ICs) are offered to learners [11]. ICs are assignments made available through the bulletin board in Scholion. They have been designed to foster in-depth understanding of a topic. Their orientation section addresses the stage of capacity building when the IC should be used and what learners can expect when accomplishing the IC tasks. This section should motivate learners. It can be compared to an advanced organizer. The objectives set the scope in terms of the topics that are addressed and the understanding that should result from exploring and processing the topic information. It reflects the didactic value of the concerned learning unit(s), and serves as a means of orientation for learners. The task section contains a list of tasks to be achieved. It comprises a documented work and an intellectual work part. The documented work lays ground for the tasks to be represented in task-based model representations (see next section). It encourages active information search and processing as well as focused communication. The conference section sets the rules for establishing a corresponding community of practice, in particular when meetings or interactions should occur. The reference section provides links to material that helps to accomplish the tasks. The bulletins can be dynamically created, and are available in the Scholion information
Didactic Models as Design Representations
229
board. They are intended to facilitate the completion of the assignment. Finally, the departmental cuts reveal the estimated individual effort to meet the objectives. It might be expressed in terms of credit points or the estimated IC completion time. The structure combines organizational with subject-specific information. The information is arranged from a mathetically relevant perspective. For instance, the orientation section in the beginning informs coaches and learners when to use this IC, addressing competencies, the content involved, the rationale for choosing this content and for co-constructing mental representations. Initially, the learners are encouraged to identify those blocks of learning units where information is already available, i.e. part of the prepared content. In case of an IC for learning UML this can be design class diagrams. Then the learners are asked to enrich particular content items, e.g., UML class diagrams with task-specific information stemming from a case study of their choice. Typically after some practical work, such as modeling in UML, all results are shared with peers. In Scholion it is enabled by dedicated views on the original content. Views are like transparent slides put on top of the prepared content. They contain all content enrichments, such as comments, individual examples, and various types of links. As such, the content items are directly linked to discussion items according to the learning task, e.g., to develop an understanding about the proper use of UML class diagrams. All results can be validated by the coach through feedback, in order to ensure correct learner representations. The annotation facility of Scholion is considered as the key to individualization. It is based on a flexible hypermedia scheme for the representation of content elements. It enables learners to (i) mark a specific position in a content element for learning, (ii) post questions, answers or comments, and (iii) additionally link the contribution to a discussion theme from the system’s discussion board when working with content. The latter link may guide the user to the adjacent discussions of content. In case of realtime online connections, e.g., chats, the questions and answers can pop up immediately on the displays of all connected users (available in a buddy list). The referenced content elements can be displayed at the same time. In the discussion board topics of discussions can be created either manually by users or triggered by coaches posting a learning input. For the sake of traceability, each discussion contains a vector of contributions, being part of a certain discussion group, and ordered by relevant themes.
3 Model-Based Design Representations In model-based development the identification of models has been dominated by a top-down approach to the design and implementation of user interfaces[8]: The task and domain (object) model is located at the top layer of abstraction, as this model is expected to capture all activities that need to be performed in order to reach user goals. They might be detailed through properties (attributes), and arranged along a hierarchy representing causal or temporal mutual relationships. In addition, this model is expected to capture (business) objects that have to be manipulated in order to perform tasks. The abstract user interface model is located below the top layer, as it contains the logical structure required for task accomplishment at the user interface.
230
C. Stary
Abstract elements of interaction styles are captured in their mutual context, e.g., an area for data manipulation in a browser-like way. The relationships to user tasks or goals are encoded, e.g., to indicate that at a certain point in time options for navigating in a task hierarchy have to be offered. The final codality of information for presentation (text, graphics a.t.l.) and user control (modality) are not specified at that layer of abstraction. The concrete user interface model captures the refinements of all abstract interaction elements to more concrete ones. The initial abstract specification of style elements is replaced with one containing concrete attributes and relationships between interaction elements, including the modality of interaction. At that point, platform- and media-specific information is attached, in order to enable physical access to an application. The instantiation of the concrete model leads to an actual user interface, utilizing (standard) programming technologies, such as Java. The envisioned interplay of the models in the course of task-based design can be exemplified as follows: Consider booking a course at the university. It requires several sub tasks, such as finding proper options, selecting class data a.t.l.. This information is kept on the task (model) level, in addition to information for selection and manipulation, such as course. The subsequent abstract user-interface model captures all (categories of) interaction elements that are supposed to support each subtask. For instance, to identify course options an entry field for search has to be provided. Finally, the concrete user-interface model captures all specific interaction elements. For instance¸ in a web interface, search entry fields are supported by text bars after selecting a search engine (and site). The actual user interface can be constructed based on the specification of the concrete user interface model. Such a top-down approach focusing on layered design perspectives facilitates the use of multiple platforms and various arrangements of interaction elements. It encodes rather than makes transparent Gestalt design knowledge - knowledge that is mainly created in the course of mapping the results of user and work analysis (located in the top level task/domain model) to abstract interaction elements. The initially acquired (and represented) context of an interactive application does not directly guide concrete user-interface designs. A contextualized process throughout development requires a shift in model-based design (support), namely (i) a more detailed and flexible representation of users, work tasks, and domain elements, at least to the same extent as user-interface representations, and (ii) explicit conceptual relationships among design elements (inter- and intra-model-specific), considering universal access from a usability and softwareengineering perspective in a more integrative but self-contained way. Such an approach has to take a more networked rather than hierarchic perspective on model-based development. It requires a set of models that still enable the specification of an application model (in the sense sketched above), but details the user- and use-relevant elements, e.g., in terms of a task, user, domain, and an interaction model. The traditional engineering perspective has to be enriched with design variants capturing interactivity before considering functional implementation. The following set of models lays ground for model-based tools that recognize the need for supporting diverse user communities and situations of use [10]. The task model contains details of the organization at hand, the tasks, and the users’ perception of work. Task modeling includes modeling of the objectives
Didactic Models as Design Representations
231
users want or have to meet using an interactive application, probably in response to particular situations or events, as well as the different activities users have to perform to accomplish their tasks, as, e.g., given by global business processes. The user model contains a role model by reflecting specific views on tasks and data (according to the functional roles of users in organizations). It also captures individual user characteristics, capabilities or specific needs that developers need to take into account. The domain (data) model addresses all data, material, and resources required for task accomplishment in the application domain. The interaction model is composed of device and style specifications that designers need to construct a user interface or to generate a user-interface prototype. Hence, the structure of modalities and its related behavior are captured in that model, both at an abstract, and concrete level. For the developers’ convenience platform-specific specifications can become part of the interaction model. In the case where behavior specifications are missing, the preprogrammed, platform-inherent behavior might restrict the design space. The adaptation model contains all rules that trigger a particular structure and/or behavior of an interactive application. As such it enables interaction adapted to usermodel elements, such as left-hand assignments to interaction media (e.g., mouse buttons) based on consistent representations. It also enables the multiple presentation of task-information for navigation, e.g., tree views and acoustic trees. Since its scope includes all the previously listed models and their relationship, adaptation may concern task, data, interaction elements, and their inter-relationship.
Fig. 1. Model-based specification for e-learning tasks in ProcessLens
232
C. Stary
The application model integrates all five models from a static and dynamic perspective. In this way, it defines the entire structure and behavior of an interactive application. For interaction the users’ privileges, preferences, special needs, and tasks to be performed are taken into account when mapped to a concrete device or widgets of the interaction model. The ProcessLens approach [2] also expands the model-based design space, providing a design ontology for task-driven design. It can be used for task-based e-learning design as shown. Figure 2 demonstrates the enrichment to traditional design. It leads to a context-sensitive specification based on the mentioned models. For learning management the task model has to contain the tasks of the documented work-part of the IC for learners (see section 2) and the preparation of the IC by the coach. ‘Capture UML’ is decomposed into ‘IC-Specification’, ‘View Management’ and ‘Discuss’. ProcessLens also shows the role-specific responsibilities through a role-based user model. The coach ‘handles’ the preparation and the view management, whereas the learners handles views and forum entries in the course of discussion (see subtasks of ‘Discuss’ in Figure 2).
Fig. 2. Part of e-learning task hierarchy and roles in ProcessLens
A typical example of a design relationship is ‘handles’ between roles and activities. It binds responsibilities and allows for role-specific interaction domains. ‘Before’ demonstrates a causal relationship that has to be transformed to a set of temporal constraints in the behavior specification. A behavior specification is exemplified in Figure 3. It shows a coherent propagation to problem domain elements for being invited to a discussion. In the course of ‘Discuss’ peers have to be invited to join a certain community of practice. It is done by e-mail which requires to handle e-mails from the task and the problem-domain perspective. As content and communication are of equal importance in e-learning, communication facilities have to be represented in the problem domain model besides the content (decomposed to block types).
Didactic Models as Design Representations
233
Fig. 3. Model-based behavior coupling in ProcessLens
The interaction model is also decomposed into a structure and behavior part. The latter contains the life cycle of each widget and interaction style as required for device types. For instance, Scholion instances can also be used on web mobiles (see www.mobilearn.at). In ProcessLens the abstract and concrete layer of description is captured within a single model. The coupling of behavior sequences from other models is done analogously to the synchronization shown in Figure 3. Dedicated synchronization relationships have been developed to ensure behavioral consistency of design specifications. In order to implement a user interface for learners, relations have to be set between the task model and interaction model, as well as between the problem domain and the interaction model. The first link is required to navigate according to the tasks of the IC-documented work part. The latter is required to annotate content, and to communicate on the basis of linking content elements to communication entries. For interaction modeling besides generic UML structures (as used in ProcessLens), e.g., for modalities [7], XML derivates, such as USiXML [4] are widely used. They facilitate transformations as well as the dynamic creation of models, relying on various models. For instance, XIML [9] as user-interface description language supports the specification of various (linked) interaction models simultaneously. Its definition allows to capture task, user, presentation and dialog models as well as platform details. However, there is no executable relationship between the various interaction model parts, as provided by ProcessLens.
234
C. Stary
4 Conclusions As the pedagogical value of conveyed content and the related interaction features for learning support has become a distinctive factor of e-learning environments, design representations have moved to the center of interest in development. They have to capture the structure and dynamic context of subject items. Model-based design techniques not only allow different perspectives to be addressed, they also allow for capture of mutual relations between design models, both from the structure and the behavior point of view. In this paper we have proposed a model-based frame of reference and a corresponding tool supporting the development of e-learning applications. It focuses on the relationships between model elements to ensure consistent tuning and coherent propagation of design knowledge for self-managed learning processes. Of major importance are learning tasks to be accomplished by learners, as they require dynamic handling of domain content and communication elements at the user interface. Although the proposed adaptation model provides design constructs to represent this system dynamics, there is still research to be performed to achieve algorithmic support for automated model transformation.
References 1. Calvary, G., Coutaz, J., Dâassi, O., Balme, L., Demeure, A.: Towards a new generation of widgets for supporting software plasticity: The ”Comet”. In: Bastide, R., Palanque, P., Roth, J. (eds.) DSV-IS 2004 and EHCI 2004. LNCS, vol. 3425, pp. 306–324. Springer, Heidelberg (2005) 2. Dittmar, A., Forbrig, P., Heftberger, S., Stary, C.: Tool Support for Task Modelling – A ‘Constructive’ Exploration. In: Bastide, R., Palanque, P., Roth, J. (eds.) DSV-IS 2004 and EHCI 2004. LNCS, vol. 3425, pp. 59–74. Springer, Heidelberg (2005) 3. Farmer, R.A., Hughes, B.: A Situated Learning Perspective on Learning Object Design. In: Proc. ICALT 2005. IEEE, Los Alamitos (2005) 4. Limbourg, Q., Vanderdonckt, J.: Addressing the Mapping Problem in User Interface Design with UsiXML. In: Proceedings TAMODIA 2004, pp. 155–163. ACM, New York (2004) 5. Meder, N.: Didaktische Ontologien. In: Ohly, G.R.H.P., Siegel, A., Ergon, W. (eds.) Globalisierung und Wissensorganisation. Neue Aspekte für Wissen, Wissenschaft und Informationssysteme, vol. 6, pp. 401–406 (2000) 6. Norman, D.A., Spohrer, J.C.: Learner-Centered Education. Communications of the ACM 39(4), 24–27 (1996) 7. Obrenovic, Z., Starcevic, D.: Modeling Multimodal Human-Computer Interaction. IEEE Computer 37(9), 65–72 (2004) 8. Paternò, F.: Model-Based Design and Evaluation of Interactive Applications. Springer, Berlin (1999) 9. Puerta, A., Eisenstein, J.: XIML: A Common Representation for Interaction Data. In: Proceedings IUI 2002 International Conference on Intelligent User Interfaces. ACM Press, New York (2002)
Didactic Models as Design Representations
235
10. Stary, C.: TADEUS: Seamless Development of Task-Based and User-Oriented Interfaces. IEEE Transactions on Systems, Man, and Cybernetics, Part A 30(5), 509–525 (2000) 11. Stary, C.: Intelligibility Catchers for Self-Managed Knowledge Transfer. In: ICALT 2007, 7th International Conference on Advanced Learning Technologies, pp. 517–522. IEEE, Los Alamitos (2007) 12. Stary, C.: Sustainable e-Learning Communities. In: Akoumianakis, D. (ed.) Virtual Community Practices and Social Interactive Media: Technology Lifecycle and Workflow Analysis, IGI Global, Hershey, pp. 398–411 (2009) 13. Verpoorten, D., Pumay, M., Ledercq, C.: The Eight Learning Events Model: A Pedagogic Conceptual Tool Supporting Diversification of Learning Methods. Interactive Learning Environments 15(2), 151–160 (2007)
Interactive Learning Panels Ricardo Tesoriero, Habib Fardoun, José Gallud, María Lozano, and Victor Penichet LoUISE Research Group – Computer Science Research Institute (I3A), Campus Universitario S/N, Castilla-La Mancha University, 02071 Albacete, Spain {ricardo,habib,jgallud,mlozano,vpenichet}@dsi.uclm.es
Abstract. New sensing technologies, as RFID readers, are being incorporated into mobile devices to provide users with new interaction experiences. And, the combinations of these new technologies open up new challenging application scenarios. One of the areas that could exploit this potential is the design of interactive solutions for context-aware applications applied to learning environments. This article describes an m-learning environment that is enriched with new interaction features that are, or may, be provided by actual or future mobile device technologies. The proposed gesture based interface allows users to relate ideas and concepts through the improvement of traditional methods. This environment is based on the reuse of existent physical resources, such as the learning panels used in school classes. For instance maps, historical posters, timelines, and so on. These panels are improved with the low cost and widely used RFID technology that enables students to interact with them through mobile devices, encouraging the interest the students applying the constructivist education theory. Keywords: HCI, RFID technology, mobile devices, context awareness, collaborative environment, m-learning, social software.
New developments in mobile phone technologies are not only offering rich multimedia experiences to the users, but the possibility to implement new applications that exploits the physical environment of the users; for instance, GPS, accelerometers and RFID technologies among many other. The application of mobile technologies to the learning process is basically named mobile learning. So, from a pedagogical perspective, mobile learning provides a whole new dimension to support either traditional or novel educational processes. Some characteristics of this type of applications described in [1] include: the urgency of learning need, the initiative of knowledge acquisition, the mobility of learning setting, the interactivity of the learning process, the “Situatedness” of instructional activities and the integration of instructional content. These characteristics make mobile learning quite different compared to traditional classroom learning environment, where all the educational activities are carried out at a designated time and place. Although the use of laptops extends the range of education to places where wired connections are available; mobile technologies go beyond wires and take education possibilities to places where wireless connection is feasible. Another issue to take into account is the adaptation of mobile technology to contextual life-long learning, which is defined as the knowledge and skills people need to prosper throughout their lifetime. These activities are not confined to scheduled times and places as traditional education requires that are so difficult to achieve when people finishes formal learning. Thus, mobile technologies become a powerful tool to support contextual life-long learning, by being highly portable, individual, unobtrusive, adaptable to the context of learning, the learner’s evolving skills and knowledge [2]. Our proposal can be seen as an interaction improvement to m-learning systems inheriting m-learning advantages, as long-life learning is. So, mobile learning frees users from being anchored to a specific space, providing the possibility to explore the environment and interact with the world outside the desktop. This is an interesting point to highlight because it restricts student explorations to a fixed place limiting one of the most meaningful ways of building knowledge from a constructivist point of view that encourages the idea of discover to learn. Therefore, the main idea of this proposal is a novel interaction way where users are able to interact with the environment to build knowledge from relationships acquired from the ambient. There are many ways of building these relationships, in this paper we present a new device called the “Interactive Learning Panel” where students relate information that is presented by the mobile application to a physical region of a panel. It is based on the idea of relating concepts with lines, or multiple choice questionnaires. Concretely, we have implemented a PDA web based application where users have to relate a flag to a region on a map. For instance, if a flag is given on the PDA screen, students have to relate it to the country depicted on the map and vice-versa. These panels are equipped with RFID tags that represent concepts on the panel and the PDA is equipped with a RFID reader that is able to read these tags and detect how user relate these concepts through their readings. So, to relate the concepts exposed in the PDA screen to those on the panel, users have to put the reader (mobile device) over the graphical representation in the panel.
238
R. Tesoriero et al.
The structure of the article is organized as follows. In Section 2 we present the informal learning concept that is the learning theory foundation of this approach, and then in Section 3 we perform a review to the most related informal learning applications in the field. Section 4 presents the most relevant functional and technological aspects of “Interactive Learning Panels”. Then, in Section 5 a case of study where we expose a fully functional application applying these technologies. Finally on Section 6 we present conclusions and future work.
2 Formal and Informal Learning Combs in [3] defines informal learning as “the spontaneous, unstructured learning that goes on daily in the home and neighborhood, behind the school and on the play field, in the workplace, library and museum, and through the various mass media, informal learning is by far the most prevalent from of adult learning”. As principal features we can mention that it has no place in education to be carried out in normal life or professional practice. There is no curriculum and is not organized professionally instead, originates accidentally, sporadically, and sometimes, depending on the requirements of practical situations. Besides, it is not systematically planned pedagogically according to unconsciously, related to problems in a holistic manner, and related to the situations management. And it is directly experienced in his “natural” role as a tool to live and survive. The concept of informal learning, as used by Dewey in an initial stage and then by Knowles, has experienced a renaissance, especially in the context of development policies. At first, informal learning was bounded on formal school and non-formal learning in courses [3]. From organizational informal learning processes are not formally organized and are not funded institutions [4]. A broader approach is that of Livingstone which is geared towards learning and self-directed and self emphasizes self of the process by the student [5]. The form of learning from people in their informal work is 80%. The workers learn a lot more than look to others, trial and error, asking colleagues for help that formal training. However, defining informal learning is not so simple and is subject of an ongoing debate [6, 7, 8 and 9]. A complete vision is presented in [9] and suggests that informal learning should be defined as a learning process that occurs independently and casually without being tied to a directive or instructive curriculum by presenting a typology. Thus in the intentional informal learning, learning goals and process are explicitly defined by the teacher or the institution. And the learner is the one who defined the goals and the process of the intentional informal learning. On the other hand, in the unintended informal learning targets are not defined beforehand, and there is no prescribed learning process, but can be developed at runtime when a learning occasion arises. There are also hybrid types of learning, such as museums and schools. Although the case study presented in this article is suited to formal learning, it is not limited to it. As reader may have noticed, panels can be placed almost anywhere, for instance bus stops, train stations, museums, libraries, town halls, schools, etc., leading to a every time – everywhere learning environment.
Interactive Learning Panels
239
We have to highlight the fact that in this article we present a new way of interaction to learn, not the learning method itself. Besides, this interaction capability can be applied in both formal and informal education seemingly.
3 Learning Using Digital Technologies in Museums, Science Centers and Galleries This section describes the most relevant aspects of m-learning as part of learning in general. In this case, we relied on the report by Roy Hawkey, from King’s Collage of London. This report describes the experiences in the field of education at the scene, inside museums, science centers and galleries. The goal of this section goal is to show that we can relate ubiquitous computing concepts, through context awareness (mainly location awareness), to the learning activity. Moreover, our intent is to provide specific points of contact on this domain and the collaborative learning networks. 3.1 On-Site Learning The work presented in [10] indicates that there are two types of exhibits, static and dynamic. The later one is divided into three categories: automatic, operational and interactive. According to this classification, this approach suits into the dynamicinteractive category. On the other hand, [11] makes a distinction between exhibit (broadcasts facts) and informal learning. Main difference is the support level of personal interaction among users involved in the system. Participation is an important point in all kinds of social activities. Examples of different types of participation can be observed in: The “In Touch” that allows visitors to create their own page and access it after the visit, or at “Bristol, get connected” where visitors can compare their ideas in a variety of topics. Another example is the “Victoria and Albert Museum” that allows students to create their own digital images. And finally, the “Wish you were here” where visitors can use a digital camera and an editing program to create postcards. The collaboration is an important aspect related to education. The article [12] reveals that there are manly three ways of collaboration in these scenarios: real collaboration between learners (now present in an exhibition), virtual students (online) and in 3D virtual reality environment. Among the most prominent examples are: the “STEM project” [13] promotes the publication of ideas on the part of museum visitors physical or virtual, in an educational use of the National Museum of Science and Industry on the Web [14], and the “Keystone online project” of Franklin Institute, in which the activity-based research and professional development opportunities are combined into a Web site devoted to facilitating the teaching of science-based research. Our approach is based on real collaboration through the Web. Students are able collaborate with their teachers to check practices and even see exam corrections by email automatically. Even more, from this point of view, students are collaborating with their environment. In a producer-consumer scenario, panels play the role of information producers; and students play the role of information consumers.
240
R. Tesoriero et al.
3.2 Personalization and Mobility Flexibility is crucial for students to pursue their own paths and can do so in their times. In [2] they have developed the theory of long-life learning through mobile devices such as PDAs and others, considering the hardware, software, communications and the interface design. Among the features of these systems we can mention: y The high portability and availability to anyone who wants to learn. y They should be individually adapted to the learners’ abilities, knowledge and learning styles; and designed to withstand the personal learning, instead of the general office work. y They should be not obstructive, so that we can capture the apprentice situation and retrieve knowledge that the technology clogging the situation. y They should be available on either side, to allow communication between teachers, experts and peers. y They should cater the evolution of knowledge and skills of the apprentice. y They should be able to manage learning over time, in this way the apprentice accumulates resources and knowledge that will be immediately accessible despite changes in technology. For instance, reference work. y They should be intuitive for people who are inexperienced with the technology. The Sparacino article [15] describes a study in the MIT Museum, more specifically in the exhibit “Robots and Beyond” where the system tries to “understand the use and produce an output based on the interpretation of the user context intention”. To do so, it is based on the behavior (places time, visited objects, etc.). On the other hand, the project “The Electronic Guidebook” at the Exploratorium in San Francisco makes effective use of the PDA to make “Bookmarks” materials and then visit them [16] One negative aspect appears in two dimensions, mechanical and cognitive skills. The need to hold the device reduces the activities related with their hands while reading the conversation inhibited demand. The article viewpoint [17] describes the “investigation nomadic” in which the trainees can manipulate the information and conduct research while moving the physical exhibits. In this learning type, is threatening to replace the Excel interaction with gestures “mediated talks with others and cognitively challenging”. Discarding this, which requires a careful design of instruction, trainees may benefit from their mobility within a context of physical objects and without feeling exhibits socially or physically isolated. There are many alternatives of applying this approach. The case study set in this paper is about evaluation. However, the same idea can be applied to retrieve information that may be hidden at first sight. It provides users with the ability to explore information through a learning panel just by going over the panel with the mobile device. For instance, students can easily explore the panel surface discovering or augmenting information that may not fully available for physical space.
4 ILP Interactive Learning Panels This section will explain the implementation details of the system. This description starts with a conceptual description of how the system works, it continues with the
Interactive Learning Panels
241
software system architecture, then it explains the interaction model it was employed, and finally it presents the interface design that has been followed to model the interaction. 4.1 Conceptual Description The System is based on three key elements: the panel, the mobile device and the service provider. The panel acts as a medium to interact between the user and the system. Basically it defines “hot spots” where: commands can be selected from the panel just by the gesture of putting the device near a “hot spot”. For example, in a questionnaire, this gesture may be used to select an answer for a question, to go to the next question, go back to the previous one, to end the questionnaire and so on. On the other hand, the mobile device is a tool that enables users to interact with the panel and perform tasks according to the user behavior. It provides the physical link between a physical representation (in the panel) and information received from service provider. Thus, the service provider: It is responsible for interpreting information of the user (acquired through the mobile device) and the information store into the database system. Although the system can be easily extended, it supports the recovery and playback of multimedia content (audio, text, video, etc.). It also supports a learning specific application, the questionnaire. This tool asks users about information that is represented into the panel, and they have to answer pointing the right graphical representation on the panel. For instance, if users are asked about which region on the map is related to a defined flag, they have to point the region on the map to answer that question. The application will be exposed in detail on Section 5. 4.2 System Architecture Users receive multimedia content on their mobile device through a Web based architecture. As explained on Fig. 1, in a first step, the user perform a simple gesture of bringing their mobile device to the panel, exciting, through the mobile device RFID reader magnetic field, any tag deployed on the reverse of the panel. This action results on the retrieval of the tag unique identifier. Immediately afterwards, the mobile device sends this identifier to the Web server. The server processes this request interpreting this identifier as a command, and generates a response to the action. The
Fig. 1. System Architecture
242
R. Tesoriero et al.
answer may result on a resource transfer (video, photography, audio, text, etc.) or information, stored in the database system (for example, the result of a question). Thus, users finally perceive on their mobile devices the result of the interaction between them and the panel. 4.3 Interaction Modeling The system was implemented using the MVC (model-view-controller) architecture depicted on Fig. 2. Through this architecture we mapped the rugged natural gestures of user commands to certain events and actions that are processed by our domain model.
Fig. 2. The distributed Model View Controller implemented by the system
We are facing a distributed system. So, on the client side, the RFIDReader entity signals to the EcoPanelModel entity that a tag was read. The EcoPanelModel transmits the identifier to the server through a proxy represented by the ServerProxy class. This identifier is processed by the server and is mapped to a Command (for instance: next question) or ToolModel (a questionnaire) as necessary. Thus, as a result of the previous process, EcoPanelModel, which is part of the customer, receives the result as a set of serialized instances.
5 Case of Study For the case of study we will present one of the developed projects we made in our research laboratory. It treats the idea of teaching the Geography in the school, the library or any other learning environment. What we want to present is a connection between the student and the material provided by this subject. This material will be presented in such a simple and intuitive way that students can understand and interact with it without the help of any tutor. The goal of the panel is the identification of spatial entities as countries, cities, seas, oceans, etc. in a given map (the panel). On Fig. 3-a the system asks students for personal information and operation modes. Students are free to choose between two operation modes: the exam or a practice. The difference between these two modes lies on whether results are sent to the professor or not.
Interactive Learning Panels
(a)
(b)
243
(c)
Fig. 3. Main application screens
Once an operation mode is selected, a question is displayed. The question may be the identification of a region in the map or a flag corresponding to a defined region on the PDA screen (Fig. 3-b). For instance: “Select a state flag and then locate it into the map”. At the right bottom of the page the user can observe the number of the question he/she is answering and the total number of questions. As mentioned before, to select something, he/she has just to bring the PDA device near to the flag or region image depicted in the panel. It is a simple and clear process to develop. Commands and information retrieval are preformed analogously. Once questionnaire was fulfilled by a student, the results are displayed on the PDA screen (Fig. 3-c). The panel contains the map, flags, and other interactive buttons that the user will need to realize the practice or evaluation process. The next item to describe is the interaction panel based on RFID tags. While Fig. 4-a shows the map panel from the student point of view, Fig. 4-b shows the implementation of the panel based on a RFID tag matrix. On the right of Fig. 4-a we can see the map of the political division of the country and on the left we can see the list of flags that represent all the states of this county. At the bottom of the panel, there should appear a control panel. Usually, they are: the go back or forward panel buttons, to go back and forward through the questions, the confirm and cancel panel buttons to accept or cancel any action, and finally the exit panel button, to exit with or without saving the activity sending results by email.
(a)
(b) Fig. 4. The map panel
244
R. Tesoriero et al.
6 Conclusions This article describes an m-learning solution that exploits mobile technologies that are actually available, or will be available in the near future allowing students to interact directly with the materials they have to learn. Besides, ILPs also provide information to teachers about students’ progress. This system is implemented using several technologies, including mobile devices which support communication, RFID technology enabling gestural interaction, and the subject material which is introduced to enhance the student participation. And among main advantages of the system is worth to highlight the following issues: y y y y y y
Implementation of new interfaces for any mobile device. New resources can be introduced in a scalable and open way through the Web. Accessibility features, as audio-text for blind people, are easy to add. Multi-lingual support for resources on the PDA side. Great flexibility due to the panels can be updated quickly and efficiently. Economic deploy, because passive lo priced RFID tags and RFID popularity.
As future work, we can mention the extension of this interfaces to existing social networks to expand the service, for instance to Google Maps. Besides this form of interaction is not limited to retrieving information, instead it allows the execution of parameterized commands that can lead to the creation of active control panels that can evolve dynamically. Acknowledgements: We would like to thank CENIT project (CENIT 2008-1019MIO!) and CICYT project (TIN2008-06596-C02-01) for funding this work.
References 1. Chen, Y.S., Kao, T.C., Sheu, J.P., Chiang, C.Y.: A Mobile Scaffolding-Aid-Based BirdWatching Learning System. In: Milrad, M., Hoppe, H.U. (eds.) IEEE International Workshop on Wireless and Mobile Technologies in Education, pp. 15–22. IEEE Computer Society, Los Alamitos (2002) 2. Sharples, M.: The design of personal mobile technologies for lifelong learning. Computers & Education 34, 177–193 (2000) 3. Coombs, P., Ahmed, H.: Attacking rural poverty. How nonformal education can help, Baltimore (1974) 4. Watkins, K., Marsick, V.: Informal and Incidental Learning in the Workplace (1990) 5. Livingstone, D.W.: Mapping the Iceberg. NALL Working Paper number 54 (2002) 6. Mocker, D.W., Spear, G.E.: Lifelong Learning: Formal, Nonformal, Informal and SelfDirected. Information Series No. 241. ERIC Clearing House on Adult, Columbus (1982) 7. Hawkey, R.: Learning with Digital Technologies in Museums, Science Centres and Galleries. A Report for NESTA Futurelab (Report 9). Future Lab. King’s College, London (2004), http://www.nestafuturelab.org/research/reviews/09_01.htm (Last access 26.08.05)
Interactive Learning Panels
245
8. Sefton-Green, J.: Literature Review in Informal Learning with Technology Outside School (2004), http://www.nestafuturelab.org/research/reviews/07_01.htm (Last access 26.08.05) 9. Vavoula, G.: KLeOS: A Knowledge and Learning Organization System in Support of Lifelong Learning. PhD Thesis, University of Birmingham, UK (2004) 10. Miles, R.S., Alt, M.B., Gosling, D.C., Lewis, B.N., Tout, A.F.: The Design of Educational Exhibits. George, Allen & Unwin, London (1982) 11. Bradburne, J.: A new strategic approach to the museum and its relationship to society. Museum Management and Curatorship 19(1), 75–84 (2001) 12. Galani, A., Chalmers, M.: Can you see me? Exploring co-visiting between physical and virtual visitors. In: Museums and the Web 2002, Archives & Museums Informatics, Pittsburgh (2002) 13. The STEM project, http://www.sciencemuseum.org.uk/education/item 14. Bazley, M.: The internet: who needs it? Journal for Education in Museums 19, 40–43 (1998) 15. Sparacino, F.: The museum wearable: real-time sensor-driven understanding of visitors interests for personalized visually-augmented museum experiences. In: Museums and the Web 2002, Archives & Museums Informatics, Pittsburgh (2002) 16. Semper, R., Spasojevic, M.: Devices and a wireless web-based network to extend the museum experience. In: Museums and the Web 2002 (2002) 17. Hsi, S.: A study of user experiences mediated by nomadic web content in a museum. Journal of Computer Assisted Learning 19(3), 308–319 (2003)
WebELS: A Content-Centered E-Learning Platform for Postgraduate Education in Engineering Haruki Ueno, Zheng He, and Jingxia Yue National Institute of Informatics, Japan 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, Japan {Ueno,Hezheng,Yue}@nii.ac.jp
Abstract. This paper proposes a general purpose e-Learning platform WebELS to support higher education in engineering and science especially for PhD education. WebELS consists of three major modules, i.e., Learning for selflearning, Meeting for Internet-based on-line meeting, and Lecture for Internetbased distance lecture, as an “all-in-one” system. Using an easy-to-use authentication interface non-IT users can edit their own e-Learning contents on their personal computer as a series of slides from PPT, pdf, image and video data and upload to the WebELS server for e-Learning. Audio and cursor can be recorded onto each slide and be play-backed in a synchronized manner for helping understanding. WebELS is a Java-based server system and functioning in a low-speed Internet environment. The WebELS software is available as an open source system and is used in universities and industry in Japan and Asian countries. Keywords: e-Learning, Internet, higher education, postgraduate education, engineering education, software platform, open source, Internet meeting, distance lecture.
such as advanced networking technologies, multimedia information processing technologies, and sophisticated software technologies, at lower cost and higher quality at global scale [2]. The role of universities is requested to change in the circumstance, in such a stream. The traditional model of Engineering Education in university is based on a combination of undergraduate education and graduate education by classroomoriented lecture. Most of Engineering Education in Japan is still in this way. Some universities are trying to introduce Internet-based e-Learning for correspondence courses for working students. This is because most professors are conservative on introducing e-Learning. However, social demand for graduate education in international scale is increasing, that results in a use of Internet-based e-Learning especially in a PhD program. In this paper we propose a general purpose e-Learning platform WebELS to support flexibility and globalizing of higher education in engineering and science especially for PhD education by means of advanced information technology. According to our analysis of PhD education features of e-Learning system should be different from those of undergraduate education. For example, typical lecture style in PhD program is discussion-based meeting. Presentation at an international conference is also a typical style of education. Test for evaluating the quality of knowledge that is one of key functions in undergraduate education is not applied. Standardized texts are not used in lecture. In addition, globalization of higher education in science and technology become strongly important. It should be noted that e-Learning platform should be available not only in advanced countries where high quality Internet is widely in use but also should be available in developing countries such as in Asia and Africa where low speed Internet is normally used. WebELS is designed to provide high quality eLearning environment for higher education in science and technology to meet these demands. Since WebELS is designed as a general purpose e-Learning platform, it is useful not only in education but also for business meeting in institutions and industry. Combined use of WebELS with Moodle [4] is also suitable idea of use such that functions of both systems would be featuring together. WebELS consists of three major modules, i.e., WebELS Learning for self-learning, WebELS Meeting for Internet-based on-line meeting, and WebELS Lecture for Internet-based distance lecture, as an “all-in-one” system. Using an easy-to-use authentication interface non-IT users can edit their own e-Learning contents very easily as a series of slides converted from PPT, pdf, image and video data on their personal computer and upload to the WebELS server for dissemination. Audio and cursor can be recorded onto each slide and be play-backed in a synchronized manner for helping understanding. WebELS is a Java-based server system that consists of Editor, Viewer and Manager, and functioning on a small PC server in a low-speed Internet environment. Every functioning module is downloaded from the server onto a user’s computer automatically as a Java Applet when he/she chooses a specific function in using the system via Internet browser such as IE. The WebELS software is available as an open source system and is used in universities and industry in Japan and Asian countries.
248
H. Ueno, Z. He, and J. Yue
2 Background Concepts of WebELS WebELS is designed to provide an advanced e-Learning platform for globalizing graduate education in science and technology focusing on authoring and dissemination of multimedia contents, aiming to assist instructors to archive their learning materials on the web for on-demand learning, on-line meeting and on-line lecture. We have analyzed the characteristics of PhD education from the point of views of eLearning [1,6,7]. Some key characteristics are as in the following. 1. PhD students are research partners as well as students, and activities as individual scientists are involved in education. Joining research meetings, presenting at international conferences are typical examples. PPT-based presentation followed by discussions is a typical style. 2. PPT-based lecture is a typical style of classroom lecture, and PPT-based playback with voice and synchronized cursors on a learner’s computer seems to be reasonable for on-demand self learning. High quality PPT slides with voice and cursor are requested to replay in a narrowband Internet. 3. Powerful authoring features for non-IT users are strongly requested so that professors can edit their own educational materials on their personal computers and upload them onto the WebELS server. 4. The e-Learning system must be used on multiple operating systems which include Windows, Mac OS and Linux in a global situation over the Internet. WebELS was designed to meet these requirements mainly for supporting global PhD education as a contents management e-Learning system (CMS). Java programming language was chosen for achieving a multiple OS system to be used in Windows, Mac OS and Linux. WebELS is an all-in-one e-Learning system consisted of three modules, i.e., Learning for self-learning, Meeting for Internet-based on-line meeting, and Lecture for Internet-based on-line distance lecture. PPT contents are converted into a series of image slides in an authoring session using the easy-to-use editor. Then, voice and synchronized cursor are attached to each slide by means of a voice recording function of the editor. Pdf contents are converted to images, i.e., slides as well. Video clip can be inserted into the list of slides by means of specially designed function of the editor. So, it is easy to combine PPT data, pdf data and video data on a user’s computer using the editor. The editor is automatically downloaded from the WebELS server when the user clicks editing function of the contents list as discussed in the following chapters. Image-based contents are easy to maintain and safety for copyright protection. Export and Import functions are available so that the author of specific contents could be moved from a server to a server. This function is useful to maintain WebELS contents in a distributed WebELS server systems environment. Same multimedia contents can be used for three styles of usage in a similar manner. That is, a content edited and uploaded to the WebELS Learning database is able to use for WebELS Meeting as well by copying it within the server by means of the Export-Import function. By means of an embedded voice meeting function it is easy to achieve an Internet meeting and Internet lecture in a narrowband Internet environment with high quality image slides. The quality is accomplished by means of a pre-downloading feature of the system. On-line zooming and cursor pointing functions support
WebELS: A Content-Centered E-Learning Platform
249
easy-to-understand slide-based on-line meeting and lecture. It should be stressed that the design concepts as well as key features have been carefully considered by the analysis of users. We are going to keep this policy in the future. An offline-viewing feature, named OFV, was added according to student’s request. By means of OFV a user can download a specific content from the server onto his/her computer and bring it without Internet connection for viewing later in offline. An OFV content is a package of viewer and content. In addition to bring it on user’s computer for personal learning it is possible to send by an e-Mail attachment and insert to another contents list maintained by another platform such as Moodle for distribution. Another policy in developing the system is that the WebELS software should be distributed as an open source. Because of this universities in developing countries will be able to install the WebELS system on their PC server through the Internet with free of cost. We believe that this policy would be one type of international contributions in globalization of higher education in science and technology.
3 Outline of the System Fig.1 shows an overview of the WebELS system. As shown in the figure, WebELS is a server-based system functioning on a Linux server. Every user can use the system over the Internet using a browser, such as IE. The editor is downloaded onto user’s computer by accessing to the server for authoring contents, and viewer is downloaded to user’s computer when he/she access to the server for self-learning, Internet meeting and distance lecture. The editor and viewer are implemented in Java and thus downloaded as Java Applets. Windows users are requested to pre-install JRE (Java Runtime Environment) through the Internet. Java programming language was chosen because it is powerful in developing a variety of functions and a multi-OS system was realized, i.e., the system provides every function for Windows, Mac and Linux users equally. Multi-language feature is another important character of the system to support a global use of e-Learning in science and engineering. The user interface is automatically switched to English or Japanese language by detecting Browser language. It is possible to append other languages in this way. It should be noted that the editor as well as the viewer, i.e., player, is working on a client computer in off-line processing. The processing in the Learning session is monitored by the server for managing the user’s operations that include Login, Logout, Downloading, Uploading, and so on. For on-line meeting and lecturing the server’s role is more. It includes On-line slide changing, On-line cursor positioning, On-line zooming, Voice meeting, and so on. Since PPT-based slides are predownloaded onto every members of a meeting as well as a distance lecture, high quality slide sharing by multiple locations are available in a low-band Internet environment. Downloading time is varying according to the speed of data transmission over the Internet from several seconds to several minutes. Figure 2 shows an example of viewing windows for on-demand self learning of WebELS Learning. Synchronization of sound and cursor makes easy to understand PPT-based lectures on user’s computer in a narrow-band Internet environment [3]. A
250
H. Ueno, Z. He, and J. Yue
Fig. 1. Overview of the WebELS system
Fig. 2. An example of playback for self-learning (Shimamoto)
content with viewer is downloaded from the server onto the user’s computer in advance, then playback is executed on it offline. The zooming function is available so that a learner can zoom-in at any position of the slide. A voice and cursor synchronization helps him/her in understanding a lecture. A user-friendly editing interface to create the content is installed within the editor as discussed later. Fig. 3 shows an example of two slides in on-line multi-point Internet meeting by using WebELS Meeting module. Several groups are able to organize Internet meetings independently by using the embedded functions such as ad-hoc presenter
WebELS: A Content-Centered E-Learning Platform
251
management, embedded voice line, on-line cursor control, on-line zooming. A virtual meeting room is automatically created when several users access to a specific content with specific password. One of the members can obtain the presenter’s right to control presentation just by clicking a presenter bottom on the viewing window. Presenter can be switched to another member by clicking this bottom any time during the session. The server has the monitoring function to manage this kind of multi-point meeting. Since WebELS is a download-based system, every user is requested to download the content onto his/her computer before starting the meeting. During the meeting session only control signals for changing slides and pointing cursor in addition to voice signal are transmitted over the Internet, and then high quality slides can be shared even in a narrow band Internet environment.
Fig. 3. An image of Internet meeting (Bourgeois/UNESCO)
One of the most important functions of WebELS is that easy-to-use authoring interface is available for non-IT users. Users can create their own contents on their own computer from existing PPT, pdf, image and video data by means of a specially designed editing interface. Fig. 4 shows the editing interface of the WebELS system. In creating a new multimedia content a user is requested to click the “Create a New Course” bottom on the contents list window assigned to him/her first. Then the authoring system is downloaded from the server onto the client computer as a Java Applet followed by initialization process to start functioning, and the editing interface appears as in the figure. Editing procedure is quite simple as follows. For a newly creating content, enter the title of a course into the first column, choose a category and subcategory, set a password to protect from unauthorized viewing if needed. Creation of a content is done as in the following procedure. Choose one of four types of data from PPT, pdf, image, and video by clicking a button and press the “Add” button as shown in the
252
H. Ueno, Z. He, and J. Yue
Fig. 4. Editing interface
figure. Then, choose a data-file on the computer according to the guidance. The identified data are converted to a standardized WebELS slide format and recorded onto the working file for further editing procedure. The slide list appears in the window as in the figure. It is possible to combine four types of data into one content using this interface by means of Add, Remove, Up and Down functions.
Fig. 5. Voice and cursor recording interface
Fig. 5 shows the user interface for recording voice and cursor onto a specific slide that is identified in the editing interface of Figure 4. Choose one in the slide list window, click the “Preview Record” button, then the slide appears as in Figure 5. Recording voice and cursor can be done by quite simple manner. Just click the “Record
WebELS: A Content-Centered E-Learning Platform
253
Audio & Cursor” button and recording process begins. Voice recording is achieved just by talking toward a microphone. If a cursor is used during talking, then synchronized record of cursor is done. By clicking the “x” button (erasing-window button) of the window the recorded slide is moved to the working file and a “recorded mark” appears in the front of each slide in the window.
4 Implementation Some key technologies in the implementation of the WebELS system are discussed in this chapter. More discussions are in [3,5]. 4.1 System Configuration In WebELS system, implementation of editor, player and manager is based on a typical Browser/Server (B/S) structure. Under this structure, all user interfaces are implemented on web pages. Main tasks are executed by server side and only small work is carried out on browser side, it will much lighten the loads on client-side computers. Moreover, this kind of system can be accessed and used without downloading or installing other tools beforehand. Fig. 6 describes the architecture of WebELS system.
Fig. 6. Architecture of WebELS system
Web server is responsible for users’ requirements. A customized HTTP server handles web-based user interface that includes HTML and JSP pages. Java Servlets response the interaction with the Java tools on client-side and perform data transformation with database server. Database server is to provide data storage for achieved contents, supports contents editing, manage user information, and verify course dependency. And all user account information is maintained by SQL database server.
254
H. Ueno, Z. He, and J. Yue
4.2 Voice and Cursor Synchronization In the WebELS system, a cursor file is embedded into each slide of the contents to simulate lectures’ action. Users can edit the slide and point a cursor onto corresponding slide to tip the topic they are talking on, or show the points they want to strengthen. Each slide has its own timeline in order to: !Syntax Error, .) record the cursor positions (x, y) chronologically; ii) store the information (s, f) for the embedded audio stream file simultaneously, in which s is the current state (play or stop) and f is frame numbers.
Fig. 7. Synchronization of cursor positions with audio stream
As in Figure 7, the cursor positions are synchronized with audio stream in each slide file by the time shaft. For examples: i) when t=t0, (s, f)=(play, 0) and (x, y)=(0, 0), the audio stream is started, and there is no cursor action; ii) when t=t1, (s, f)=(stop, f1) and (x, y)=(x1, y1), the audio stream is paused and the cursor is pointed to the location (x1, y1) on this slide; iii) when t=ti, (s, f)=(play, fi) and (x, y)=(xi, yi), the audio stream has processed fi frames and the cursor is pointed to the location (xi, yi) on this slide. Also a continuous audio stream can be dispersed at different cursor positions according to the frame numbers recorded by the timeline. Therefore the duration of audio stream is normally shorter than cursor action although both of the actions are simultaneously recorded. Compared with real-time recording system, the file size of the audio stream which was edited based on timeline was decreased.
5 Concluding Remarks WebELS is designed to provide an all-in-one e-Learning platform to be used in a narrow band Internet to support flexible education and internationalization of graduate education based on the analysis of requirements of Ph.D education in an international situation. WebELS is also useful for such as undergraduate education, employee education, life-long education and online-meeting for international projects. WebELS is available over the Internet at a Web-site such as [9]. Since WebELS is designed as a content-centered e-Learning platform an integrated use with LMS such as Moodle [4] should be useful for under-graduate education.
WebELS: A Content-Centered E-Learning Platform
255
Acknowledgement The authors would like to express sincere thanks to all persons who support the WebELS project of NII Japan, especially to Dr. Vuthichay Unpornaranveth for his contributions in designing and implementing the WebELS system, and to Prof. Nobuo Shimamoto for his contributions in concept formation and design of user interface from a point of views of non-IT users. The project is funded by such as Science Research Foundation of Japan, Research Organization of Information and Systems and The Graduate University of Advanced Study. We express sincere thanks to the eLearning project of UNESCO Jakarta office for further collaborations using WebELS.
References 1. Ampornarambeth, V., Zhang, T., Hadiana, A., Shimamoto, N., Ueno, H.: A Web-Based eLearning Platform for Postgraduate Education. In: Proc. the 5th IASTED International Conference on Web-Based Education, pp. 388–393 (2006) 2. Blinco, K., Mason, J., McLean, N., Wilson, S.: Trends and Issues in E-Learning Infrastructure Development. DEST, Australia and JISC-CETIS, UK (2004) 3. He, Z., Yue, J., Ueno, H.: WebELS: A Multimedia E-Learning Platform for Nonbroadband Users. In: Proc. ICCET 2009 (2009) 4. Moodle: http://moodle.org/ 5. Rahman, M.M., He, Z., Sato, H., Ampornaramveth, V., Shimamoto, N., Ueno, H.: WebELS e-learning system: Online and offline viewing of audio and cursor synchronized slides. In: Proc. ICCIT 2007, pp. 1–6 (2007) 6. Rahman, M.M., He, Z., Sato, H., Ampornarambeth, V., Shimamoto, N., Ueno, H.: WebELS E-Learning System: Online and Offline Viewing or Audio and Cursor Synchronized Slides. Proc. ICCIT 2007, 106–110 (2007) 7. Ueno, H.: Role of e-Learning in Engineering Education - Background and Outline of WebELS. In: Proc. International Workshop on Engineering Education (2008) 8. Ueno, H.: Internet-Based Distance Learning for Lifelong Engineering Education - A personal view and Issues. J. of Information and Systems in Education. 1(1), 45–52 (2002) 9. WebELS: Web E-Learning System, http://webels.ex.nii.ac.jp/
A Pen-Based Teaching System for Children and Its Usability Evaluation Danli Wang1, Tingting Ying1, Jinquan Xiong2, Hongan Wang1, and Guozhong Dai1 1
Institute of Software, Chinese Academy of Sciences, Beijing 100090, China 2 Jiangxi Institution of Education, Nanchang 330029, China [email protected]
Abstract. The computer has become more and more important in children’ life and learning. Various issues exist in the application of multimedia edutainment software and courseware. Therefore, we analyze the current situation and requirement of preschool education software, and present the development of pen-based teaching system for children. Lecturing courseware, annotation, adding contents have been provided, which make targeted classroom teaching convenient, and children interactive courseware function has also been provided to arouse their learning initiative and enthusiasm. After analyzing the feedback, we design and develop the improved version of the system, and evaluate the two versions through experiments. Finally, we propose some suggestions for its modification. Keywords: pen-based interaction, children, teaching system, usability.
A Pen-Based Teaching System for Children and Its Usability Evaluation
257
teachers’ need. Some software for Intelligence and skill development lacks of systematic, and not suitable for realizing whole classroom teaching. In addition, most of software is using the keyboard and mouse operation basically. Lecture activities need frequent outline, so the keyboard and mouse operation is less free and easy to operate than pen, and against children’s participation in classroom activities. In recent years, pen-based interactive technology has been widely used. Pen-based interaction has great advantage in teaching: the operation is free and simple in keeping with cognitive and operation habit. It will provide an effective means of interaction in teaching activities; In addition, Pen-based interaction is an effective tool to solve the input problem of Chinese characters computers. It is convenient to writing for teachers and helpful to learn Chinese writing for children [3]. According to the current situation of preschool education software and requirement of preschool education, combining with advanced teaching theory, we analyze the characteristics of children, and propose an pen-based teaching system and realize a teaching environment where teachers can fully play a leading role, while children can play more initiative and active, and participate in the teaching in order to promote the teaching effect. Through using the software in kindergarten, software functions and performance have been improved continuously according to the users’ feedback, and moreover based on the earlier version (version 1), a new version(version 2) has been designed and developed. These two versions have been evaluated in usability. We will analyze the evaluation process and results in detail and give the system improvement comments and suggestions.
2 Related Works 2.1 Pen-Based Interaction Technology Natural interaction and human interface is the development trend of human-computer interaction. Pen-based user interface develops with the emergence of pen-based interaction devices like the handwritten pen, the touch screen and so on. The earliest pen-based interaction devices appeared before mouse and graphical user interface, which appeared in the system of Sketchpad in 1963[4]. With the progress of computer software and hardware technology, pen-based computing environment with features of pen-based interaction is rapidly developing. One of the obvious characteristic in pen-based interaction is easy to control, effective to use, natural to outline [5]. Both text and graphics can be naturally outlined and displayed on the screen; therefore, applying pen-based interaction to education has become a new hotspot. It not only makes the full use of natural pen and paper way of working, but also make up for the limit of estimating the position of mouse click on and contents input from keyboard, simply teachers’ operation, reduce controlling difficulties [3]. 2.2 Educational Software for Children Computer technology is affecting the education and learning, especially network and multimedia technology, which is very suitable for helping learning. Computer technology has enough potential to improve students' score. Multimedia electronic
258
D. Wang et al.
courseware combined with text, video, and images has completely changed the traditional teaching and learning mode. The multimedia teaching process can be regarded as the process in which teachers make use of multimedia technology to present study materials and let students to obtain information/knowledge. In the learning process by using multimedia, multimedia is not only an information transmission tool, but also means of helping students understand knowledge [6]. Multimedia technology which combines various media such as text, images, audio, video, animation and so on, can attract children's attention, arouse their interest, create the scene atmosphere, arouse their emotional development, enhance the teaching effect through the organic combination of teaching information, promote children's multisensory development through improvement of interpersonal interaction [7]. Currently, many researchers perform in-depth studies in children’s education software, and have developed some education software. Britain’s Sussex University studied the learning environment that supports children’s learning of biological concepts [8]. Carrie Heeter et al. studied software suitable for girls to learn animals [9]. Several large software companies also marketed multimedia course development tools, e.g. Microsoft’s PowerPoint allows users to conveniently produce texts, graphs, adding images, audio, motion pictures, video, etc. It also allows design of presentation effects based on needs. Macromedia’s Authorware is an editing platform based on icons and lines, multimedia production software for developing Internet and on-line learning applications [10]. Software such as Action, Flash provides tools for producing course materials. However, the cost of learning this kind of software is high, the operation is complex, and special training is necessary. Therefore it will take ordinary teachers long time to make courseware, the efficiency is very low. Software are not satisfied with the requirement for preschool education, because of not considering the characteristics of preschool teachers and the situation of children education, therefore, they cannot be used for most preschool teachers. In addition, there are some series of courseware, such as WaWaYaYa etc. [11], which are made according to some teaching materials. Coursewares provide abundant content of courses and multimedia display effect. However, this kind of software does not allow teachers to modify the content of courses and reuse courseware materials, which constraints teachers’ activities. Therefore, developing the teaching system for children is proposed in the paper. It is suitable for Chinese children, convenient to teaching activities for teachers. Simultaneously, allow children to participate in teaching activities, improve their learning enthusiasm and initiative.
3 Design and Realization Pen-Based Interaction System 3.1 Platform Architecture Based on above analysis, we developed a pen-based children teaching system, the framework of which is shown in figure 1. The top layer is pen-based user interface; the middle layer is data processing layer; the lowest layer is system database, primarily comprised of courseware base, resource base, constrain base, gesture base and text base. Users can communicate with system through pen-based user interface. Input
A Pen-Based Teaching System for Children and Its Usability Evaluation
259
Fig. 1. Architecture of System
gestures are transmitted into system. The system first analyzes and comprehends the input. Through the support of recognition algorithms, the system obtains the input intents of the user, and performs the corresponding operations. Then the system’s responding results are output to the user interface. 3.2 System Function The function of the system is shown in figure 2. It is divided into four modules: basic operation module, media interaction operation module, handwriting operation module and auxiliary operation module. Among them, the media interaction operation includes pictures interaction, text interaction, audio, Gif animation control, and track-based animation control, all of which have their own interaction properties (such as: click to show, click to hide etc.), and allow users to interact freely.
Fig. 2. System Function
3.3 System Realization The teaching system was developed in VC++ 6 based on our lab existing pen-based software development platform. This platform provides basic pen-based interface and gesture lib. Users go through pen to interact with the system. The system first analyzes and comprehends the input, carry on the according processing and operation, then the system’s responding results are output to the user interface. The system interface of Version 1 is showed in figure 3-4. This system has been provided for more than kindergartens to try out. Analyzing the users’ feedback, we propose an improved version (Version 2), which is showed in figure 5-6.
260
D. Wang et al.
Fig. 3-4. Interface of Teaching System (Version 1)
Toolbar in left
Toolbar in bottom
Fig. 5-6. New Interface of Teaching System (Version 2)
4 Usability Evaluation of the Teaching System 4.1 The Evaluation Method Among all of the evaluation methods, user testing and questionnaire are selected to evaluate the teaching system. The tester first finishes a group of tasks according to the requirements, then he or she answers the questions. In order to let the tester learn the teaching software quickly, the training scheme and testing tasks were devised using interface scenarios, and this method has been proven to efficiently improve learning rate of the user and finishing rate of the tasks. 4.2 Evaluation Process Participants and Software The users of our system are mainly teachers who teach students by the teaching system. Therefore, we choose 6 testers three male and three female. All of them are 20 to 30 years old. In addition, some of the testers were ever teachers. The evaluation software is the teaching system developed earlier (to be called version 1 indicated with V1 later), another is the new teaching system in this article (called version 2, indicated with V2 later). The differences between two versions are that we add a dragged function toolbar in the version 2. The purpose of this evaluation is to understand what kind of interface testers like more and test various availability indicators of the teaching system. In this evaluation we adopts contrast experiments for two versions, we will also carry on the contrastive analysis for two versions’ test data.
A Pen-Based Teaching System for Children and Its Usability Evaluation
261
Evaluation training The staff member first introduced the purpose and arrangement of this experiment, then explained the usage of the software with a scenario-based method, and offered a demo. Then, the testers had 10 minutes exercise. At last, the instructor introduced the testing requirements and tasks. Testing and questionnaire After the exercise, the testers began to test according to the task requirements. The staff members recorded the errors and other problems during testing. All the errors and problems did not be prompted. After testing, the testers filled out the questionnaires. 4.3 Result Analysis We mainly analyze the test result from two aspects. One is objective aspect, analyze data recorded during the test procedure, such as wrong number, time of finishing tasks and so on; The other is subjective aspect, testers’ questionnaire survey results. Objective data z Integrity Table 1 shows the integrity which the testers finished each task in, and the data is analyzed using statistical method, and the average integrity and deviation of finishing every task were obtained. Table 1. Integrity of finishing tasks (the highest is 1) Task V1 Lesson preparation V1 class begin V2 Lesson preparation V2 class begin
： ： ： ：
Mean 1 0.98 1 0.98
s.d. 0 0.03 0 0.03
From table 1, we can see the achievement of testers in two versions is the same. According to task completion rate, the whole task can be basely finished. In version II, we have joined the dragged toolbar, however from the objective data, it is not difficult for testers to understand and operate. Integrity of the task doesn’t reduce. Therefore, the data shows that testers feel tasks difficult to understand or to implement during the test, and the tasks and operations are within their cognitive workload and operation scope. z The errors During the whole evaluation process, when tester operates, problems may occur. The reason may be the habit of the testers, or the operation error, or it may be that the system produced the errors. The statistical results of the errors are: during the process of finishing tasks for two versions, the numbers of system errors are both 0. It shows that our systems are running stable relatively. The error frequency of operation is also low overall. The number of errors in version 1 is more than that in version 2, such as that the pen attributes Toolbar in version 1 has hidden page turning buttons, thus results in wrong operations. The data is shown in the Table 2.
262
D. Wang et al. Table 2. The statistics of the errors Version V1
Group System error Operation error System error Operation error
V2
Error number 0 2 0 1
The occurrence rate of both in version 1 and version 2, errors are very low. Testers used the teaching system very fluently. It implies that the software has high reliability and naturalness. However, since errors occurred, there were some aspects should be improved in the system. We will focus on these errors, and get the clue of the further perfection. The results of questionnaire For the design questionnaire, the teaching system includes three parts: the first one is the overall evaluation of the teaching system, the second is evaluation of version 1, and the last one is the evaluation of version 2. In the 8 questions of part 1, the 1st, 3rd, 5th and 7th questions are proposed to version 1, the others are proposed to version 2. The average score of version 1 is 2.71, and 3.79 for version 2. Therefore, the synthetic evaluations for version 2 are higher than version 1. Testers prefer to use version 2 which has the dragged function toolbar. The questionnaire is consisted of 18 questions, adopting 5 degree evaluation table. It provides the overall evaluation of our teaching software version 1, including easy-learning, easy use, reliability, naturalness, interaction and satisfaction. The results are shown in Table 3. The usability evaluation score of version 1 shows in figure 7. Table 3. Usability evaluation for version 1
A Pen-Based Teaching System for Children and Its Usability Evaluation
263
Similarly, we also evaluate version 2 on easy-learning, easy-use, reliability, naturalness, interaction and satisfaction. The results are shown in Table 4. The usability evaluation score of version 1 shows in figure 8. Table 4. Usability evaluation for version 2
From above data, the average scores of each availability indicator in both versions are more than 4, which are high scores. The standard deviations are also in the normal scope. It shows that the evaluation result of the teaching system is satisfied. Especially in terms of Naturalness, the system has been given higher scores. It is mainly because we fully taken into account the users’ habits during the process of design and realization. Synthesized comparison between two versions is shown in Table 5 and in Finger 9. The scores of two versions are not very different, synthetical score in version 2 is higher. It shows that as a whole, synthesized evaluation for version 2 is higher than version 1. And in easy-learning, easy-use, reliability and satisfaction, the evaluation score in two versions is closed. It shows that joining the dragged toolbar doesn’t make users feel more difficulties. However, the evaluation of version 2 is higher than that of version 1 in two aspects of naturalness and interaction. It shows the dragged function toolbar makes users more convenient when using the teaching system, while in version 1, users cannot drag the toolbar at will. If the position of teachers is distant from the function buttons like page up or page down buttons, then the teacher may have to go to distant locations to find those function buttons, causing a certain amount of inconvenience. The introduction of the dragged function toolbar strengthens interface interaction of the entire system. According to testing situation and questionnaire survey, users authorize this interactive approach of the dragged toolbar.
264
D. Wang et al. Table 5. Synthesized comparison between two versions
Fig. 9. Synthesized comparison between two versions
4.4 Overall Evaluation For both version 1 and version 2, the integrity of finishing task is very high. Simultaneously the error rate is also quite low. The testers basically can complete the testing tasks well. Looking from questionnaire survey, testers have given good evaluations in both version 1 and version 2. The improvement that is the dragged function toolbar in version 2 has been approved. In addition to evaluation questionnaires, we have gotten some testers’ ideas from their feedback for the system, such as their favorite functions, improvement points. Testers feel that adding animation in the course of teaching has enhanced the effectiveness of teaching. Whiteboard function is also very appropriate for the system design; teachers can open "white board" for writing at any time. However, through the evaluation process, we have found the shortage of the software design. For instance, the dragged function toolbar can sometimes hide other buttons, or when the dragged toolbar moves to upper boundary, it cannot automatically close the border in accordance with the placing of the border, as well as some users want to operate more naturally and so on. We will improve it and do further research on how to make teachers interact with the teaching system more smooth and natural.
5 Conclusion This paper, focused on the issues in multimedia teaching and the needs of children’s education software, presented the development of a pen-based children courseware system. The platform provided for the teachers a convenient environment for lecturing
A Pen-Based Teaching System for Children and Its Usability Evaluation
265
courseware to facilitate multimedia teaching for the teachers, as well as for the children an interactive and interactive environment, where children can be allowed to participate in multimedia courseware explanation to deepen their understanding and memorization of the knowledge, so as to improve their learning results. This paper combined pen-based interface technology and the design of natural user interface to provide for the teacher and the children an easy-to-learn, easy-to-use teaching system. Through the use of system and feedback, we have modified the design of the system and obtain the new software version. Through comparing the new version with the old one, we present some comments and suggestions for improvement. Acknowledgments. The authors would like to acknowledge the support of Zhang Jie, Peng Fang, Wu Guangyu, Xu Li Shuang, among others. This research is supported in part by the National Natural Science Foundation of China (under Grant No. 60373056), and the CAS Pilot Project of the National Knowledge Innovation Program (Grant No. KGCX2-YW-606).
References 1. Mai, N., Neo, T.K.: Multimedia in education: developing a student-centred learning environment in the classroom, http://www.vschool.net.cn/english/gccce2002/lunwen/ gcccelong/41.doc 2. He, K.: The challenges in the development of educational informationalism. China Educational Technology 6, 5–11 (2006) 3. Wang, D., Chiu, S.C., Wu, G., Dai, G., Wang, H.: Pen-Based Children’s Instructional Platform Based on Cognition Theory of Multimedia Learning. In: PDPTA 2008, pp. 742–748 (2008) 4. Sutherland, I.E.: SketchPad: A Man-Machine Graphical Communication System. In: AFIPS Spring Joint Computer Conference, pp. 329–346 (1963) 5. Li, Y.: Research on Pen-based User Interfaces—Theory, Technique and Implementation. PhD dissertation (2002) 6. Mayer, R.E.: Multimedia Learning. Cambridge University Press, Cambridge (2001) 7. Li, X., Atkins, M.S.: Early Childhood Computer Experience and Cognitive and Motor Development. Pediatrics 113(6), 1715–1722 (2004) 8. Michael, S., Yvonne, R., Frances, A., Matt, D.: Designing for or designing with? Informant design for interactive learning environments. In: CHI 1997 (1997) 9. Carrie, H., Brian, M.W., Darcy, D.G.: Theories meet realities: designing a learning game for girls. In: DUX 2005: Proceedings of the 2005 conference on Designing for User Experience (2005) 10. Calvert, Children Journeys Through the Information Age (1999) 11. WaWaYaYa, http://www.wawayaya.net/guide/
Development of a Visualised Sound Simulation Environment: An e-Approach to a Constructivist Way of Learning Jingjing Zhang1, Beau Lotto2, Ilias Bergstom2, Lefkothea Andreou2, Youzou Miyadera3, and Setsuo Yokoyama4 1
Department of Education, University of Oxford, UK [email protected] 2 Lottolab Studio, University College London, UK 3 Division of Natural Science, Tokyo Gakugei University, Japan 4 Information Processing Centre, Tokyo Gakugei University, Japan
Abstract. In this paper, the design and implementation of a visualised sound simulation environment is presented as an initial step to further laboratory experimentation. Preliminary laboratory experiments showed a positive learning curve in human auditory perception. Learning occurred when new information was processed with relevant existing knowledge in this simulation environment. While the work towards the truth of the empirical hypothesis is still under discussion, this project has been expanded beyond the scope that was originally envisaged and the developed environment showed its potential to be adopted on mobile devices for many educational purposes. This initiative not only brings scientists and educators together, but it is also hoped that it represents a possible e-approach to a constructivist way of learning. Keywords: visualisation, simulation, constructivist, learning, mobile, visual, auditory.
Development of a Visualised Sound Simulation Environment
267
information from arrays of stimulation that specify the events, layout and objects of the world”. In this paper, we put focus on parts of the learning process that look to be somewhat independent from conscious or critical forms of learning (such as learning to solve a mathematical problem) but that are involved with rather low-level acquisition through the central perception system. Perception has long been considered as a modular function, with the different sensory modalities working separately from each other (Schlottmann, 2000). Recent studies in neuroscience and psychology, however, challenge this view by suggesting that crossmodal interactions are in fact more common than originally thought (Martino & Marks, 2000). A number of experiments involved with cross modality have been carried out (e.g. Butler & Humanski, 1992; Carlile, 1990). Recently, research on sensory perception has become both popular and integrated with fast development of computer simulation technology. However, the fundamental problem for any sensory-guided system (natural or artificial) is that the information it receives from its environment is ambiguous. Particularly, most computer simulated sound systems fail under natural conditions, or when forced to contend with an environment that is not explicitly represented in the system. How natural systems resolve this challenge is currently not known, though one increasingly popular hypothesis is that the problem is resolved empirically. That is, the system encodes the statistics of its experience with past sources of stimuli1. Despite of the increasing support for this view, however, there is currently no one that has tested it. One reason is that to truly test this view we would have to provide someone with a wholly new sensory modality, and then look to see how that person’s brain deals with the underlying ambiguity of the information it receives. This is of course impossible. What is possible, however, is to present one of the senses with a new kind of statistical experience in a simulated environment. Therefore, a visualised sound simulation environment was designed and implemented, as an initial step to further laboratory experimentation. The environment presented the visual stimuli with a new auditory experience in order to prove the ambiguity problem in vision is resolved empirically. Preliminary laboratory experiments showed a positive learning curve in human auditory perception. Learning occurred when new information was process with relevant existing knowledge in this sound simulation environment. While the work towards the truth of the empirical hypothesis is still under discussion, this project is extended beyond its initial establishment and the developed environment shows its potential to be adopted in different real-world educational settings, e.g. a music lesson and a drawing course. This initiative not only brings scientists and educators together, but it is also hoped that it represents a possible e-approach to a constructivist way of learning.
2 Mapping Model A Mapping Model (M) was proposed for translating any pixel in a real 2D image to a unique virtual sound pixel of an imaginary sound panel in two continuous steps. That 1
Constructivists such as Helmholtz H. and Gregory R. argue that external world cannot be directly perceived because of the poverty of the information in the retinal images. Since information is not directly given, we have to interpret the sensory data in order to construct perception.
268
J. Zhang et al.
is, a real image pixel is mapped to a ‘visual pixel’, which is subsequently converted as a ’sound pixel’ by using this Mapping Model (M). M
=
{R, I, S,
,
}
The elements of Mapping Model (M), are justified as follows: R: a matrix, which represents the real image input (which can be anything from hand drawn images to photographs or streaming video). I: a
matrix, which represents an imaginary visual panel.
S: a a virtual sound panel.
matrix, which symbolises
: a mapping function from R to I. A scaled and projected representation I of the real image R is produced by this mapping function : Map any point , to , by:
: a mapping function from I to S. It creates a sound panel isomorphic to the visual panel: Map any point , to , by:
1 1 1
Fig. 1. Mapping Model - w is the width of the matrix R, while h is the height of the matrix R. xNum is the width of the matrix I; yNum is the height of the matrix I; (panMAX -panMIN) is the width of the matrix S; y(freqMAX -freqMIN) is the height of the matrix S.
As we can see, The Mapping Model shows the process of converting a real image pixel (Pr) in an image R to a visual pixel (Pi) on a visual panel I and then subsequentis a ly mapped to a sound pixel (Ps) on a sound panel S. The mapping function scale formula to trim the input of a real image into a required standard image, which
Development of a Visualised Sound Simulation Environment
269
can be then mapped to a virtual sound panel. The mapping function then converts , to a sound pixel with an each pixel of the product of the mapping function unique property, such as loudness, frequency, and pan.
3 System Design and Implementation The invention of the screen, either on a computer or a mobile device, has completely changed the traditional work of designing. However, while it adds more interactivity and flexibility into most of our products in the daily life, it also unavoidably brings in complexity and ambiguity. It is said that miscellaneous designs has given most users a difficult time to get used to the interface. Building upon the proposed model, the designed system updated from the first working prototype with pop-up panels to a unique three-tier structure for maximising such seamless translation. The final design was based on and has been further refined by the feedback obtained during the usability testing. The look of such design was thus simple and appealing to users.
Fig. 2. Three Hierarchy Design - - showing the three-tier architecture
The top architectural level of the system was a two part top-and-bottom skeleton (Fig. 2), which included (in the top part) a menu bar and a painting canvas (in the bottom part). The second architectural level within the ‘painting’ canvas also consisted of a top-and-bottom structure. On the top, there was a ‘Space Panel’, which included an ‘Image Space Panel’ and a ‘Sound Space Panel, and at the bottom a ‘Control Panel’ (i.e. an ’Image Control Panel’ and a ’Sound Control Panel’). The third level was outlined as a left-and-right structure contained within the painting canvas, separated as ‘image’ and ‘sound’ (from left to right) in both the ‘Space Panel’ and ‘Control Panel’. That is, on the painting canvas, there were four panels, from top left - an ‘Image Space Panel’, top right - a ‘Sound Space Panel’, bottom left – an
270
J. Zhang et al.
‘Image Control Panel’ for controlling the ‘Image Space Panel’, bottom right - a ‘Sound Control Panel’ to control the ’Sound Space Panel’. The four panels were independently resizable and can be switched on and off for maximising the screen of any device, such as a computer or a mobile phone. The ‘Image Control Panel’ allowed users to change the resolution of the above image panel and the sound panel. There was also an image colour chooser panel consisted of two parts: the Preview panel and the Colour palette. The Colour Palette consisted of four tabbed panels: Swatches, HSB, RGB, and Gray Scale. The Gray Scale was a slider bar which adjusted the colour intensity of image pixels. The ‘Sound Control Panel’ provided different selections of 128 MIDI sounds and 1 sine sound predefined for sound pixels above with the option to choose Loop or not. The system was implemented in Java, and so can be run as a standalone program on a local computer or an applet on the Web (the interface is shown in 2). The success of the developed system was tested in three methods: Code Reading testing, Black Box testing, and Integration testing. In addition, the system has been used to carry out several experiments to test the empirical hypothesis on nearly 20 subjects. Both the testing of the system and the results of the experiments indicated that the algorithms behind the Mapping Model successfully mapped simple visual images into sound ‘images’. It is suggested that it can be further developed and installed on mobile devices in the future. It is also believed that this sound simulation would work better on a touch screen and could receive better learning outcomes whilst on the move.
Fig. 3. System Interface (two screen captures) - showing the main windows of the user interface, which consists of four split panels with changeable size. They are: Visual Space Panel (on the top left), Sound Space Panel (the top right), the Visual Control Panel (on the bottom left), the Sound Control Panel (on the bottom right). The five menu bars are on the top of the interface: file, experiment, training, subject and help.
4 Laboratory Experiments Early work shows that vision dominates in multi-sensory processing. This means vision is always considered as active, but hearing is usually thought of as passive. However, research has shown that while vision may dominate spatial processing, hearing dominates temporal processing (Guttman, Gilroy, & Blake, 2005). Therefore, the success of testing the ’empirical’ basis of learning possibility by translating visual
Development of a Visualised Sound Simulation Environment
271
information into auditory information lies in the right design of appropriate experiments, which can take best advantage of the temporal processing of hearing. A series of experimental tasks were developed, and the data from such experiments can be analysed in a great many ways, resulting in the discovery of many unknown aspects of learning. In this paper attention focuses mainly on two sets of experimental tasks accomplished by eleven subjects. There were 6 males and 5 females of them, ranging in age from 20 to 30 years old. The hearing abilities of the 11 subjects were at average level. 4.1 Experiment 1 - Sound Localisation The sound localisation experiment was designed to measure the ability of the brain to process spatial information. This experiment differed from other sound localisation experiments in several ways. Firstly, earlier experiments always used special facilities to organise the experiment such as geodesic spheres housing an array of 277 loudspeakers (e.g. Hartmann & Acoust, 1983). This increased the cost of experiments and decreased the practicality. In contrast, this experiment used a computer simulated system, which can automatically accomplish the experiment without manual input and supervision. In addition, the system was developed in Java and can be easily extended to a mobile device in the future. Secondly, real sound sources (e.g. speakers) were usually used in other experiments earlier whereas our experiments used virtual 2D sound space, in which each ’sound pixel’ represented a unique virtual sound source thereby again saving on cost, practicality and time. Finally, unlike 3D cave-like virtual environments where sound localisation system involved with head-related transfer functions (HRTFs), each image pixel in this developed system was translated directly to a sound pixel. To add more detail, in the horizontal direction, the interaural time difference (i.e. Inter-aural level difference and Inter-aural time difference) was used to determine horizontal position. In the vertical direction, frequency was used to perceive vertical position in this experiment. These two parameters are used to distinguish each ‘sound pixel’ in experiment with respect to the Mapping Model (M). All the mouse-click responses were recorded and stored in a text file. In the series of this experiment, the resolutions of the sound panel increased from 3 × 3, 7 × 7, to 11 × 11. In each one, four different combinations of trials were used to
Fig. 4. Experiment 1 Sound Localisation - the Square with gray colour is the square where the subject clicked to predict the location of the sound
272
J. Zhang et al.
Fig. 5. Results of Experiment 1 - showing the total number of correct clicks from step 1 to step 5. The step 1 and step 5 were the test stages, while the step 2, 3, and 4 served as the training. The number of correct clicks of the step increased after 3 different kinds of training. The step 4 (adjacent sound with a reference sound) achieved the best results. The number of correct clicks decreased as the sound space dimension increased.
explore a better pattern for identifying sound. Subjects were first introduced to a random sound, and then were given a reference sound from the pixel in the centre of the sound panel followed by a random sound. After this, subjects were given a series of sounds which were adjacent to each other. Then, subjects were given a reference sound followed by an adjacent sound to it. Last, subjects were provided by a random sound again to find out whether learning occurred from the training. As we can see in Fig. 5, the sound localisation ability is clearly improved with training. Subjects were also better able to locate sound if given an initial reference sound. The chart on the right shows that the ability of the subjects to locate sound decreased with increased resolution. By comparing the ability of identifying sound horizontally and vertically, if the space dimension is smaller than 7 × 7, subjects can find the horizontal position of a sound better than its vertical position in the space; if more than 7, it seemed to be easier to identify a sound in the vertical axis. As expected, the subject’s ability to specifically locate sound correctly in the 11 × 11 space was very low. Despite this, as shown in Fig. 6, while the absolute position was incorrect, subjects were very good at finding the relative location of the sound source, given the previous test sound. Thus, in deciding on the location of the pixels in this sound space, subjects used their previous responses to previous stimuli in order to estimate the positions of future test sounds. This indicates that the human auditory system predicts new information based on past information. Thus the past information can be used to aid in the ambiguity that they experience when trying to predict the new test sound, and enhance the current attempt. The positive parts of the experiments have shown that humans are able to learn from the wholly new translated sensory information (as shown by the increasing learning curves from the sound localisation experiment data). We suspect a long-term training process would help to better develop subjects’ visual and audio sense with this new kind of audio-visual experience.
Development of a Visualised Sound Simulation Environment
273
Fig. 6. Total Number of Correct Clicks (General Pattern of Movement) – The red numbers show the sequential locations of the test sounds sound generated by the experiment. The black numbers represent an example subjects clicking responses.)
4.2 Experiment 2 - Object Recognition While Experiment 1 was based on one sound, the object recognition experiment was further designed to increase the sound sources. Nine 20 × 20 simple black and white images, consisted of the same number of pixels (and therefore of the same overall intensity), were used. The images were randomly loaded from a local folder on a computer. There was the same number of white pixels in each image. Pixels in white were the one able to produce sound simultaneously, while pixels in black were silent ones. Subjects were first introduced by a compound sound from a random image, and asked to pick up a right image to pair with the sound. They were then presented again with a compound sound with feedback. That is, if subjects chose a wrong image, the right image would be highlighted to correct them. Then, an identical image panel would be switch on automatically on the left to allow subjects to mark a small area (in which sound pixels were able to be generated) and navigate to different directions. Sound would be generated if subjects moved onto corresponding white pixels hidden underneath. Last, subjects were presented with a random compound sound from images again to test their learning experiences. It is clear that the ability of object recognition without training was poor (16%). However, their ability to recognise the object increased sharply when they were given the opportunity to explore it (72%), which increased further after four trial explorations (where the subjects were correct 100% of the time). The statistical data also showed that motion aided in recognition of the masked object as opposed to polyphonic compound sounds generated by the static object. That is, motion was easier to recognise than the static object. By moving around, the brain could construct an internal map of the image from experience. Although it is clear that in moving the rectangular window around in a sometimes arbitrary fashion, only a temporary memory was utilised to intuitively construct a mental model of the 2D environment. We believe that without using any exploration of the masked Sound Space to obtain temporary experience and perceptions, the brain instead needs the experience stored in the brain by long-term training.
274
J. Zhang et al.
Fig. 7. Experiment 2 – the interface on the left shows the object recognition experiment in Step 1, 2 and 4. The interface on the right shows the object recognition with exploration in step 3. There is the masked image panel on the left containing one of the nine images on the right hand side underneath. The subject can drag a rectangle by mouse and move it by pressing the up, down, left, right arrow keys to explore this image. After exploration, the subject clicked the one he/she thought was correct on the right hand side. The red image button then showed the correct answer.
5 From Laboratory to Real World As stated earlier, this study, was at first intended to be a three-year project in a neuroscience laboratory to prove the empirical vision theory, but later expanded beyond the scope that was originally envisaged, as the adoption of this developed sound simulation in real educational settings became increasingly feasible. Especially, when mobile technologies hold out the promise of ubiquitous leverage to this Java-based application, the study moved on to the discussion of the social values of such sound simulation on mobile devices. Statistical data showed that global mobile phone use was set to pass 3.25 billion by 2007 (Reuters, 2007) - around half the world's population. With regard to this, it is widely accepted that mobile and ubiquitous computing can genuinely support learning, but it is crucially important to ask how such simulated environment can be combined with mobile technologies to be immersed into various kinds of learning experiences in different contexts. It is a very complex undertaking to design interfaces for mobile devices. Visual interfaces have unpleasant limitations in small-screen devices, as they have a confined space on which to display information (Rinott, 2004; Smith & Walker, 2005; Walker & Lindsay, 2006). While mobile devices were getting smaller and lighter, this developed environment with its three-tier structures was able to switch on and off four different panels in the real time. That is, it potentially enlarged the small screen four times. Such an interface with layered view also promised seamless switching between different layers to a great extent of graphic rending. More importantly, the designed interface was sonically enhanced, and required less visual attention. The use of audio feedback thus to certain extent augmented the display of a mobile device, and helped users to be able to navigate in unfamiliar interface more easily. Not only can the interface of a mobile device be improved by this developed sound visualisation simulation environment, but also can the created applications widely be used for many educational purposes. Visual and auditory sensory are closely involved with each other. On the one hand, hand drawing exercises with sound in this
Development of a Visualised Sound Simulation Environment
275
developed environment provided a good opportunity to bring sound or music into the drawing practice. On the other hand, the database of 128 midi music sounds in two dimensions (frequency and pan) enhanced the traditional music teaching. The concise design integrated with drawing function was likely to be of more appeal to learners. More importantly, the environment held a record of every single experiment in the background, and was capable of providing a learning curve or analysis. Furthermore, the possibility to tailor learner-centred experiments by learner themselves gave this environment a great deal of flexibility and openness. It allowed a healthy cycle of further development in the future.
6 Conclusion The developed sound visualisation environment was able, to a certain degree, to translate the visual stimuli into sound stimuli although we did encounter some shortcomings in the experimental implementation. Furthermore, this system is planned to be implemented on mobile devices. It is believed that this sound simulation would work better on a touch screen and could result in better learning outcomes whilst on the move. This initiative not only brings scientists and educators together, but it is also hoped that it represents a possible e-approach to a constructivist way of learning.
References 1. Butler, R.A., Humanski, R.A.: Localisation of Sound in the Vertical Plane with and without High-frequency Spectral Cues. Percept. Psychophys. 51, 182–186 (1992) 2. Carlile, S., King, A.J.: Monaural and Binaural Spectrum Level Cues in the Ferret: Acoustics and the Neural Representation of Auditory Space. Journal of Neurophysiology 71 (1994) 3. Dewey, J.: How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process (New edn.). D.C. Heath, Lexington (1933) 4. Gibson, E.J.: An Odyssey in Learning and Perception. MIT Press, Cambridge (1991) 5. Guttman, S., Gilroy, L., Blake, R.: Hearing What the Eyes See: Auditory Encoding of Visual Temporal Sequences. Psychological Science 16(3), 228–235 (2005) 6. Hartmann, W.M., Acoust, J.: Localisation of Sound in Rooms I. Soc. Am. 74, 1380–1391 (1983) 7. Martino, G., Marks, L.E.: Cross-modal interaction between vision and touch: the role of synesthetic correspondence. In: Perception 2000, vol. 29, pp. 745–754 (2000) 8. Mayer, R.E.: Memory and Information Processes. In: Weiner, I.B. (ed.) Handbook of Psychology, vol. 07. Wiley, Hoboken (2003) 9. Purves, D., Lotto, R.B.: Why We Wee What We Do: An Empirical Theory of Vision (November 2002) 10. Reuters: Global Mobile Phone Use to Pass 3 Billion (2007) 11. Rinott, M.: Sonified Interactions with Mobile Devices. In: Proceedings of the International Workshop on Interactive Sonification, Bielefeld, Germany (2004) 12. Schlottmann, A.: Is perception of causality modular? Cognitive Sciences 4 (2000) 13. Smith, D.R., Walker, B.N.: Effects of Auditory Context Cues and Training on Performance of a Point Estimation Sonification Task. Applied Cognitive Psychology 19(8) (2005) 14. Walker, B.N., Lindsay, J.: Navigation Performance with a Virtual Auditory Display: Effects of Beacon Sound, Capture Radius, and Practice. Human Factors 48(2), 265–278 (2006)
Causal Links of Presence Donghun Chung1 and Chae-Hwan Kim2 1
School of Communication Kwangwoon University 447-1 Wolgye-Dong, Nowon-Gu, Seoul Korea 139-701 [email protected] 2 Department of Mass Communication and Journalism Tongmyoung University [email protected]
Abstract. The purpose of this paper1 is to examine antecedent variables and an outcome variable of presence. Presence has been used to explain the extent to which technology users are immersed and involved in a technology-created experience. In video gaming, gamers frequently don’t distinguish between reality and the game world, and they identify characters with themselves. This comes from a high level of presence. So what makes technology users have greater presence? The present study proposes a causal model which includes attitude and empathy as antecedent variables that lead to a degree of presence and then para-social interaction as an outcome of presence level. The results showed that path analysis of the model was successfully supported. Keywords: Presence, Attitude, Empathy, Para-Social Interaction, Wii.
1 Introduction Presence is a psychological state or subjective perception in which, even though part or all of an individual's current experience is generated by and/or filtered through human-made technology, part or all of the individual's perception fails to accurately acknowledge the role of the technology in the experience [1]. Actually, there are many different conceptual and operational definitions of presence, and scholars both within the same field and in different fields use their own terminologies. Yet these terminologies are for the most part interchangeable. Many scholars commonly agree that presence is, at least on a fundamental level, the perception of non-mediation [2]. Many scholars have examined causes or effects of presence [2][3][4][5][6]. However, only a few studies have revealed causal relationships of presence with some antecedent variables. This is important to know because when we find out reliable variables that increase or decrease the level of presence, we can manipulate them. Consequently, we can find out what outcomes presence influences and how it does so, for presence is not an ultimate outcome itself. Therefore, presence has an important role as a mediated or moderated variable. 1
The present Research has been conducted by the Research Grant of Kwangwoon University in 2008.
In the development of state-of-the-art video game technology, people are more likely to experience presence. There are a few reasons why gaming is important in presence research. First, gamers are active users. According to uses and gratification theory, most game players have motivations which lead to spending more time playing games and other activities [7][8]. Since they are active players, it is easy to be immersed in the gaming situation. Second, gaming has developed very quickly and dramatically. Many aspects of gaming have increased in depth and complexity, including the game story, graphics, characters, background, and even controllers, and these are all significant factors in influencing the degree of presence. Third, interactivity has been shown to be an important function in gaming. Interaction definitely plays an important role in presence [2]. Fourth, the systems on which games are played have also been developed. For instance, the modern TV set provides game players with clear images, high resolution (even HD quality), big screens, and vivid colors in console games. Computer monitors offer the same advantages in computer games. In addition, modern surround sound stereo systems enable a fully immersive auditory experience with crystal-clear quality. These are all significant factors affecting presence. Finally, based on the previous results, game research has been successfully performed. Many studies have raised presence issues using games; for instance, aggression [9], arousal [5], social relationship [10], violence [11], etc. In the present study, we will investigate causal relationships among attitude, empathy, presence, and para-social interaction in gaming. These variables directly or indirectly linked with one another in the previous literature. More rationales will be described in the next section.
2 Proposed Model Witmer and Singer [6] assert that involvement and immersion are necessary for experiencing presence. Involvement depends on focusing one’s attention and energy on a coherent set of stimuli while immersion depends on perceiving oneself as a part of the stimulus flow. More specifically, involvement is defined as a psychological state experienced as a consequence of focusing one’s energy and attention on a coherent set of stimuli or meaningfully related activities and events. Although immersion is a psychological state like involvement, it is characterized by perceiving oneself to be enveloped by, included in, and able to interact with an environment that provides a continuous stream of stimuli and experiences. They propose that a valid measure of presence should address factors that influence involvement as well as those that affect immersion. Many studies have investigated causes and effects of presence. One of the more interesting studies was Cummins’ dissertation [12] which examined the effects of direct address on empathy, interactivity, presence, and entertainment value in reality television shows. He found significant causal links among empathy, presence and para-social interaction as a partial path model. In his model, he disclosed that empathy as a predictor leads to presence and subsequently para-social interaction in watching television entertainment programs. This model can be applied in the gaming study because gaming also has entertainment value as an ultimate goal. Unlike Cummins’ model, this research has
Causal Links of Presence
281
attitude as an antecedent variable. A few reasons exist as to why attitude plays an important role. First, as a driving force, attitude works as a predisposition. Although it is hard to have one universal definition of attitude, the central ideas are readiness for response and motivation [13]. This is a preparation for behavior. Second, as an evaluation, attitude leads to involvement or attention. Attitude can be defined as favorableness or unfavorableness which ultimately decides a person’s further psychological disposition or behavior. Lastly, according to previous attitude and empathy literature [14][15], there was a significantly positive relationship between the two variables. From the extensive literature, the present research has the following proposed model . This model explains that 1. attitude towards a game character influences empathy with it, 2. empathy with a game character influences presence, and 3. presence influences para-social interaction with a game character. Ultimately, because these relationships are interrelated, the fit of the causal model will be tested. Attitude
Empathy
Presence
Para-Social Interaction
Fig. 1. Proposed path model
3 Method 3.1 Participants All of the participants were women and non-game players. Before they joined this project, they answered Witmer and Singer’s (1998) immersive tendency measure and were found to be homogeneous. Fourteen participants joined this project. Ages ranged from 19 to 25 and the average age was 21.36 (SD=1.55). 3.2 Procedure All participants filled out a consent form and then entered into a game lab. A researcher explained how to play the tennis game in the Nintendo Wii. The researcher showed them how to serve, do a forehand, and do a backhand. They were asked to practice the tennis game for 5 minutes as a training session. All of the lights were turned off to get a more immersive experience. The researcher helped guide them during that time. After confirming that no problems existed, the researcher restarted the game and let them play alone. After 15 minutes of playing, the game was stopped and the participants were given a main questionnaire that asked about their attitude toward character, presence, empathy, and para-social interaction. 3.3 Instruments The questionnaire was mainly composed of four parts: attitude toward character, empathy, presence, and para-social interaction. Attitude toward character had four items: “useless/useful,” “unimportant/important,” “foolish/wise,” and
282
D. Chung and C.-H. Kim
“unpleasant/ pleasant.” This measure simply asked participants’ general feeling of favorableness or unfavorableness for their characters and five-point semantic differential scale was employed. One item (unimportant/important) was deleted. (M=3.45, SD=0.67) and it was found reliable (α=.70). Second, eight questions were asked to measure empathy defined as feeling the same way that an observed character is feeling. Two items were deleted and it was reliable (M=2.75, SD=0.72, α=.85). Third, presence was composed of two parts: involvement and immersion. Fourteen questions were provided and all of the items were retained and reliable (M=3.03, SD=0.83, α=.93). Lastly, para-social interaction was measured. Para-social interaction, which is the imaginary one-way relationship that viewers develop with people on television, was coined by Horton and Wohl [16]. Para-social interaction between gamers and game characters was measured by eight items and all of the items were retained and reliable (M=2.54, SD=0.93, α=.92). 3.4 Gaming System The Nintendo Wii was chosen for this research. The Wii is what emerged from an attempt to change the way video games are perceived. The Wii has several significant differences from the other next-generation gaming platforms. For one thing, it is very small, at 8.5 cm x 6 cm x 2 cm and 3.84 lbs. It also has wireless connectivity.
Fig. 2. Overview of the experimental condition
Causal Links of Presence
283
The most innovative feature of the Nintendo Wii is the controller, called the Wii Remote. It contains sophisticated motion-sensing technology that enables a variety of gaming functions. Gamers can swing the controller like a tennis racket to play a tennis game, grab the controller with both hands and steer it like a steering wheel, or point and shoot in first-person shooting games. With an additional controller connected to the Remote, gamers can box an opponent by engaging in a punching and blocking motion using both hands. The Remote also contains other features that may contribute to a more immersive experience. It has a rumble feature to supply kinesthetic feedback. It also has a small speaker built into the controller, adding an auditory component to a player’s movements. A LG 42 inch LCD TV was used with the Wii. It is 46.3 x 30.2 x 11.8 (in) and 90.4 (lbs) with the stand. The resolution is 1366 x 768 (Dot) and the television system is NTSC-M, ATSC, 64 & 256 QAM. For better sound, a Panasonic SA-HT940 home theater system was used. It has 5 +1 channels.
4 Result Path analysis was used to test the fit of the causal model. The AMOS program was used to obtain estimates of global fit. Various fit indices such as chi-square, the Comparative Fit Index (CFI), the Goodness of Fit Index (GFI) and the Root Mean Squared Error of Approximation (RMSEA) were examined. Hu and Bentler [17] note that CFI values range from 0 to 1.0, with values greater than .90 indicating close fit. A GFI of greater than .9 is conventionally considered to indicate an acceptable fit. With RMSEA, values less than .05 indicate close fit. In addition, the obtained chi-square value is compared with the predicted value and the resulting p value is provided. Overall, the fit of this model to the data with the AMOS analysis was good (χ2(3) = 1.82, p > .05, CFI = 1.000, GFI = .942, RMSEA = .000) . Table 1 presents the zero-order correlations of the study variables.
Attitude
Empathy
.67**
Presence
.70**
Para-Social Interaction
.63*
Note. ** indicates p < .01, * indicates p < .05.
Fig. 3. Obtained path model Table 1. Zero-order Correlations of the Study Variables Attitude Attitude 1 Empathy 0.633* Presence 0.517 Para-social interaction 0.490 Note. * indicates p < .05, two-tailed.
Empathy
Presence
Para-social interaction
1 0.603* 0.474
1 0.559*
1
284
D. Chung and C.-H. Kim
5 Discussion This study aimed at disclosing causal links of presence composed of attitude, empathy, and para-social interaction. More specifically, the causal model proposed that attitude influences empathy, empathy influences presence, and presence influences para-social interaction. Since they were not independent of each other, path analysis was performed to test the causal model. To test the proposed model, participants played Nintendo Wii tennis games. Since the Wii system has unique interactive functions, we expected gamers to experience a greater sense of presence. Results indicated that all hypotheses were supported and the data were judged to be consistent with the model. Each causal link has significant implications. First, the relationship between attitude toward a game character and empathy with it is significantly positive. Attitude is simply a favorable or unfavorable manner toward the game character in this research. This link confirmed that the more positive the attitude toward a game character, the greater empathy with it. Containing affective, cognitive, and behavioral components, attitude influences a person’s psychological traits, attributions, characteristics, and behaviors. Does having a positive or negative attitude toward the game character influence intent to play the game? What if gamers have negative attitudes toward the game character? Are they going to play it? They may want to change their characters or even the game. For emotional reconciliation, the role of attitude toward the character may be as important as stories, images, and so on. Of course, attitude also plays an important role in presence research because the degree of involvement and immersion in the game environment partially comes from people’s characteristics. This will be further described in the empathy and presence relationship. A question about attitude toward the character will be developed as above. Therefore, we need to know gamers’ or technology users’ attitude toward an object when we have individual characteristics variables. Second, empathy and presence have a significant positive relationship. It means that the greater the empathy with a game character, the greater the sense of presence. Many presence studies have been interested in particular technological attributes as causes of presence; for instance, screen size, resolution, interactivity, and so on. However, as Cummins [12] pointed out, this relationship revealed that individual characteristics can influence presence as well. More specifically, an individual’s ability to engage in empathic processes was a significant predictor of presence. Empathy in this research was defined as feeling the same way that an observed other is feeling (Hoffman, 1975). Of course, an observed other means the game character here. Since gamers had a positive attitude toward the game character, they were drawn into the empathic development, and subsequently empathic engagement increased immersion and involvement in the game. In order to have homogeneity, the present research filtered out samples using gender and game experience as well as Witmer and Singer’s immersive tendency scale [6]. Even after this filtering process, individual characteristics predicted the level of presence experienced. Future studies will find out more promising attribute variables for presence. Lastly, there is a significant positive relationship between presence and para-social interaction, meaning that the greater the experience of presence, the greater the parasocial interaction with a game character. Simply speaking, para-social interaction
Causal Links of Presence
285
means gamers’ perceived or psychological interaction with game characters. Lombard and Ditton [2] explained that para-social interaction had mostly been discovered with TV characters and less empirical evidence of it existed with computer characters. As a matter of fact, little research has empirically examined the relationship between presence and para-social relationship. Therefore, this study demonstrates evidence that presence leads to para-social interaction with the game character. In addition to the outcome of presence, para-social interaction should be investigated concerning what this will bring about as a predictor and an influence of presence. This study has some limitations by nature. First, the causal links are too simple. There were neither moderated variables nor multiple exogenous and endogenous variables. Although it is an advantage that the simple relationships clearly explain the outcomes, it cannot explain various causes and effects of presence. Second, the sample size is too small to perform path analysis. However, generally speaking, the reason that we need relatively big samples (about 200) is to detect a difference between the model and the data because standard errors are large in the small sample size. In this research, although small samples were provided, the path model was successfully supported. Finally, it is necessary for future research to find out various cause-and-effect variables of presence that are not limited to the technology (or its functions) but psychological attributes, characteristics, and traits.
References 1. International Society for Presence Research.: The Concept of Presence: Explication Statement (2000), http://ispr.info/ (Retrieved February 10, 2007) 2. Lombard, M., Ditton, T.: At the Heart of It All: The Concept of Presence. Journal of Computer-Mediated Communication 2 (1997) 3. Chung, D.: Something for Nothing: Understanding Purchasing Behaviors in Social Virtual Environments. CyberPsychology & Behavior 6, 538–554 (2005) 4. Skalski, P., Tamborini, R., Glazer, E., Smith, S.: Effects of Humor on Presence and Recall of Persuasion Messages. In: 56th Annual International Communication Association Conference, Dresden, Germany (2006) 5. Tamborini, R., Westerman, D., Skalski, P., Weber, R., Kotowski, M., Chung, D.: Repeated Exposure to Virtual Video Game Violence: Presence and Hostile Thoughts. In: The 91st annual meeting of the National Communication Association, Boston, MA (2005) 6. Witmer, B.G., Singer, M.J.: Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 3, 225–240 (1998) 7. Jansz, J.: Gaming at a LAN Event: The Social Context of Playing Video Games. New Media & Society 3, 333–355 (2005) 8. Sherry, J.L., Lucas, K., Greenberg, B.S., Lachlan, K.: Video Game Uses and Gratifications as Predictors of Use and Game Preference. In: Vorderer, P., Bryant, J. (eds.) Playing Video Games: Motives, Responses, and Consequences, pp. 213–224. Lawrence Erlbaum Associates, New Jersey (2006) 9. Nowak, K.L., Krcmar, M., Farrar, K.: Examining the Relationship Between Violent Video Games. Presence, and Aggression. In: The Presence Conference, Cleveland, Ohio (2006)
286
D. Chung and C.-H. Kim
10. Lee, K.M., Park, N., Song, H.: Can a Robot Be Perceived as a Developing Creature? Effects of a Robot’s Long-Term Cognitive Developments on Its Social Presence and People’s Social Responses Toward It. Human Communication Research 4, 538–563 (2005) 11. Tamborini, R., Eastin, M., Lachlan, K., Skalski, P., Fediuk, T., Brady, R.P.: Hostile Thoughts, Presence and Violent Virtual Video Games. In: The 51st annual convention of the International Communication Association, Washington, D.C. (2001) 12. Cummins, R.G.: The Entertainment Appeal of Reality Television: The Effect of Direct Address on Empathy, Interactivity, Presence, and Entertainment Value. Unpublished Dissertation, University of Alabama, Tuscaloosa, AL (2005) 13. Allport, G.W.: Attitudes. In: Murchison, C. (ed.) A Handbook of Social Psychology, pp. 798–844. Clark University, Worcester (1935) 14. Ascione, F.R.: Enhancing Children’s Attitudes about the Humane Treatment of Animals: Generalization to Human-Directed Empathy. Anthrozoos 3, 176–191 (1992) 15. Bagshaw, M., Adams, M.: Nursing Home Nurses’ Attitudes, Empathy, and Ideologic Orientation. International Journal of Aging & Human Development 3, 235–246 (1986) 16. Horton, D., Wohl, R.R.: Mass Communication and Para-Social Interaction: Observations on Intimacy at a Distance. Psychiatry 3, 215–229 (1956) 17. Hu, L.-t., Bentler, P.M.: Evaluating Model Fit. In: Hoyle, R.H. (ed.) Structural Equation Modeling: Concepts, Issues, and Applications. Sage, Thousand Oaks (1995)
Games Design Principles for Improving Social Web Applications Ines Di Loreto DICO – Dept. of Informatics and Communication Università degli Studi di Milano, Via Comelico, 39, Italy [email protected]
Abstract. Most young people (at least in countries were social communication technologies are established from a long run) carry mobile devices, surf the Internet and download music. They are always connected and live in a world where the distinction between virtual and real fade. The design of new interfaces becomes, in this context, a complex activity that involves a series of methodological problems. To one side designers have to create interfaces using basic HCI principles, on the other side they have to merge them with others ICT principles able to support social aspects, bearing in mind that they are addressing the above described generation of younger. Is our opinion that designers can find useful suggestion in game design strategies. Looking at games interface design choices, in fact, can help software engineers to improve the usability of other types of - more conventional - applications. In order to demonstrate this we will compare two social web applications: the new Facebook website and a French online game, Hordes. Keywords: Game Design, HCI, Social Interaction.
Y”1, that include people born between 1981 and 2000. However this way of partitioning generations doesn’t works anymore. Digital natives [2] cannot be defined only through vital statistics, but through a set of characteristics and common experiences. Most young people in many societies (at least in countries were social communication technologies are established from a long run) carry mobile devices, surf the Internet and download music. They are always connected, with a preference for multitasking, and using technology in new ways. For instance they use technologies for staying always in touch with friends trough Instant Messaging services like Messenger and Skype, for finding old friends lost over time (Facebook), for sharing any kind of material (e.g., using Flickr and YouTube), and so on. This “generation” explore the world in an entirely new way as they interact with these technologies: they live in a world where the distinction between virtual and real fade. As Palfrey and Gasser claim, digital natives “are connected to one another by a common culture. Major aspects of their lives- social interaction, friendships, civic activities – are mediated by digital technologies. And they’ve never known any other way of life.” [3]. We can say that digital natives are characterized by a set of common practices, including the amount of time they spend using digital technologies, rather than by their age. However, being familiar with these technologies doesn’t necessarily implies a deep understand of the used technologies nor implies the desire to understand them. In fact, for digital natives Internet and cell phones are “goods” as for other generations were Radio and TV. What this scenario imply for interface design? As for the previous generations, the User Interface digital natives use influences significantly their impression of any application. However, is our opinion that, because of the pervasive role digital technologies (consoles, internet games, mobile devices, social networks) play in the early life of digital natives, the user interface they are used to see and interact with in these early moments influence their expectations about the interface of any other application they will eventually interact with. Studies at the Pew Internet & American Life Project [4] show that virtually all college students play video, computer or Internet games and 73% of teens do so (as a result, for example, they become accustomed with a style of learning that takes place informally[13]). This game adoption rate (and its consequences) is, in our opinion, of crucial relevance in order to understand new users expectations. For example, observing that players become frustrated or confused if interfaces don’t give them the right feedback and the right amount of control, we claim that new generation of users (that are often, as we have already said, technologically literate, but that does not necessarily are media literate) feel frustrated if they don’t have the right amount of control over their computer-use experience. The design of users interfaces becomes then -more than before because of the above described scenario- a complex activity. In fact, on the one side designers have to create interfaces using basic HCI principles -mostly applied to single humanmachine interaction, on the other side, they have to merge them with others ICT 1
Generation Y is a cohort which consists of those people born after the Generation X cohort. Its name is controversial and is synonymous with several alternative names including The Net Generation, Millennials, Echo Boomers, and iGeneration.
Games Design Principles for Improving Social Web Applications
289
principles, able to support social aspects. It is our opinion that, designers of social interactive systems, as social web applications are, can find some useful suggestion in game design strategies. In fact, we believe that looking at games interface design choices can help software engineers to find useful guidelines for addressing interaction problems (as we will show in par.2) and for improving the usability of other types of (more conventional) applications. 1.1 Game Design and HCI Firstly it is worth recognizing that most of the applications used by digital natives in the above described scenario well fit in the framework of computer-mediated human interaction (i.e. the design of human-computer interfaces that will be used for social interaction). We can think, for example, to social networking software applications in web environments (Facebook, MySpace, and the like – Facebook is also accessible through cell phone), but also to consoles (Windows Xbox live community, Nintendo Wii and Mii channels, and PlayStation Home). Finally, online collaborative game play (e.g. Everquest and World of Warcraft) has increased dramatically over the last five to ten years with the growth of usage of the internet. The phenomenon has led to over 4 million users of Everquest worldwide and 6 million users of World of Warcraft.[5] Because of the focus on user performance and user satisfaction, a lot of interesting ideas about new ways of “communication” between users and interfaces arose in games in order to fulfill users expectations. Game designers face the challenge of creating games that can be easily learned, effectively played, and emotionally enjoyed by gamers (otherwise the game will not sell). As Ye and Ye claim in [6] no guidelines or well founded principles have been established to help game designers do their work. With limited theoretical foundations and little research or data on gamers, they have to mainly rely on their intuition and experience in order to find solution to their problems. Most of these solutions, are practical, non conceptualized, solutions, which, however, fit perfectly into HCI principles. Moreover, they are more appealing for new generations because of their look and feel. Without any ambition of completeness, we believe worth considering the following basic criteria used in game design: 1. Ease the learning curve. Players - as users often do in other settings - do not read the manual, so game designers have developed a quite sophisticated approach to ease the learning curve of a game (and appeal to a more casual gamer). On the one hand, they present the basic instructions as part of the game, on the other hand, they hide instructions in the interface itself, making them available on demand. 2. Support modifications of the UI. Game designers enable the gamers to modify and extend their UI and consider this possibility a fundamental feature of the system. For instance, most games allow the player to disable interface features in order to increase the size of the action window. 3. Avoiding information overload. Presenting too much information and features to the players all at once causes many players to give up on a game. So, quite often, designers hide non-essential information in places that the player can access when
290
I. Di Loreto
needed. Since the interface is part of the feedback mechanism, they make sure that the players cannot only access information they need –only when they need it - but also that the information they receive is useful.[7] In this paper we use the above mentioned criteria to analyze help facilities in web environments, comparing them with help facilities in games environments. In particular, we focus on metalinguistic signs[8]- signs that communicate a message about the use of the application itself - and we analyze in which way these signs in existing websites can be improved using games interfaces ideas. In particular, we consider the new Facebook (www.facebook.com) interface and the interface of a French online game, Hordes (www.hordes.fr). This choice is due to the fact that both of them - in different ways - can be seen as social web applications, i.e., applications characterized by dynamic public content that changes on the basis of the input of many people. Facebook private home pages are exactly this. In fact, each homepage is the result of friends posts, comments, and others kinds of inputs (video, links, and so on). In the same way, each game session in Hordes is the sum of the dynamic “real” game part, plus the overall discussions about strategies (we will explain this more in depth in par.4). 1.2 Contextual Help in an Online Social Networking Site: Facebook Facebook is a popular social networking website that is owned by Facebook, Inc. People can add friends, send them messages, and update their personal profile to notify friends about themselves. The website currently has more than 150 million active users worldwide [9]. Facebook has a number of features for people to interact with. They include the Wall, a space on everyone profile page that allows friends to post messages for the user to see; Photos, where participants can upload albums and photos; Status, which allows people to inform their friends of their whereabouts and actions, and so on. On September 2006, a new feature, News Feed, was introduced, which appears on everyone homepage and highlights information including profile changes, upcoming events, and birthdays related to the user's friends. Initially, the News Feed caused dissatisfaction among Facebook users: most of them were concerned it made it too easy for other people to track down individual activities (such as changes in relationship status, events, and conversations with other users)[10]. In response to this dissatisfaction, developers included appropriate customizable privacy features. Since then, users have been able to control, in all the Facebook applications, what types of information they want to share with friends.[11] Bearing in mind the above mentioned games design principles, we can claim that this experience demonstrate that Facebook designers understood that people need multiple levels of detail and personalization in their applications (the second principle) and let people customize their interface. On July 2008, Facebook introduced "Facebook Beta", a redesign of the user interface (one for all: profiles were separated into tabbed sections, and an effort was made to create a "cleaner" look). This upgrade can also be seen as another attempt to give users a (little) more customizable interface. However if we look at the way Facebook implemented in this occasion the help facilities (essential for the transition from the old to the new interface), we find that these feature are totally unaware of the
Games Design Principles for Improving Social Web Applications
291
concept of different level of details. In fact, the developers used a main (textual, static) help (Fig.1) where the users can find what they need by search (however, the link to the help is well hidden in the bottom of the page). Moreover, for a short amount of time, a set of contextual (textual and static) little windows of helps have been available for the new added features (Fig.2). Contextual help can only be closed, but not being opened again if closed by accident. In this way developers not only were not able to lower the learning curve – the first criteria we listed - but they also weren’t coherent in their design choices. While they succeeded in avoiding the information overload, they failed providing the right amount of information. New users’ first impression of Facebook was of a complicated service with a complex interface. 1.3 Contextual Help in an Online Game: Hordes The principles we listed in par.2 were draw basing on “classical” games, with a dedicated client. But what happens to these principles when the client is the web browser? We will try to understand it through the analysis of a website game, Hordes. Hordes is a game create by Motion Twin, a provider of online games for the French-speaking public. Created in 2001, Motion Twin has 7 million accounts, divided into more than 40 casual minigames, a word game, a RPG, a multiplayer online portal, an arcade platform game, and so on[14]. The game is a strategic game played entirely in a browser. However, unlike the majority of web sites games, Hordes is a social game, where everything depends on the relationship each gamer establishes with other players.
Fig. 1. Classical help in Facebook
292
I. Di Loreto
Fig. 2. Interactive help in Facebook
In a few words, Hordes is a game of survival in a world invaded by zombies. The main goal is to take the city you are living in as long as possible, constructing various buildings and defenses to deal with zombies attacks. By registering, the player joins a town of 40 players (the great majority of the players don’t know each other). To maintain the town as long as possible, players must learn to cohabit and to aim towards a common goal. In order to organize themselves, citizens can use a set of tools. The most important one is a forum where every player can freely post. It’s very interesting to note that the town’s forum reflect different social behavior: some of the players (without a top-down decision or an election) candidate himself as major, deciding the objectives of the town; others refuse the authority; others again report negative actions of the neighbors, and so on. In this case we are talking about a particular social application -an online gamewere a set of features are given to the users in order to accomplish their task (play and chat with others). The set of features developers give to the players are not so innovative (but for an interactive map of the city and of the environment, where all the players are showed), and the interface doesn’t offers personalization features. However they found an interesting solution to the problem of the help. They used a main (textual, static) help (Fig.3), with a list of the more frequently asked questions. Moreover, they created also a dynamic step-by-step contextual help (Fig. 4) that can be easily reached every time the user wants and needs (if closed it can be easily opened again).
Games Design Principles for Improving Social Web Applications
293
Fig. 3. Classical help in Hordes
Fig. 4. Interactive help in Hordes
In this case the designers well understood they have to avoid information overload, and they also eased the learning curve. However, they didn’t provide a sufficient level of personalization: e.g., not all the players use the supplied tools, while several of
294
I. Di Loreto
them prefer external applications created by other players. This result in various webbrowser tab opened at the same time, and the user have to switch between them while playing. The integration of external application into the game would be an interesting upgrade in the direction of personalization.
2 Conclusions While it is true that the main purpose of an interface - either for a social networking software, a game, or any other kind of application - is to allow users to communicate with the software[8], we cannot forget that for the kind of applications we are analyzing the main purpose is the communication between peoples, so the interface have to dissolve[15]. With a view on what we have just said, both of the sites developers have to learn something from the game design principles we described in par.2. It is also evident from the analysis, that the use of a web browser as client doesn’t allows a straightforward application of game design practical solutions, but require further elaboration. Finally, from this analysis and comparison, we want now draw some considerations on “good practice” in web applications design. First we suggest that designers have to take into account the de facto standards used in interfaces adopted by members of social groups that can become future lead users. Second, designers cannot forget the HCI assumption that the functioning of a technology have to be self-evident. In order to do this, designers have to provide multiple levels of detail, at increasing depth of complexity, allowing people to choose the features they need. More in general, we claim for the re-use of some solution and features developed in games design (as seen in par.2), in order to improve social web applications usability.
References 1. Lotito, G.: Emigranti digitali. Origini e futuro della società dell’informazione dal 3000 a.C. al 2025 d.C., Mondadori Bruno, Milano, IT (2008) 2. Prensky, M.: Digital Natives, Digital Immigrants (October 2001), http://www. twitchspeed.com/site/Prensky%20,%20Digital%20Natives, %20Digital%20Immigrants%20-%20Part1.htm (Retrieved January 13, 2009) 3. Palfrey, J., Gasser, U.: Born Digital: Understanding the First General of Digital Natives. Basic Books, New York (2008) 4. Lenhart, A., et al.: Teens, Video Games, and Civics, Pew Internet & American Life Project (2008), http://pewInternet.org/pdfs/PIP_Teens_Games_and_Civics_ Report_FINAL.pdf (Retrieved January 13, 2009) 5. Freitas, S.d.: Learning in immersive worlds: a review of game-based learning. London: Joint Information Systems Committee (2006) 6. Ye, Z., Ye, D.: HCI and Game Design: From a Practitioner’s Point of View (2004), http://www.yebrothers.com/documents/HCIGAMEDESIGN.pdf 7. Saunders, K., Novak, J.: Game development essentials. Thomson Delmar Learning, Clifton Park (2007)
Games Design Principles for Improving Social Web Applications
295
8. de Souza, C.S.: The Semiotic Engineering of Human–Computer Interaction. The MIT Press, Cambridge (2004) 9. Facebook Statistics, http://www.facebook.com/press/info.php?statistics (Retrieved January 13, 2009) 10. Lacy, S.: Facebook Learns from Its Fumble. Business Week, http://www.businessweek.com/technology/content/sep2006/tc200 60908_536553.htm?campaign_id=rss_tech (Retrieved January 13, 2009) 11. Gonsalves, A.: Facebook Founder Apologize. In: Privacy Flap; Users Given More Control. Information Week, http://www.informationweek.com/news/internet/ebusiness/show Article.jhtml?articleID=192700574 (Retrieved January 13, 2009) 12. Kundanis, R.M.: Children, teens, families, and mass media: The millennial generation. LEA, Mahwah (2003) 13. Kirriemuir, J., McFarlane, A.: Literature review in games and learning. Report 8. Bristol. Nesta Futurelab (2004) 14. Motion Twin description, http://www.motion-twin.com/english (Retrieved January 13, 2009) 15. Weiser, M.: Creating the invisible interface (invited talk). In: ACM Conf on User Interface Software and Technology (UIST), 1 (1994)
A Multiple-Level 3D-LEGO Game in Augmented Reality for Improving Spatial Ability Trien V. Do and Jong-Weon Lee Mixed Reality & Interaction Laboratory, Sejong University 98 Gunja-dong, Gwangjin-gu, Seoul, 143-747, Republic of Korea [email protected], [email protected]
Abstract. Inspired by the real LEGO game, an Augmented Reality 3D LEGO game is introduced. With multiple levels, the game provides a tool to improve spatial ability for a wide range of ages. Through the game, users can practice many spatial skills such as analyzing a 3D model’s structure, figuring out what to do to make a primitive geometry become a component of a 3D model, assembling components to create a complex model. The users mainly use their hands controlling physical makers to play the game. A user study was also carried out to evaluate the game and to compare it with the real LEGO game. The game is believed to be a useful and interesting tool to enhance not only human’s spatial ability but also human’s creation in 3D reconstruction. Keywords: LEGO, Augmented Reality, Human Computer Interaction, Spatial ability, Serious Game.
A Multiple-Level 3D-LEGO Game in Augmented Reality for Improving Spatial Ability
297
LEGO game, it does not require any special devices. Just with a personal computer, a cheap webcam, and some printed markers, any users can start playing the game at once. The 3DAR-LEGO introduces the users a flexible, convenient, and effective human-computer interaction to play the LEGO game in computer by combining both the traditional input method (mouse/keyboard) and the tangible input method (markers). The users use a mouse to select operations on a toolbar then control markers to interact with the game.
2 Related Work Lego is a line of construction toys manufactured by the Lego Group [7]. It consists of colorful interlocking plastic bricks and an accompanying array of gears, small figures and other parts. Lego bricks can be assembled and connected in various ways in order to construct such objects as vehicles, buildings and even robots. Normally, players will look at 2D images of 3D models and reconstruct physical models. Certainly, the 2D images cannot convey all the information of 3D models. Recently, the Lego Group has collaborated with Metaio to launch the “DIGITAL BOX” [8]. Metaio utilizes the innovative technology, Augmented Reality, to develop software programs which together with a camera and a display screen let LEGO packages reveal their contents fully-assembled within live 3D animated scenes. Therefore, the potential buyers can hold both the box and the finished product in their hands and are able to see exactly how much fun there is inside. Thanks to this idea, players now have a better view of what they have to assemble. However, the idea of the DIGITAL BOX just stops at intuitively displaying 3D models to assist in playing the game. Nothing is changed in the way of interacting with the Lego. The image of a DIGITAL BOX terminal from the LEGO Group in Fig. 1 shows exactly how a Lego model will look like when it has been fully assembled, even before the box is opened.
Fig. 1. A Lego digital box
298
T.V. Do and J.-W. Lee
Before the introduction of the Digital Box, the use of Lego had been already brought to the Augmented Reality in several systems. However, bricks of Lego are often used as the tangible parts for these systems. AR Lego [9] implements a playful machine assembly and maintenance scenario. The system provides step-by-step assembly instructions of which block should be mounted next and how to verify whether the user is at the correct stage in the construction. LEGO House Platform [10] is a real miniature house using LEGO bricks. Instead of building the house completely with furniture and decorations by LEGO bricks, authors augment the physical LEGO house with virtual furniture using AR technology.
3 Game Overview Inspired by the famous Lego game, the idea of the 3DAR-LEGO is that: first the 3DAR-LEGO allows users to load a sample 3D model from a file, then it decomposes this model into “pieces”; the users have to use these pieces to reconstruct the 3D model while observing the sample model. Depending on the selected level of the game, a piece can be a precise component of the sample model or just a primitive geometry: − Level 1: each piece is a precise component of the sample model. All the users have to do is to pick it and put it at an appropriate position to recreate the model. No transformation operation is required. − Level 2: each piece is a primitive geometry such as a sphere, a cube, a cone etc., which can be customized to become a precise component of the sample model. The users have to customize pieces then place them together to reconstruct the model. The customizing operations include scaling, rotating, resizing, coloring or texture mapping. − Level 3: no piece is provided at this level. By observing the sample model, the users have to analyze to figure out its structure. Then they select, customize and place primitive geometries together to reconstruct the model.
4 Hardware Setup Keeping the wish of bringing the 3DAR-LEGO to a large volume of audiences in mind, a desktop marker-based AR game with minimum requirement of computer skills and hardware devices is developed. After printing all the markers, turning on the camera, the users can at once enjoy the game. Fig. 2 shows a typical hardware setup for the game. In order to play the 3DAR-LEGO, a user needs: − A personal computer with a normal monitor: no kind of 3D glasses is required. − A webcam with a tripod (preferable) − A Cardboard containing multiple markers: this is the working space of the game, so the users can freely interact with the game without too much worry about the marker occlusion problem. − An Object Marker: a component of a model is displayed on this marker and added to a model on the cardboard later. This marker is attached to a block to provide a convenient tangible interaction device.
A Multiple-Level 3D-LEGO Game in Augmented Reality for Improving Spatial Ability
299
− A Sample Marker: this is a block with five markers so that the users can observe a sample 3D model from any direction. − A Selecting Marker: this marker is used to select any components of a model. All transformation operations are applied to selected components. − Two Editing Markers: change attributes of a piece.
Fig. 2. A typical hardware setup for the 3DAR-LEGO
5 How to Play the Game The 3DAR-LEGO has three levels as mentioned above. Depend on each level, skills are required differently. In this section, how to play the game at the Level 1 will be described in details first. The level 2 and 3 will be explained after that. At the Level 1, playing the 3DAR-LEGO is similar to playing the real Lego game. It follows the steps below: 1) Loading a sample 3D model from a file: the game provides some sample models (from simple to complex ones). Later, users themselves can create their own sample models. After a model is selected, it will be displayed on the Sample Marker (See Fig. 3). By observing this visual 3D model, the users will know what they have to assemble. On the other hand, it will be decomposed into pieces. These pieces are stored in a palette on the right side of the windows. Each piece is located in a box of the palette. This palette is dynamically resized depending on the number of the pieces (See Fig. 3). 2) Observing pieces: when the users move the mouse over a box, the piece in that box will be displayed on the Object Marker. Thanks to this, the users can have a comprehensive observation of that piece.
300
T.V. Do and J.-W. Lee
3) Picking a piece: if the current piece on the Object Marker is suitable, the users can click the mouse to firmly bind it to Object Marker (See Fig. 3). 4) Assembling a piece to reconstruct the 3D model: holding the Object Marker, the users freely move their hands to an appropriate position to place it down by clicking on the Add button on the toolbar. The aligning feature makes this operation simple and accurate (See Fig. 3). If by mistake, a piece is placed at a wrong position, it can be attached back to the Object Marker by clicking on the Cut button on the toolbar. 5) Step by step, the users will complete their model. Transformation operations may be required depending on the complexity of the sample model.
Fig. 3. The 3DAR-LEGO interface
Level 2 and Level 3 require the users to have good ability to analyze a 3D model’s structure. The users have to figure out what to do to customize a primitive geometry to a model’s component, and then assemble components to recreate a complex model. To create and customize primitive geometry, the users need to use functions provided by the 3DARModeler [11] which is already integrated into the game. The 3DARModeler can be considered a simple version of 3D Studio Max but in Augmented Reality environment. It provides necessary functions for a modeling system such as creating 3D objects, customizing object’s attributes, aligning objects, grouping, applying texture, adding animation etc. The models created by the 3DARModeler are saved to files to serve as sample 3D models. Therefore, sample models can be designed by users themselves to meet their own need in a specific area which they want to improve spatial ability. For example, Fig. 4 shows the 3DAR-LEGO at Level 3 where no piece is provided. In this game, the sample model is a tower. While observing it, the users have to realize that the bottom component is essentially a cube. Then a cube should be created and textured. After that it needs to be scaled along the three axes by Editing
A Multiple-Level 3D-LEGO Game in Augmented Reality for Improving Spatial Ability
301
Fig. 4. The 3DAR-LEGO interface
Markers (Details of how these Editing Markers perform other operations are explained in [11]). Step by step, other components are added to complete the 3D model.
6 Evaluation and Conclusion Through the game, players can practice many spatial skills such as analyzing a model’s structure, figuring out what to do to customize a primitive geometry to a model’s component, arranging components to create a complex model. To get feedbacks from users, a user study has been carried out through an HCI class, and the Kid & Edu exhibition in Korea. The main goal is to evaluate the system and to compare it with the real Lego game. 63 participants were divided into three groups − Group 1: from 7 to 11 years old (29 participants) − Group 2: from 12 to 15 years old (13 participants) − Group 3: 16 years old above (21 participants)
Fig. 5. A simple model of a table
302
T.V. Do and J.-W. Lee Table 1. Evaluation results of the 3DAR-LEGO
Aspects Interesting Practical Similar to the real Lego Intuitive Easy to play at Level 1 Easy to play at Level 2 Easy to play at Level 3 Able to improve spatial abilities Able to enhance the ability of analyzing a model’s structure
All the participants were asked to assemble a simple model of a table (see Fig. 5) at all the three levels then fill in a questionnaire. We used the 7 scale to evaluate each aspect of the 3DAR-LEGO. The Table 1 above shows the results of the user study. All the participants were able to play the game at Level 1 easily. However only 2 participants of group 1, 4 participants of group 2, and 7 participants of group 3 rated above 6 for the “Easy to play” at Level 2 and 3. In compare with the real Lego, all the participants agreed with the following advantages of the 3DAR-LEGO: − − − − −
Easy to play Do not worry about the loss of pieces Do not need space to store pieces Easy to carry to any place Can see the complete 3D models before playing the game
With the supportive results of the user study, we believe that our system will be a useful and interesting tool to improve not only human’s spatial ability but also human’s creation in 3D reconstruction.
Acknowledgments This work was sponsored and funded by Seoul R&BD Program (20070387).
References 1. Spatial Intelligent Home Page of University of Limerick, Limerick, Ireland, http://www.ul.ie/~mearsa/9519211/ 2. Eckstrom, R.B., French, J.W., Harman, H.H., Derman, D.: Kit of Factor-Referenced Cognitive Tests. Educational Testing Service, Princeton, NJ (1976) 3. Souvignier, E.: Training räumlicher Fähigkeiten (Training spatial abilities). In: Klauer, K.J. (Hrsg.) Handbuch Kognitives Training, pp. 293–319. Hogrefe, Göttingen (2001)
A Multiple-Level 3D-LEGO Game in Augmented Reality for Improving Spatial Ability
303
4. Durlach, N., Allen, G., Darken, R., Garnett, R.L., Loomis, J., Templeman, J., Von Wiegand, T.E.: Virtual environments and the enhancement of spatial behavior: Towards a comprehensive research agenda. In: PRESENCE - Teleoperators and Virtual Environments, vol. 9, pp. 593–615 (2000) 5. Rizzo, A.A., Buckwalter, J.G., Neumann, U., Kesselman, C., Thiebaux, M., Larson, P., Van Rooyen, A.: The Virtual Reality Mental Rotation Spatial Skills Project. Cyber Psychology and Behavior 1, 113–120 (1998) 6. ARToolkit Home Page, http://www.hitl.washington.edu/artoolkit/ 7. Lego Home Page, http://www.lego.com 8. Metaio Home Page, http://www.metaio.com/media-press/press-release/ lego-digital-box/ 9. István, B., Dieter, S.: Augmented Reality Agents in the Development Pipeline of Computer Entertainment. In: Proceedings of the 4th International Conference on Entertainment Computing, pp. 345–356 (2005) 10. Irawati, S., Ahn, S.C., Kim, J.W., Ko, H.D.: VARU Framework: Enabling Rapid Prototyping of VR, AR and Ubiquitous Applications. In: Proceedings of IEEE Virtual Reality Conference, March 2008, pp. 201–208 (2008) 11. Do, T.V., Lee, J.W.: 3DARModeler: a 3D Modeling System in Augmented Reality Environment. International Journal of Computer Systems Science and Engineering 4(2), 145–154 (2008)
An Online Survey System on Computer Game Enjoyment and Personality Xiaowen Fang, Susy Chan, and Chitra Nair School of Computing, College of Computing and Digital Media DePaul University, USA [email protected], [email protected], [email protected]
Abstract. This paper discusses the development of an online survey instrument to measure the game enjoyment and player characteristics like age, gender and personality traits. A research framework of game play is proposed based on a review of prior research on computer game enjoyment, game characteristics, personality theories, effects of computer game play, and technology acceptance model. The proposed framework suggests that an appropriate fit between characteristics of the player and gaming technology will result in greater enjoyment while social influence may moderate effects of the fit. The survey will allow the researcher to establish the fit profiles between player characteristics and game play. Keywords: computer games, game play, personality, enjoyment.
An Online Survey System on Computer Game Enjoyment and Personality
305
It will be important from the perspective of practitioners in computer game industry to understand the player characteristics and personality differences in their target audiences. A comprehensive framework for examining the interaction between player characteristics and game features is needed for a better understanding of the process of game play and its impacts on players. Based on literature review, we propose a framework of computer game play. As discussed in section 4, this framework emphasizes a fit between the player’s characteristics, such as personality, and game technology could lead to enjoyment. This paper reviews relevant research for the framework and discusses the development of an online survey system to investigate computer game enjoyment and personality traits. In the following sections, we will discuss: (1) theories and research findings of computer game enjoyment and media enjoyment, (2) game players’ characteristics; (3) computer game play; (4) the proposed framework, (5) research method, and (6) next steps.
2 Background Literature on Game Play and Enjoyment This section provides a review of studies in game play and enjoyment, video game preference, and several theories about media enjoyment. 2.1 Game Play and Enjoyment Player enjoyment is central to playing computer games ([10] and [11]). Few studies to date measure the player’s emotional responses, or game enjoyment, during game play ([12], [13], and [14]). In spite of a rising interest in academic game research, the actual experience of playing digital games is underrepresented in the gaming literature ([12]). Rajava and colleagues ([14]) have measured psychophysiological responses (facial EMG, skin conductance and cardiac RBI to game events) in the video game Monkey Bowling 2. Their study tests the premise that emotions would be expected to play an important role in gaming behavior and enjoyment. They note that different video game events elicit a reliable psychosociological response and arousal, and there may be large individual differences in the player’s emotional response to games and game events. Emri and Mayra ([15]) have studied immersion in the game world and proposed a model consisting of three different components of immersion: sensory, challengebased and imaginative immersion (SCI-Model). Sensory immersion refers to multisensory properties of a game; challenge-based immersion involves cognitive and motor skils of the games; imaginative immersions refers to immersions within the fantasy world created through the game and depends on the richness of the narrative structure of the game. Klimm ([16]) proposes that game enjoyment is based on three experiential factors: experience of affectence or immediate feedback to the player as a causal agent, feelings of suspense, relief and self esteem, and the feeling of being drawn to a fictional world.
306
X. Fang, S. Chan, and C. Nair
Karolein and colleagues ([12]) suggest that the social context (game competition between friends) and negative experiences such as in-game frustration or tension should be examined to understand the overall game experience. They used the focus group metohodolgy and summarized their finding into a comprehensive categorization of digital game play experiences. Based on their categorization, fun, amusement, pleasure and relaxation are the in-game experiences of enjoyment and the feelings of being energized, satisfaction, and relaxation are the post-game experiences. Other researchers have also investigated the emotional responses during game play. Chumbley and Griffiths ([17]) have found that skill and in-game reinforcement characteristics significantly influence a number of affective measures, most notably excitement and frustration. Some researchers have used physiological responses to objectively evaluate a player’s experience with entertainment technology ([18] and [19]). 2.2 Video Game Preference Research on video game uses and gratifications has focused on the main appeal of video games. Based on a survey of 244 10- to 24-year olds, Selnow ([20]) isolates five gratification factors that attract players to arcade video game play. These factors show that a video game: (1) is preferable to human companions, (2) teaches about people, (3) provides companionship, (4) provides activity/action, and (5) provides solitude/escape. Another study of video games ([21]) reveals a similar set of gratifications for arcade game use: excitement, satisfaction, and tension reduction. A survey conducted by Phillips, Rolls, Rouse, and Griffiths ([22]) suggests several uses of video game play, including “to pass time,” “to avoid doing other things,” “to cheer oneself up,” and “just for enjoyment.” Furthermore, Griffith’s ([6] and [23]) research on video game addiction includes additional uses and gratifications: arousal, social rewards, skill testing, displacement, and stress reduction. In several comprehensive studies, Sherry and his colleagues ([24], [25], and [26]) have enumerated a set of video game uses and gratifications based on focus group research and surveys of over 1,000 participants ranging in age from 10 to 24 years old. These factors include competition—to prove to other people who have the best skills and who can react or think the fastest; challenge—to solve the puzzles to achieve goals such as reaching the next level or beating the game; social interaction—to use video games to interact with friends and learn about the personalities of others; diversion—the use of games to avoid stress or responsibilities and to fill time, relax, escape from stress, and/or because there is nothing else to do; fantasy—to do things they normally would not be able to do, such as driving race cars, playing professional football, or flying; and arousal—the stimulation of emotions resulting from fast actions and high quality graphics. Grodal ([27]) explains that much of the fascination with video games can be attributed to the ability of players to control the game in terms of outcomes (i.e., deciding how the “plot” will unfold), the speed at which the game progresses, and mastery of the game or mastery over other players. Grodal further argues that video games are a tool for emotional control, whereby desired arousal levels can be maintained through playing. As such, video games are enjoyed most when the level
An Online Survey System on Computer Game Enjoyment and Personality
307
and speed of the game match the player’s optimal mental and motor capacity. Vorderer, Hartmann, and Klimmt ([28]) have provided support for the idea that game play is more enjoyable when there are a variety of ways to solve a challenge offered in a video game. 2.3 A Tripartite Model of Media Enjoyment Based on a review of various terms for capturing the concept of media enjoyment, Nabi and Krcmar ([30]) introduce a tripartite model of media enjoyment—affective, cognitive, and behavioral dimensions of enjoyment (Figure 1). In this conceptual model, the underlying affective dimension focuses largely on empathy; positive and negative moods and specific affective states (e.g., horror, sadness, and suspense) could also be considered to feed this component. The cognitive component focuses primarily on judgments of game characters’ actions, though other judgments, like general enjoyment as attitudes toward story assessments (e.g., perceived realism, story coherence, message quality) or more personal evaluations (e.g., relevance, similarity) could also be included in this category. Finally, the behavioral component is logically connected to selective exposure in terms of the player’s viewing intent as well as behaviors during viewing, including the act of viewing itself. A player’s affective, cognitive, and behavioral reactions are influenced by a number of other factors including prior knowledge, direct experience, personality traits, and current mood. However, these factors are expected to operate by influencing the three components of enjoyment, which, in turn, serve to inform perceptions of enjoyment.
Fig. 1. A tripartite model of media enjoyment ([30])
3 Research on Game Player’s Characteristics Several individual characteristics may influence enjoyment of computer game play. A literature review reveals gender, age, and personality as key characteristics.
308
X. Fang, S. Chan, and C. Nair
3.1 Gender One of the most consistent results in studies of video game usage is the striking difference between boys and girls in the amount of play (e.g., [22] and [23]). Some scholars argue that this difference may be due to access. The annual Annenberg Public Policy Center survey on family media use points out that video games are played in 76% of homes with at least one boy, as compared to 58% of homes with at least one girl ([35]). Other studies emphasize that the gender gap in game use may have less to do with access than the content of the games ([36] and [37]). Bonanno and Kommers ([38]) discover that a high percentage of females prefer puzzle, adventure, fighting, and managerial games, and males prefer first-person shooters, role playing, sport, and strategy games. They suggest that these tendencies can be viewed as a process of accommodation to different underlying gender-related neurocognitive processes. 3.2 Age An online survey conducted by Griffiths, Davies, and Chappell ([39]) compares adolescent and adult online game players. Significant patterns emerge among adolescent gamers. They were more likely to be male, less likely to gender swap their characters, and more likely to sacrifice their education or work to play video games. In relation to favorite aspects of game play, significantly more adolescents than adults claimed violence as their most favorite aspect of playing. Results also show that, in general, the younger the player, the more time they spent each week playing online games. 3.3 Personality Personality can be defined as a stable set of tendencies and characteristics that determine the commonalities and differences in people’s psychological behavior (thoughts, feelings and actions) that have continuity in time ([40]). Personality is one of the most elusive areas of psychology, difficult to understand, and difficult to test. Nevertheless, psychologists have developed several theories to explain personality ([41]) based on two principles: core of personality and periphery of personality ([40]). Core of personality addresses the inherent attributes of human beings which do not change over the course of living. They are used to explain the similarities among people. Periphery of personality, on the other hand, focuses on learned attributes. It helps to identify the differences among people. There are many personality theories: psychoanalytic theories, biopsychological theories, behaviorist theories, cognitive theory, humanistic theory and many more ([42]). Personality type and trait theories have served as the basis of several personality tests widely used commercially and in academic research. Type theory focuses on personality type or the psychological classification of different types of people. It originated from the ideas of Swiss psychiatrist Carl Jung. The basic assumption is that the personal unconscious is a potent part of human
An Online Survey System on Computer Game Enjoyment and Personality
309
psyche. Reliable communication between the conscious and unconscious parts of the psyche is necessary for wholeness ([43]). Myers Brigg Type Indicator or MBTI ([44]), one of the most widely used instrument for non-psychiatric populations in the area of counseling and personality testing ([45] and [46]) is based on Jungian based type theory. The inventory has four bipolar discontinuous scales: Introversion-Extraversion, Sensation-Intuition, Thinking-Feeling and Judging Perceiving. Respondents are classified into one of 16 point score obtained for each scale. Carlson ([47]) points out that although MBTI has been used unsystematically in a wide range of areas, it has a favorable validity assessment. However, critics of this tool ([48], [49], [50], and [51]) have reported its psychometric shortcomings and observed that the types are quite strongly stereotyped by professionals. Another personality type theory is Type A and Type B personality theory ([52]). Jenkins Activity Survey ([53]) is a self administered questionnaire to assess an individual's A/B type. Type A tends to be intense and hard-driving while type B people tend to be relaxed and less competitive, and lower in risk. Though widely used in clinical setting, Type A/B theory has been reported as having questionable perspective validity ([54], [55], and [56]). Trait theory addresses personality traits, or a person’s habitual patterns of behavior, thought, and emotion ([57]). Trait theorists generally assume that traits are relatively stable over time; they differ among individuals and influence behavior. Many studies and questionnaires are based on the trait theory, for example, a 300 item Adjective Checklist -ACL ([58]), Catell's 16 PF ([59]), 44 Item Trait Descriptor Adjectives (TDA) ([60]) . Over the years, the five-factor model (e.g. [61], [62], [63], [64], and [65]) has gained acceptance among researchers because it establishes a common taxonomy ([66]). It contains the following five dimensions of personality: Extraversion outgoing and stimulation-oriented vs. quiet and stimulation-avoiding ; Neuroticism emotionally reactive, prone to negative emotions vs. calm, imperturbable, optimistic; Agreeableness - affable, friendly, conciliatory vs. aggressive, dominant, disagreeable ; Conscientiousness - dutiful, planful, and orderly vs. laidback, spontaneous, and unreliable ; Openness to experience - open to new ideas and change vs. traditional and oriented toward routine. The Big Five traits have been researched and validated by many different psychologists (e.g. [67], [65], [68], and [69]) and are at the core of many personality questionnaires. Many studies have also established that four constructs of MBTI converge with the Big Five traits (e.g., [70]). Among the many measurement questionnaires, NEO-PI-R developed by Costa and McCrae ([71]) is well validated and widely used. International Personality Item Pool (IPIP) ([72]) also gained significant attention over the past decade. It is available over the internet and includes over 2000 items. The rationale for the collaboratory effort was to allow faster and continuous refinement of the set of personality scales by other scientists over a public domain. It is non proprietary and allows free use of items and constructs by researchers.
310
X. Fang, S. Chan, and C. Nair
Perceived Ease of Use
Player Characteristics -Demographics - Personality Traits
Fit Profiles
Enjoymen t
Intention to Play
Gaming Technology
Social Influence
Fig. 2. A framework of computer game play
4 A Framework of Computer Game Play Based on media enjoyment theories, personality theories, and the technology acceptance model ([8]), we propose a conceptual model of computer game play as depicted in Figure 2. This framework suggests that enjoyment derived from game play is the result of a fit between characteristics of the player and elements of gaming technology. The better the player-technology fit, the more enjoyment. Social influence will moderate the effects of player-technology fit. Based on this framework, enjoyment and perceived ease of use are two determinants of user intention to play computer games. This framework conceptualizes the impact of player characteristics and social influence on game play enjoyment that have been observed in many earlier studies ([3], [4], [5], [7], [9], [22], and [23]). Furthermore, the framework incorporates the gaming technology and its interactions with the player in affecting enjoyment of game play and intention to play. The following sections (4.1 to 4.6) discuss the main constructs of this framework. 4.1 Fit Fit has been examined in some detail in the strategic management literature. Three definitions of fit in structural contingency theory have been identified: fit as congruence, fit as interaction, and fit as internal consistency ([73]). Venkatraman ([74]) has extended these ideas to six unique perspectives on fit in the strategy literature: fit as moderation, as mediation, as matching, as gestalts, as profile deviation, and as covariation. The most promising perspective for player-technology fit in a gaming context is the idea of fit as an ideal profile. From this perspective, fit is viewed as feasible sets of equally effective alternative designs of games. Each design should be internally consistent and matched to a player’s characteristics. This conceptualization of fit translates well into a gaming environment. The player-technology fit can be defined as ideal profiles composed of an internally consistent set of player characteristics and gaming elements that affect game play enjoyment. A higher degree of adherence to an ideal profile increases the game
An Online Survey System on Computer Game Enjoyment and Personality
311
play enjoyment. A test of player-technology fit would require three steps: (1) identifying distinct player characteristics, (2) specifying ideal gaming technology for each set of player characteristics, and (3) testing player enjoyment resulting from player-technology alignment. 4.2 Enjoyment Strong empirical evidence indicates that the motivational basis of human activity relies on two rather independent systems: a so-called approach system and an avoidance system ([75]). Activation of the approach system results in pleasure, whereas activation of the avoidance system leads to pain ([76]). Research in psychology and neuroscience most often uses the term pleasure to describe agreeable reactions to experiences in general. Most communication researchers have used the term enjoyment to describe and explain such positive reactions toward the media and its contents. Our framework uses enjoyment to describe and explain positive reactions derived from game play. The tripartite model of media enjoyment proposed by Nabi and Krcmar ([30]) suggests three types of reactions to enjoyment: affective, cognitive, and behavioral reactions. Their model effectively conceptualizes player reactions to enjoyment as discussed in other media enjoyment theories such as model of complex entertainment experience ([29]), transportation theory ([31]), disposition theory of drama ([32]), and flow theory ([33]). These three types of reactions to enjoyment can be applied to develop an instrument to measure the degree of enjoyment derived from game play. 4.3 Player Characteristics Previous research has shown that game play is linked to gender (e.g., [22] and [23]), age ([39]), and personality ([3], [4], [5], and [7]). Media enjoyment theories also imply close relationships between a player’s personality and enjoyment ([29], [30], [31], [32], and [33]). In our proposed framework, a player can be described in terms of gender, age, and personality traits. 4.4 Social Influence Various theories suggest that social influence is crucial in shaping user behavior. For example, in the theory of reasoned action (TRA) ([77]), a person’s behavioral intentions are influenced by subjective norms as well as attitude. Hsu and Lu ([78]) indicate that social influences, including perceived critical mass and social norms, significantly and directly, but separately, affect player attitude and intention of playing online games. Choi and Kim ([9]) also note the importance of social interactions on continuing to play online games. Our proposed framework of game play posits that social influence impacts enjoyment as a moderator. It moderates the effects of player-technology fit. 4.5 Extending TAM to Game Play The technology acceptance model (TAM) ([8]) is one of the most widely used models for IT adoption. According to TAM, an individual’s IT adoption is influenced by
312
X. Fang, S. Chan, and C. Nair
perceived usefulness and perceived ease of use. Perceived usefulness (PU) is defined as the degree to which a person believes that using a particular system would enhance his or her job performance. Perceived ease of use (PEOU) refers to the degree to which a person believes that using a particular system would be free of effort. In a gaming context, perceived usefulness is no longer applicable and therefore not an appropriate measure of extrinsic motivation ([79]). Enjoyment is deemed as a more appropriate measure of extrinsic motivation to play games because enjoyment measures how the gaming technology helps achieve the task-related objective -enjoyment -- and it becomes an outcome expectancy.
5 Method 5.1 An Online Survey System In order to test the framework depicted in Figure 3, an online survey system will be developed to continuously collect data on player characteristics and the enjoyment of playing different computer games. This survey system is hosted on a permanent web server. It will allow researchers to establish the fit profiles between game technology and player characteristics. We have set up a web server and are in the process of completing a website for online surveys. The online survey system will also allow researchers to conduct longitudinal studies.. The initial survey we plan to conduct contains three types of questions about each player: demographics and gaming experience, personality traits, and enjoyment of playing a particular computer game. The instruments measuring computer game enjoyment and personality traits are discussed next. 5.2 Game Enjoyment Instrument Based on the tripartite media enjoyment model, we have developed an 11-item instrument ([80]) to measure computer game enjoyment following the method recommended by Moore and Benbasat ([81]) for measurement development. We developed an initial set of 60 items to measure affective, cognitive, and behavioral dimensions of game enjoyment. Afterwards, we conducted expert consultation, exploratory and confirmatory card sorting, and an extensive survey. These steps resulted in an 11-item instrument with strong validity and reliability measures. 5.3 Personality Instrument Personality traits will be measured using the 50-item IPIP inventory in the five domain constructs -- E: Extraversion vs. Introversion; A: Agreeableness vs. Antagonism; C: Conscientiousness vs. Lack of direction; N: Neuroticism vs. emotional stability O: Openness vs. closedness. IPIP items have been developed to measure the constructs in several inventories, for example, NEO-PI-R ([71]), 16 Personality Factor Questionnaire -16PF ([82]), Hogan Personality inventory-HPI ([83]) etc. Tables comparing the psychometric characteristics of the original scales with IPIP item are available for all the major inventories on the website: http://ipip.ori.org/newValidity.htm
An Online Survey System on Computer Game Enjoyment and Personality
313
6 Next Steps This paper presents a conceptual framework of computer game enjoyment. Based on the framework, we propose to design and conduct online surveys to understand the role of player characteristics and personality differences in game enjoyment. An appropriate fit between characteristics of the player and gaming technology could result in greater enjoyment. A study of such nature has not been done before. We have developed an instrument to measure game enjoyment. The next step is to establish the relationships between player characteristics, personality traits in particular, and game enjoyment. The findings are expected to have profound implications to future development of computer games. Based on our research finding, we plan to conduct additional studies to validate our proposed framework of game play.
References 1. Griffiths, M.D.: The therapeutic use of videogames in childhood and adolescence. Clinical Child Psychology and Psychiatry 8, 547–554 (2003) 2. Robillard, G., Bouchard, S., Fournier, T., Renaud, P.: Anxiety and presence during VR immersion: A comparative study of the reactions of phobic and non-phobic participants in therapeutic virtual environments derived from computer games. CyberPsychology & Behavior 6, 467–476 (2003) 3. Anderson, C.A.: An update on the effects of playing violent video games. Journal of Adolescence 27, 113–122 (2004) 4. Anderson, C.A., Bushman, B.J.: Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: a meta-analytic review of the scientific literature. Psychological Science 12(5), 353–359 (2001) 5. Anderson, C.A., Dill, K.E.: Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. Journal of Personality and Social Psychology 78(4), 772–790 (2000) 6. Griffiths, M.D.: The observational analysis of adolescent gambling in U.K. amusement arcades. Journal of Community and Applied Social Psychology 1, 309–320 (1991) 7. Uhlmann, E., Swanson, J.: Exposure to violent video games increases automatic aggressiveness. Journal of Adolescence 27, 41–52 (2004) 8. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3), 319–339 (1989) 9. Choi, D., Kim, J.: Why people continue to play online games: in search of critical design factors to increase customer loyalty to online contents. Cyberpsychology & Behavior 7(1), 11–24 (2004) 10. Peter, V., Tilo, H., Christoph, K.: Explaining the enjoyment of playing video games: the role of competition. In: Proceedings of the second international conference on Entertainment computing (2003) 11. Vorderer, P., Bryant, J.: Playing Video Games: Motives, Responses, And Consequences. Lawrence Erlbaum Associates, Mahwah (2006) 12. Karolien, P., Yvonne de, K., Wijnand, I.: It is always a lot of fun!: exploring dimensions of digital game experience using focus group methodology. In: Proceedings of the 2007 conference on Future Play (2007)
314
X. Fang, S. Chan, and C. Nair
13. Niklas, R., Mikko, S., Jussi, H., Timo, S., Jari, L., Aki, J., et al.: Emotional response patterns and sense of presence during video games: potential criterion variables for game design. In: Proceedings of the third Nordic conference on Human-computer interaction (2004), http://portal.acm.org/citation.cfm?id=1028068# 14. Rajava, N., Saari, T., Laarni, J., Kallinen, K., Salminen, M.: The Psychophysiology of Video Gaming: Phasic Emotional Responses to Game Events. In: The Changing Views: Worlds in Play (2005) 15. Ermi, L., Mayra, F.: Fundamental components of the gameplay experience: Analysing immersion. In: de Castell, S., Jenson, J. (eds.) Changing Views: Worlds in Play. Selected papers of the 2005 Digital Games research Association’s Second International Conference (2005) 16. Klimm, C.: Dimensions and determinants of the enjoyment of playing digital games: a three-level model. In: Copier, M., Raessens, J. (eds.) Level up: Digital games research conference, Utrecht University, Faculty of Arts (2003) 17. Chumbley, J., Griffiths, M.: Affect and the Computer Game Player: The Effect of Gender, Personality, and Game Reinforcement Structure on Affective Responses to Computer Game-Play. Cyberpsychology & Behavior 9(3), 308–316 (2006) 18. Hazlett, R.L.: Measuring Emotional Valence during Interactive Experiences: Boys at Video Game Play. In: Proceedings of CHI 2006, pp. 1023–1026. ACM, New York (2006) 19. Mandryk, R.L., Inkpen, K.M., Calvert, T.W.: Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour & Information Technology 25, 141–158 (2006) 20. Selnow, G.W.: Playing videogames: The electronic friend. Journal of Communication 34(2), 148–156 (1984) 21. Wigand, R.T., Borstelmann, S.E., Boster, F.J.: Electronic leisure: Video game usage and the communication climate of video arcades. Communication Yearbook 9, 275–293 (1985) 22. Phillips, C.A., Rolls, S., Rouse, A., Griffiths, M.D.: Home video game playing in schoolchildren: A study of incidence and patterns of play. Journal of Adolescence 18, 687– 691 (1995) 23. Griffiths, M.D.: Are computer games bad for children? The Psychologist: Bulletin of the British Psychological Society 6, 401–407 (1991) 24. Sherry, J.L., Desouza, R., Holmstrom, A.: The appeal of violent video games in children. In: The Broadcast Education Association annual conference, Las Vegas, NV (2003) 25. Sherry, J.L., Desouza, R., Greenberg, B., Lachlan, K.: Relationship between developmental stages and video game uses and gratifications, game preference, and amount of time spent in play. In: The International Communication Association annual conference, San Diego, CA (2003) 26. Sherry, J., Holmstrom, A., Binns, R., Greenberg, B., Lachlan, K.: Gender differences in video game use and preferences. In: The National Communication Association annual conference, Miami, FL (2003) 27. Grodal, T.: Video games and the pleasure of control. In: Zillmann, D., Vorderer, P. (eds.) Media entertainment: The psychology of its appeal, pp. 197–213. Erlbaum, Mahwah (2000) 28. Vorderer, P., Hartmann, T., Klimmt, C.: Explaining the enjoyment of playing video games: The role of competition. In: Proceedings of the Second International Conference on Entertainment Computing, Carnegie Mellon University, Pittsburgh (2003) 29. Vorderer, P., Klimmt, C., Ritterfeld, U.: Enjoyment: at the heart of media entertainment. Communication Theory 14(4), 388–408 (2004) 30. Nabi, R.L., Krcmar, M.: Conceptualizing media enjoyment as attitude: implications for mass media effects research. Communication Theory 14(4), 288–310 (2004)
Playability Testing of Web-Based Sport Games with Older Children and Teenagers Xavier Ferre, Angelica de Antonio, Ricardo Imbert, and Nelson Medinilla Universidad Politecnica de Madrid Campus de Montegancedo 28660 - Boadilla del Monte (Madrid), Spain {xavier,angelica,rimbert,nelson}@fi.upm.es
Abstract. Playability occupies a central role in videogame design. Heuristics may help for establishing the game concept, but some testing is essential for ensuring a wide acceptance in the target user population. The experience of designing and testing a set of web-based sport videogames is described, focusing on the heuristics employed and the testing approach. The results show that an emphasis on a simple set of game controls and the introduction of humorous elements has obtained a positive response from older children and teenagers. Keywords: videogame design, playability heuristics, testing with older children and teenagers, sport web-based games, Olympic Games.
For the purpose of designing videogames, as in any interactive software development effort, evaluation plays a very important role. Evaluation with representative users drives the whole design process, ensuring that the game concept is well understood and that playability reaches an appropriate level; and it may also provide some evidence about the degree to which the videogame is engaging and easy to learn for the target population. In this paper, the process of developing a set of web-based online sport games about the History of the Olympics is described, emphasizing the evaluation approach taken and describing the results obtained so far. The remainder of the paper is organized as follows. Section two describes the project structure. In section three, the development approach is presented. Section four includes the heuristics employed, while section five analyzes the usability testing carried out. Next, the main results of the evaluation through usability testing are presented in section six. Finally, section seven summarizes the findings and presents the conclusions.
2 Online Games about the History of Olympics Modern Olympic Games started with the 1896 games in Athens, and they have taken place every four years since then, with some gaps due to world wars. Madrid is one of the four candidate cities currently bidding to host the 2016 edition of the Olympic Games. The Playability Evaluation of Games about the History of the Olympics project aims to deliver the Olympic spirit and knowledge about the history of the modern Olympic Games, sponsored by the Madrid town council, through a series of webbased online games that portray the reality of the Olympics as they evolved starting in Athens 1896. The main target users of these games are older children (10-12) and teenagers (13-16), who will be in an appropriate age to become Olympic volunteers in 2016, when the Olympic Games might take place in Madrid. In order to act as a support to the Madrid 2016 candidature by attracting the greater number of visitors to the website, the project intends to provide an entertaining experience to the online game player, considering playability as the primary focus throughout the development effort. 2.1 Games Structure The website structure is organized around each Olympic Games edition. In a first phase, the project covers the first eight modern Olympic Games, from Athens 1896 to Amsterdam 1928. A particular sport has been chosen for every Olympic Games edition, recreating in the videogame the ambiance and rules at that time, but there are additional elements around the sport game contributing to the educational aim of the project, as shown in Fig. 1. When accessing a particular Olympic Games edition, the player is presented a world map where it is to be located the place where the Olympics took place. In case the player chooses a point that is not close enough to the target city, the player is redirected to a card game where points can be gained in order to get additional clues
Playability Testing of Web-Based Sport Games with Older Children and Teenagers
317
start geographical game
athlete selection
sport game
card game
podium
multimedia newspaper Fig. 1. Game structure for each Olympics edition
in the world map. The card game is based on swapping adjacent cards to form horizontal or vertical combinations of three or more cards with the same design. After successfully locating the city in the world map, the player chooses the gender and race of the character he or she will play with, and the competition begins in the
318
X. Ferre et al.
sport game. After winning a medal or a trophy1, a multimedia newspaper is presented to the player, with a voice-over reading its contents, so that interesting historical facts are presented to the player. Afterwards, the player is given access to the following Olympic Games edition in chronological order.
3 Development Approach Before starting the development of the sport games, a study was performed about existing similar games freely available on the World Wide Web. The great majority were written in Adobe Flash, as it is a multimedia platform that eases the tasks related to animation graphics and interaction in a web environment, so Flash was chosen as programming platform. Regarding the game size (in pixels) in the web pages where they are embedded, the average size in the analyzed web-based sport games is around 550x400. A slightly bigger size has been decided for this project, 640x480, in order to allow games with a greater interactive area, while still fitting the great majority of desktop PC screen resolutions. With respect to Flash file size, the chosen game size is still between reasonable limits in terms of user bandwidth consumption. Concerning ease of learning, while the majority of the analyzed games did not provide a training mode, a significant minority did. For the development of each game, guidance was provided by a team of sport history experts. This guidance included a detailed description of the scenery where the sport event depicted in the game took place, of judges, sportsmen and sportswomen equipment, including their attire; the content of each Olympic Game newspaper (pictures, video and text); and the rules for the chosen sport at that time. The choice of the sport to depict in each Olympic Game and the design of the game concept was performed jointly by the technical team and the team of sports history experts. Particularities or special events that characterized the chosen sport in such edition of the Olympic Games were introduced in the game concept, in order to improve the game in terms of historical resemblance (see section 6.1). Playability heuristics were considered in the early design of each game (see section 4). According to the historical guidelines and the game concept definition, a first prototype was designed, running in parallel with the design of the graphical elements and decorations. The aim of this first prototype was just allowing the game to be played, with graphical aspects reduced to a sketch level, and without additional elements like instructions, podium screen, educational newspaper and so on. With the first playable prototype some evaluation was performed with children and teenagers at collaborating schools and high schools, in order to check the understandability of the game mechanics, and the operability of the game controls. After feedback had been gathered from this first playability test, the game mechanics and controls were reconsidered, along with the development of the whole graphical part and the inclusion of the remaining details in the game. A second round of playability testing was performed afterwards with a new set of test participants in the target age range, in order to check if the objectives had been met. Sections 5 and 6 detail the usability tests carried out for playability evaluation and the results obtained. 1
Depending on the particular Olympic Games edition, event winners were awarded either trophies or medals.
Playability Testing of Web-Based Sport Games with Older Children and Teenagers
319
4 Heuristics Several sources have been considered for choosing the playability heuristics to be taken into account in early design activities: Generic ones for any kind of game [4][6], and heuristics for mobile games [7]. Chosen heuristics are the ones that better fit the kind of simple games to be developed in the project (see Table 1). Table 1. Chosen playability heuristics and their sources # 1
2 3 4 5 6 7 8 9
10
11 12 13 14 15 16 17 18 19 20 21 22 23
Heuristic description The player should be always able to identify the score/status and goal in the game. The player sees progress in the game and can compare results with other players. The interface embodies metaphors with physical or other systems that the user already understands. The interface uses randomness in a way that adds variety without making controls unreliable. The interface uses humour appropriately. Player's fatigue is minimized by varying activities and pacing during game play. The game is enjoyable to replay The game is fun for the player first, the designer second and the computer third (put non-expert player’s experience first). The first player action is painfully obvious and should result in immediate positive feedback. Pace the game to apply pressure but not frustrate the player. Vary the difficulty level so that the player has greater challenge as he or she develops mastery (easy to learn, hard to master). Challenges are positive game experiences, rather than a negative experience (results in their wanting to play more, rather than quitting). Mechanics/control actions have consistently mapped and learnable responses. Shorten the learning curve by following the trends set by the gaming industry to meet user’s expectations. Provide immediate feedback for user actions. Upon initially turning the game on the player has enough information to get started to play. Sounds from the game provide meaningful feedback or stir a particular emotion. Players do not need to use a manual to play the game. The interface should be as non-intrusive to the player as possible. Art should be recognizable to player, speak to its function and support the game. Indicators are visible The player understands the terminology The player does not have to memorize things unnecessarily The game contains help The player is in control.
5 Usability Testing for Playability Evaluation Every game has been evaluated through usability testing [8]. Representative users play with the game while test facilitators observe their performance, taking notes about their comments, and about the problems they experience when trying to play the games. After the test each user is asked to fill a satisfaction questionnaire to gather his or her impressions about the game. A total of seven test sessions have been carried out, with the participation of 148 older children and teenagers in the 10-16 year old range, belonging to five schools and high schools. This section presents the methodology followed for the usability testing. 5.1 Previous Planning Test participants fall in the defined target age range, and the task they are asked to perform is to play the game in order to be the Olympic winner. Tests are always performed in the test participants' educational institution, usually in the computer classroom. Test planning involves preparing in advance the following materials: Content of the questionnaires for test participants to fill in; selection and preparation of any promotional gift offered to thank them for their participation; consent forms to be distributed between parents; and a detailed description of test procedures, including a script for the verbal instructions to be offered to participants. Games are previously tested in the computers where the test will take place, in order to avoid possible problems either with the Internet connection or related to software compatibility. Whenever possible a pilot test is performed in order to review test setup and planning. 5.2 Test Sessions Before granting them access to the game, test participants are given a brief introduction to the project objectives and their role as game testers is highlighted. They are asked to follow the instructions on the screen and to ask about any aspect of the game that feels strange or incomprehensible to them. Test sessions usually cover the testing of two different games. A fixed time of around 15 minutes is established for playing with each game, but test facilitators may suggest a particular participant to advance to the following game (after filling the corresponding questionnaire), in case he or she gets tired of the previous game or is winning easily. Test facilitators observe test participants, taking notes about any comment or remarkable behavior from them; and answering any question test participants may ask. After playing with each game, test participants are requested to fill in a satisfaction questionnaire, and when the test session ends they are thanked for their participation and the gifts are distributed. 5.3 Usability Report Data gathered in the test session includes test participants' performance with the games (automatically logged during the test session), their answers to the satisfaction
Playability Testing of Web-Based Sport Games with Older Children and Teenagers
321
questionnaire and notes taken by test facilitators. These pieces of data are analyzed and presented in the form of a usability report that details the main usability problems identified, and the positive and negative comments to different parts of the game. When suggestions for improvement have been mentioned by test participants they are also included in the document. The report is discussed by the development team in order to evaluate the priority and severity of the problems identified, and an action plan is established in order to make the appropriate changes to game design. When test results are satisfactory according to project objectives, the game is released in the project website [5].
6 Results The tests carried out with the different versions of the games have allowed for an important improvement in terms of playability, both between the final version of a specific game compared to the previous versions tested, and between the first games developed and the rest of them. Children and teenagers who have tested the games have liked them in general terms, with the answers to the satisfaction questionnaires showing that they agreed to the sentence "I would like to play often this game" to a high (47% of participants) or very high extent (32% of participants). Additionally, most of test participants showed reactions when playing with the final version of the games that suggested they were actually enjoying the games in the test sessions. The main two issues raised in the usability testing are related to the particular elements introduced in the games to increase the "fun" aspect, and to the appropriateness of including a guided training tutorial. 6.1 Inclusion of Funny Elements In certain games, funny or peculiar elements that attract the attention of the player have been introduced in the game concept, in order to make the overall player's experience more enjoyable. Whenever possible, these elements highlight particularities of the sport event as it happened in the corresponding Olympic Games edition, specially those ones that may be considered striking nowadays. This is part of the educational aim of the project, for the users to know the reality of the first editions of the modern Olympic Games. For example, in the marathon game - for London 1908 - there are some bottles and sponges with water, which have to be picked by the player in order to maintain the stamina and be able to win the race. But a glass of champagne appears randomly as well, with the effect of losing all the stamina (see Fig. 2). In this way it is shown to players that it was not uncommon for sportsmen to receive any kind of drink from watchers willing to help. In the tests performed with the marathon, children commented about how much they liked having objects appearing on the road, and that they would have liked having even more objects appearing in the game.
322
X. Ferre et al.
Fig. 2. Objects in the marathon game: Champagne glass, sponge and water bottle
Whenever these elements are funny, the reaction from older children and teenagers is even better. In the high jump game - Athens 1896 - there is a preparation phase before the actual jump where a correct posture must be achieved, and getting the sportsman to lean forward too much makes him fall to the ground, as shown in Fig. 3.a. In the sack racing game - Saint Louis 1904 - there is a chicken appearing randomly between the sportsmen or sportswomen (see Fig. 3.b). The observed reaction of young test participants to these elements has been very positive, since they cheered out loudly and they hurried to tell their friends about these particular elements, showing that they took their attention. With the inclusion of these funny ingredients, the objective of offering players an enjoyable experience has been reinforced, since these elements have been very well received by test participants, who seemed to appreciate startling elements in the game. 6.2 Training Tutorial vs. Simple Set of Controls For a web-based online game, ease of learning plays an important role, since players need to be enjoying their playing experience from the very beginning. When the player does not understand the game concept or gets stuck because of ignorance about game controls, there is a high risk that he or she will abandon the game to go to any of the thousands of other games freely available on the Internet. In the first game, the high jump, a relatively complex game concept was designed, since there is a preparation phase before the actual jump where a correct posture must be achieved, and there is a jump phase where accurate pressing of control keys influences the jump result. For it to be challenging, the jump phase happens very quickly, but this makes it hard for the novice player to grasp the game concept. Therefore, a guided training tutorial was introduced at the beginning of the game. In this tutorial, the player may try to perform a jump slower than in competition, and feedback is offered about the posture and key pressing. Comments from test participants about this game have shown that, while they enjoyed the actual competition, they did not specially enjoy having to go through a training tutorial.
Playability Testing of Web-Based Sport Games with Older Children and Teenagers
323
Fig. 3. a) Fallen sportsman in high jump game; b) chicken in sack racing game
In the following games tackled, the design of the game concept was focused on reducing to a great extent the complexity of controls, so that there was no need for a training tutorial. A screen with brief instructions is presented instead, which is shown the first time the game is played. Test participants were asked in the satisfaction questionnaire if the game was easy to understand and play, and results showed that the first game including a training tutorial was perceived as less easy to understand and play (by 11%) than the average of the other games, which were designed with a simpler and common set of controls. Additionally, test participants had a better initial performance in the games with a simpler set of controls that didn't need a training tutorial.
7 Conclusions In the frame of the Playability Evaluation of Games about the History of the Olympics project eight games have been developed and evaluated through usability testing with older children and teenagers. Even if further testing should be done in order to discover why some particular elements work better than others, results already show some hints that could help developers of web-based simple games, similar to the sports games developed in the project. When the game concept is kept simple, it seems to favor game acceptance between users in the 10-16 age range, especially in terms of game controls. In that direction, when the game is simple enough so that basic controls may be explained in an introductory screen and no training tutorial is needed, players tend to perceive it as easier to control and their initial performance improves. Startling elements included in a game have been highlighted by test participants as funny, showing their appreciation by cheerful comments with peers. The project will go on evaluating the educational potential of the developed sport games. It is planned to carry out an experimental study in physical education classes, comparing the knowledge about the history of the Olympic Games between a group of children taught in the traditional way (lectures) and a group of children that use the games as a way to actively learn about this subject.
324
X. Ferre et al.
Only the first eight editions of the modern Olympic Games have been covered so far, with encouraging partial results, but the development of additional games to cover the rest of Olympics editions will be conditioned by the results obtained in the experimental study.
Acknowledgments The authors wish to thank the members of the sport history experts team (M. Teresa Gonzalez Aja, Vicente Gomez Encinas, Elena Merino Merino and Alejandro de la Viuda Serrano) for having the original idea of the project and making it happen, and to the Madrid town council for their financial sponsorship to the project.
References 1. OnlineGames.Net - Play Free Games Online, http://www.onlinegames.net/ 2. Games at Free Online Games.com, http://www.freeonlinegames.com/ 3. Jones, S., Fox, S.: Pew Internet Project Data Memo. Generations Online 2009. Pew Internet & American Life Project (2009), http://www.pewinternet.org/pdfs/PIP_Generations_2009.pdf 4. Malone, T.W.: Heuristics for designing enjoyable user interfaces: Lessons from computer games. In: Proc. of the 1982 Conference on Human Factors in Computing Systems, pp. 63– 68. ACM, New York (1982) 5. Playability Evaluation of Games about the History of the Olympics site (2009), http://raptor.ls.fi.upm.es/olimpiadas/ 6. Desurvire, H., Caplan, M., Toth, J.A.: Using Heuristics to Evaluate the Playability of Games. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1509–1512. ACM, New York (2004) 7. Forum Nokia: Mobile Game Playability. Version 1.0. Nokia (2006), http://sw.nokia.com/id/5ed5c7a3-73f3-48ab-8e1e-631286fd26bf/ Mobile_Game_Playability_Heuristics_v1_0_en.pdf 8. Dumas, J.S., Redish, J.C.: A Practical Guide to Usability Testing. Revised Edition. Intellect, Exeter, England (1999)
Exploring the Elements and Design Criteria of Massively-Multiplayer Online Role-Playing Game (MMORPG) Interfaces Chun-Cheng Hsu1 and Elvis Chih-Hsien Chen2 1
Department of Communication and Technology, National Chiao Tung University, 1001 Ta-Hsueh Rd., Hsinchu 300, Taiwan 2 Institute of Communication Studies, National Chiao Tung University 1001 Ta-Hsueh Rd., Hsinchu 300, Taiwan 300 [email protected], [email protected]
Abstract. A great many people play online games and sales of online games are considerable, but research has further shown that a major reason behind the failure of games in the market is poor user interface design or usability, highlighting even more the importance of these issues in games design. This research uses surveys and focus groups to explore the factors influencing the usability and interface design of online games. First the definition and different types of game are discussed, and then the composition and features of online game interfaces analyzed. Second, a review is made of Human-Computer Interaction (HCI) research literature relating to design criteria for game interfaces. Finally, in discussion with experts, this study isolates the design criteria that should be emphasized when designing each key element of an online game interface. Keywords: Massively-multiplayer online role-playing games (MMORPG), interface design, elements of game interfaces.
games fail in the market, a major reason for which is that principles of HCI research have not been implemented, forcing players to rely solely on their intuition and experience to operate games. Besides this, currently there exists in the games industry a lack of criteria and guidance that games designers can follow. Many interface design criteria are excessively abstract and general, lacking any detailed explanation of how they may be met [1].
2 Purpose of Research With this in mind, the goal of this research is to explore the elements that comprise online games interfaces and criteria for interface design. First, this study looks back over the context of the development of online games in order to understand the basic definition of online games and the types and composition of interfaces. Second, taking MMORPGs as a case study, this paper analyzes and comes to some conclusions about the elements that make up the interfaces of online games. Finally design criteria important within the field of HCI are discussed, and in the process corresponding detailed design criteria for each interface element are identified. It is hoped that these findings will provide a reference for online game development and interface design.
3 Games and Interface Design 3.1 Definition and Types of Computer Game Computer games generally refer to electronic games played on personal computers (Zhou Rong, 1998), and are one of the important forms of game to emerge in the digital age. On the basis of past definitions and revisions, Juul [7] concluded that games possess six characteristics: they are rule-based, provide different possible outcomes, attach positive or negative values to these outcomes, involve player effort, players are attached to the outcomes and negotiate any consequences of success or failure. The specialist computer game magazine Computer Gaming World has divided electronic games into eight genres: action/arcade, adventure, role-playing, simulation, sports, strategy, classic/puzzle and war games. Novak [9] alternatively categorized games into action/arcade, adventure, arcadeadventure, casino, puzzle, role-playing, simulation and strategy games. Rollings & Adams [10] divided games into action/arcade, strategy, role-playing, sports, vehicle simulation, construction and management simulation, adventure, growth and puzzle games. MMORPGs currently dominate the online games market, and so accordingly this study will focus mainly on exploring this type of game, discuss the course of the development of online role-playing games and briefly describe their characteristic qualities. 3.2 Online Role-Playing Games Online role-playing games originate with pencil and paper tabletop role-playing games (TRPG). In a TRPG, using the game’s rulebook and scenario, players play
Exploring the Elements and Design Criteria of MMORPG Interfaces
327
characters in a story and interact with other players playing other characters in accordance with the rules of the game. Dungeons and Dragons is the most wellknown TRPG, and has deeply influenced the way that later multiplayer role-playing games are played, the arrangement of their plots and framework of their stories (Shuffle Alliance, 2002). Embryonic forms of today’s online role-playing games, MUDs (Multi-user Dungeons), first appeared in the late 1970s, and were conducted entirely in text, lacking any graphics [4]. Today’s online role-playing games can be said to be a kind of graphical MUD. A MUD allows many players to simultaneously log into a computer program and participate in adventures, roam around and interact. In this virtual space, players can play their characters and interact with other players, as well as create their own objects and environments to their own liking (Su Fenyuan, 1996). In an ordinary single player role-playing game, the player tries to reach successive break points and finally complete the game. However in a MUD, there is no such end point or success and failure. In fact, players can add new objects and scenes to its database at any time, expanding the entire MUD world. As in current online games, in a MUD players can simultaneously interact, communicate and participate in social interaction with many other players [4]. Although online role-playing games have inherited the characteristics of MUDs, they differ in that they are conducted via completely graphical user interfaces (GUI). A GUI not only increases the entertainment factor of the games, but also creates a more complete virtual reality. (Zheng Rongji, 2006). According to past related research, online role-playing games share four characteristics (Lin Peiyuan, 2007; Rolling & Adams, 2003): role-playing attributes, combat and adventure, a field of play and group interaction. To summarize, MMORPGs are games conducted in a virtual environment in which players attempt to increase their standing within the game, carry out tasks and missions, or simply enjoy interacting with other players within the rules of the game. However, a game can only progress by means of players’ interaction with the game mechanism via a control interface. Below some discussion is now given and conclusions drawn regarding the constituent elements of games interfaces according to past related research. 3.3 Games Interfaces The definition and types of games interfaces. Novak [9] believes a game’s interface allows players to be able to control characters in a game, move through the whole environment of the game, and make decisions during the course of the game. If there were no games interface, then a game would be merely a story, an animation or a static environment – a game that would be impossible to actually play. Consequently, an interface provides the connection between the player and the world of the game, allowing the player to then interact with the content of the game. As far as the definition of an interface is concerned, Clanton (1998) distinguished two components of interfaces: a motor component, including for example how a joystick operates a game; and a perception component, including the design of menus, tools and buttons and how they are displayed on the screen. Ding Zhaochen (2006) defined a game interface as the spatial environment on a screen of a game and its
328
C.-C. Hsu and E.C.-H. Chen
interactive mechanism. The above shows that there are broad and narrow definitions of games interfaces, Clanton’s (1998) definition being the former, includes both hardware input devices and graphic display; the latter refers solely to the display of illustrative and operational screens. Rollings & Adams [10] point out that the main function of a game interface is to allow the user to interact with the game, and can be split into two parts. The first is to receive the player’s commands, and the second is to inform the player of the state of play during the course of the game, and to endeavor to display all options open to the player with images. These different functions and capabilities require interfaces to be made up of different constituent elements. Saltzman [11] divided a game’s interface into exterior interface and user interface components; the former refers to menu screens mainly used for game-related setup options such as adjusting sound, changing game type, equipment control and saving the game; the latter refers to the screens that the player will encounter during the course of the game, such as the appearance of avatars, maps of the environment and skill buttons. Looking at interfaces from the perspective of presentational form, Boyd [2] divided games interfaces into two types, configurational interfaces and pictorial interfaces. A configurational interface chiefly refers to the control panes in a game, the majority of which are located on the edges of the screens of a game which provide the player with appropriate hints and messages; the emphasis in a pictorial interface differs from a configurational interface’s emphasis on function, in that it resembles a real picture, namely the spatial environment of the game during play. Novak’s [9] definition of a game interface is somewhat broader, including both a non-video interface (namely operating interface) and a video interface. The former includes hardware components such as control buttons, keyboards and mice, which allow the player to carry out choices, such as use a particular weapon, walk around within the game space, or to interact with computer characters or other players. The video interface refers to those functional features displayed on the screen that allow the player to know the status of other avatars within the world of the game, the location of the enemy, distance and direction and other information important to game play. Players base decisions about their own behavior in the game upon this information. The video interface may be further divided into an active video interface and passive video interface, according to whether or not it is under the player’s control. The active video interface refers to that with which a player may interact. Normally this is carried out by clicking on options displayed on the interface, normally presented on the interface in the form of menus or actional systems. The passive video interface consists of those interface elements with which a player is unable to directly control or change, such as players’ avatar status, score and time remaining. This information may be grouped in the same place on the screen, or may be scattered in different places [9]. The above is summarized in Table 1. From this table it can be seen that Novak’s [9] video interface category encompasses the categorizations given by both Saltzman [11] and Boyd [2]. If we then differentiate according to whether or not a player has control, then the active video interface encompasses the external interface, user interface and configurational interface; and the passive video interface includes the user interfaces and pictorial interface. With these categorizations in mind, this research will now discuss the common elements constituting the video interfaces of ordinary games and online games.
Exploring the Elements and Design Criteria of MMORPG Interfaces
329
Table 1. Categorizations of game interface types (source: this paper) Scholar Saltzman [11]
Boyd [2]
Type External Interface User interface Configurational Interface Pictorial Interface
Novak [9]
Active video interface Passive video interface
Definition refers to menu screens, mainly used for game-related setup options refers to the screens that the player will see during the course of the game chiefly refers to the control panes in a game the screens displaying the spatial environment of the game during play Players can click to carry out interaction, normally presented on the interface in the form of menus or actional systems Elements of an interface the player is unable to directly control or change, such as score and time remaining
Constituent elements of a game interface. As described above, the interface within the screens of a game can be regarded as its video interface, and divided into active and passive types depending on whether the player has control. Novak [9] remarks that both the active and passive video interfaces in games share many common constituent parts, including information that the player can utilize and actions that a player must take in order to complete tasks in the game. This normally includes interface elements such as points/level, number of lives/stamina, maps, the character and the game startup menu. If one then divides these according to whether or not the player can control them, then score/level and number of lives/stamina and maps all belong to the passive interface; the character and game startup menu are both part of the active interface. This is clarified in table 2. Table 2. Game interface element types (source: this paper)
Video interface
Game interface type External Configurational interface Interface Active User Configurational interface Interface Configurational Interface User Passive interface Pictorial interface
Constituent elements of games interface Game startup menu Character (Avatar’s actions and functions) Points/level, number of lives/stamina, maps Maps (the spaces and scenes in the game), character (appearance of avatar)
The start menu and the avatar’s actions and functions are components that the player can click to control, and so both form part of the active video interface. They may be further categorized as exterior interface and user interface according to their function during the course of the game. In the passive video interface are included the player’s score, stamina and maps. The player has no way to directly interfere with or manipulate these, and they belong to the configurational interface; the pictorial interface mainly consists of a representation of the game’s virtual space and the player’s avatar. The elements that constitute interfaces differ with game type; those that make up MMORPG interfaces in particular are now dealt with below.
330
C.-C. Hsu and E.C.-H. Chen
Elements of MMORPG interfaces. This paper will look at three games that occupy a relatively large share of the market in Taiwan – ‘World of Warcraft’, ‘Maple Story’ and ‘Perfect World’, summarize the elements common to their interfaces, and then one by one categorize them according to previous scholars’ schemes (Table 4).
Fig. 1. MMORPG game user interface screen (grabbed from World of Warcraft)
A game’s active video interface chiefly incorporates two main types of interface, the user interface and the exterior interface. The former is primarily for game-related setup before the player logs in, while the latter includes the chat/message channel, avatar information, shortcut keys, backpack/toolbag, social activities, game menu/options and seek help/report abuse functions; these all closely related to the player during the course of the game. Manipulating or clicking these functions enables the player to interact with the game mechanism or other players. For details see fig.1. In a game’s passive video interface, although the player does not possess control, this kind of information informs the player of current circumstances and status, and so is very important to game play. This mainly appears at the top of the game screen, and includes the functions avatar status and map information. The former shows the present status of the player’s avatar, providing the player with information on which to base subsequent actions. The latter tells the player his physical location within the game’s spatial environment, aiding orientation. The game’s pictorial interface occupies the center of the screen, and includes a graphical representation of the spatial environment and the full view of the player’s avatar.
4 Discussion of Criteria for Game Interface Design Traditional HCI research has accumulated a great many interface design criteria and evaluation methods, and has already successfully been utilized in the design of interfaces of many systems and web pages. Scholars have repeatedly also cited
Exploring the Elements and Design Criteria of MMORPG Interfaces
331
Table 3. Correspondence of MMORPG interface elements to design criteria Design criteria 1. Affordance 2. Consistency 3. Ability to cancel 4. Grouping
5. Visibility
6. Easy to learn 7. Ability to undo 8. Feedback 9. Effectiveness 10. Easy to remember 11. Customizability
examples of design principles in games interfaces, but at present the actual use of design principles is still overly general [1]. The elements present in game interfaces differ with game type and function, and so detailed design criteria for each interface element are analyzed at this stage. 4.1 Method A review of relevant literature and a focus group are the main methods used at this stage to analyze the important criteria for the design of online game interfaces. This research summarizes 22 criteria from traditional HCI and related literature, as follows: affordance, consistency, ability to cancel, grouping, visibility, easy to learn, ability to undo, feedback, effectiveness, easy to remember, customizability, mapping, efficiency, error correction, notification upon completed actions, shortcut keys, minimizes mental load, satisfies player’s desire to control, standardization, easy to use, aesthetic appeal and distinctiveness. For sources and references for the definitions and explanations of each design criteria, please see Table 3. This study uses a focus group to collect qualitative data. A focus group is a kind of group interview where people share ideas [8]. This study uses a small group consisting of five experienced game or interface designers. First the research topic is explained to the participants, and then there follows a discussion of the relationships between 11 elements of MMORPG interfaces and the
332
C.-C. Hsu and E.C.-H. Chen
22 design criteria. The 11 interface elements all appear on a computer screen and the group must openly discuss decide the importance of the criteria in interface design. 4.2 Result The focus group decided slightly different design criteria are important for different interface types (See Table 4). The group considered that a game’s exterior interface should be easy to remember and use, allow the player to undo actions and be laid out so that related functions are grouped closely together. As for the design of a game’s chat and messaging channel, the group decided that ease of use is important in order to facilitate rapid communication with other players. They also believed that in terms the provision of information, an interface should provide rapid, timely feedback, allowing the player to know the current state of play at any time during the game. The group of experts indicated that the important criteria in avatar information design are aesthetic appeal, the ability to customize according to player preferences, visibility and mapping. Thus in the design of avatar-related images and colors, the primary emphasis should be on aesthetic appeal, and at the same time the connection between the images and their meaning should be clear, in order to enable the player to know his own status. In designing shortcut keys, it was decided that the important design criteria should include grouping of closely related functions, affordance, mapping and easiness to learn. Thus buttons with closely-related functions should be grouped together to allow a new player to easily learn how to use the interface, and at the same time the design of images should be such that the player can quickly understand their meaning, function and method of manipulation. When designing images for the toolbag and social interaction functions, first and foremost the principles of grouping related functions, distinctness of images and standardization should be borne in mind, so that the player is able to readily pick out the desired function from among many images. The design of game menus and help functions, apart from requiring the same emphasis on distinctness and standardization, also needs to emphasize affordance and customizability, to enable the player to immediately see how to operate each function, and at the same time be able to adjust the functions of the interface to suit his needs. In a game’s passive visual interface, the focus group considered it important that the design of the avatar status indicator be visually distinct and clear, enabling the player to be able to see his current status at a glance, and quickly notice any updated information. In map design, importance should be placed on good mapping, minimization of load on the player’s short term memory, ease of use and effectiveness. This is to reduce the level of difficulty the player encounters using the map, to efficiently assist the player’s movements in the game. In terms of the design of the pictorial interface, the group decided that aesthetic appeal, ease of use, and satisfying the player’s desire for control are all important design criteria, the goal being to make the player feel more entertained and a heightened sense of immersion in the game.
Exploring the Elements and Design Criteria of MMORPG Interfaces
333
Table 4. Categorization chart of elements of MMORPG interfaces and design criteria
User/Game Interface
Video Interface
Active
External/Syst em Interface
Game Interface Type
MMORPG Interface Element (function) Choose game server, create new character, choose character, delete character
Easy to use, Easy to remember, Ability to cancel, Grouping, Customizability, Aesthetic appeal
Chat/message channel
Channel selection, channel setup
Easy to use, Feedback, Effectiveness, Notification upon completed action, Aesthetic appeal
Representation of the game’s spatial environment and avatars
Grouping, Affordance, Mapping, Easy to learn, Aesthetic appeal Grouping, Distinctness, Standardization Grouping, Distinctness, Standardization Affordance, Distinctness, Customizability, Standardization Affordance, Distinctness, Customizability, Standardization Affordance, Distinctness, Customizability, Standardization Mapping, Minimizes mental load, Easy to use, Effectiveness, Easy to learn, Aesthetic appeal, Customizability Aesthetic appeal, Easy to use, Satisfies player’s desire for control, Easy to learn, Customizability
5 Conclusion The online games industry continues to flourish, and has become one of the focal points of development in digital entertainment media. How the design of an online game’s interface can enable a player to more easily get into a game, reduce difficulty in learning a game, and then more easily become immersed in the world of the game are extremely important issues. However, much research shows that the current criteria used in the design of interfaces for online games are excessively abstract and general, lacking the detail and methodicalness of real design. This study uses surveys and focus groups to explore the characteristics of the constituent elements of online game interfaces. Second, it analyzes related HCI research into the criteria in game interface design. Finally the important criteria that should be emphasized when
334
C.-C. Hsu and E.C.-H. Chen
designing the constituent elements of online game interfaces are discussed with experts. It is hoped that the findings of this research provide both practical design suggestions and also a foundation for future research into the design of online game interfaces.
References 1. Adream, B.E., Zender, M.: User interface design principles for interaction design. Design Issues 24(3), 85–107 (2008) 2. Boyd, D.S.: Representing space: The pictorial imperative. In: 2004 Workshop on Space and Spatiality, Napier University, Edinburgh (2004) 3. Cornett, S.: The usability of massively multiplayer online roleplaying games: Designing for new users. In: CHI 2004, Vienna, Austria (2004) 4. Curtis, P.M.: Social phenomena in text-based virtual realities. In: Kiesler, S. (ed.) Culture of Internet, pp. 121–142. Lawrence Erlbaum Associates, Mahwah (1997) 5. Federoff, M.A.: Heuristics and usability guidelines for the creation and evaluation of fun in video games. Unpublished thesis, Indiana University, Bloomington (2003), http://melissafederoff.com/thesis.html 6. Jorgensen: Marrying HCI/Usability and computer games: a preliminary look. In: Proceedings of the third Nordic conference on Human-computer interaction (2003) 7. Juul, J.: The game, the player, the world: Looking for a heart of gameness. In: Copier, M., Raessens, J. (eds.) Level up. Digital games research conference, pp. 166–186. Universiteit Urencht and DIGRA, Utrecht (2003) 8. Krueger, R.A., Casey, M.A.: Focus group: A practical guide for applied research, 3rd edn. Sage, Thousand Oaks (2000) 9. Novak, J.: Game development essentials. Thomson Delmar Learning, NY (2005) 10. Rollings, A., Adams, E.: On Game Design. New Riders Publishing (2003) 11. Saltzman, M.: Game Design: Secret of The Sages, 2nd edn. Brady Games, Indianapolis (2000) 12. Ye, J., Ye, D.: HCI and game design: From a practitioner’s point of view (2004), http://www.ye-brothers.com/documents/HCIGAMEDESIGN.pdf
Healthcare Game Design: Behavioral Modeling of Serious Gaming Design for Children with Chronic Diseases Hadi Kharrazi, Anthony Faiola, and Joseph Defazio Indiana University Purdue University Indianapolis, School of Informatics 535 W. Michigan St., Indianapolis, Indiana 46202 {kharrazi,afaiola,jdefazio}@iupui.edu
Abstract. This article introduces the design principles of serious games for chronic patients based on behavioral models. First, key features of the targeted chronic condition (Diabetes) are explained. Then, the role of psychological behavioral models in the management of chronic conditions is covered. After a short review of the existing health focused games, two recent health games that are developed based on behavioral models are overviewed in more detail. Furthermore, design principles and usability issues regarding the creation of these health games are discussed. Finally, the authors conclude that designing healthcare games based on behavioral models can increase the usability of the game in order to improve the effectiveness of the game’s desired healthcare outcomes. Keywords: Serious Gaming, Diabetes, Hypoglycemia, Behavioral Modeling, Patient Empowerment, Compliance to Treatment, Adolescent Diabetic Drivers.
Healthcare Game Design: Behavioral Modeling of Serious Gaming Design INDIVIDUAL PERCEPTIONS
MODIFYING FACTORS
Socio-demographics: age, gender, duration, etc
Perceived Susceptibility
Perceived Threat
Self-efficacy
337
LIKELIHOOD OF ACTION
Perceived benefit Perceived barriers
Intention to Take Action
Cues to Action
Fig. 1. The Health Belief Model
The Planned Behavioral Model (PBM) According to the theory of planned behavior, human action is guided by three components: “beliefs about the likely outcomes of the behavior and the evaluations of these outcomes (behavioral beliefs), beliefs about the normative expectations of others and motivation to comply with these expectations (normative beliefs), and beliefs about the presence of factors that may facilitate or impede performance of the behavior and the perceived power of these factors (control beliefs)” [10]. Indeed behavioral beliefs make a behavior favorable or unfavorable (attitude), normative beliefs produce a social pressure (subjective norm) and control beliefs show the intensity of control factors (perceived behavioral control). Fig. 2 represents a schematic view of the theory. Based on this model, behavioral changes can be achieved by targeting any of the factors: attitudes, subjective norms, or perceptions of behavioral control. The result of such an intervention should produce changes in behavioral intentions and, given adequate control over the behavior, the new intentions will be carried out under appropriate circumstances [11]. The common approach in behavioral models is determining the major elements that can affect participant’s (patient’s) behavior substantially enough to improve their behavior toward their chronic condition.
Fig. 2. The Planned Behavioral Model [10]
338
H. Kharrazi, A. Faiola, and J. Defazio
2 Background 2.1 Patient Empowerment and Serious Gaming Diabetic educators have defined patient empowerment as “the discovery and development of one's inherent capacity to be responsible for one’s own life. [And the necessity to] influence their own behavior and that of others to improve the quality of their lives” [12]. The major hurdle in traditional methods of patient empowerment is motivating patients to change their behavior and maintain that change. This problem becomes even more pronounced in adolescents. One of the most significant factors in better health outcomes for children with long term or chronic disorders is empowering the patients to consistently comply with the treatment regimen, even when they are not experiencing direct effects. Ubiquitous digital games, now considered a mass medium [13], can be exploited to achieve these health objectives. The factors that make digital games so engaging can be applied successfully in health contexts where motivation and engagement are necessary for the management of chronic conditions. Today, the world of computer gaming has expanded rapidly to include applications other than entertainment, such as education. This new genre of games is named “serious games” [14][15][16]. As such, games differ in their use of visual, textual, and auditory channels for feedback, scaffolding challenges, visible goal indicators, overviews and schematics, and ease of learning [17]. This is in stark contrast to the typical “casual games” that are entertainment-centric. The process of learning how to play, how to improve skills, and how to succeed is much more natural in most games, and for applications of diabetes self-management and nutrition education, they continue to show promise. 2.2 Examples of Serious Games for Type 1 Diabetes “Packy and Marlon” is a side-scrolling adventure game (played on a Super Nintendo console) that helps children and teens with type 1 diabetes self-management. “Glucoboy” is a glucometer that plugs into several video game devices, e.g., Nintendo Advance® and Nintendo Game Boy®. The game is intended to encourage blood sugar monitoring and testing in kids with diabetes, with an advanced blood glucose meter that is extremely accurate and highly precise. “Escape from Diab” is a serious videogame adventure in healthy eating and exercise, with the goal to help prevent kids from becoming obese and developing diabetes and other related illnesses. “Nanoswarm” is another game that is about diabetes education that provides the player with a rich, immersive, interactive game. The game’s developers hope that through role-playing and a blend of sci-fi action, they can bring the player to change their behavior toward a healthier life style. Developed by the USDA, the “MyPyramid Blast Off” is an interactive online game where kids can reach Planet Power by fueling their rocket with food and physical activity. Fuel tanks for each food group help children keep track of how their choices fit into MyPyramid. “Insulot” [18] is a cellular phone-based health education learning tool developed for T1D children in Japan. The game is intended to encourage, motivate, and boost the confidence of T1D patients, with outcomes to change behavior. “FatWorld” is a game about the politics of nutrition; a game the challenges the player to decide to be fit or fat, to live or to die.
Healthcare Game Design: Behavioral Modeling of Serious Gaming Design
339
3 Current Research 3.1 Serious Games and T1D Compliance to Treatment In a study with T1D patients, a game framework based on the Theory of Planned Behavior was used to increase the compliance/adherence rate to treatment. The main research question of this study is “Can games improve adherence to treatment in children with chronic diseases?” Parents reported the compliance rates of their children and the children (patients) were awarded in the game based on their compliance to treatment. Health points collected by compliance can be used in the game to buy new items or play additional mini-games. The study showed a significant increase in compliance rates [6]. Fig. 3 includes some screenshots of the game.
Fig. 3. Screenshots of a serious game designed for Type 1 Diabetic patients [6]. Left to right: Top - the login screen; main interface; using scores to buy new items. Bottom - using scores to change options; compliance report for children, and Playing mini-games based on points.
Conceptual Framework Theoretical models of empowerment, compliance and behavioral changes were broken down into its elements and possible matching game elements were identified for each of them. Many of the conceptual elements in empowerment, compliance and behavioral models overlapped each other and therefore were removed. For example, control factors are included in multiple models such as PBM, compliance models and patient empowerment models. Then, the elements that have close conceptual meaning were merged. For example, normative belief in PBM is close to family and peer support in contextual-behavioral model for patient empowerment. The remaining list was purged based on multiple focus group results. For example, none of the focus groups showed an interest in knowing the side effects of a given medication in the game. The final list of elements was refined based on the practicality of the game elements that are matched with the behavioral models. After final considerations and defining the practical solutions some feasible elements were identified and chosen as the key functional game elements that map to
340
H. Kharrazi, A. Faiola, and J. Defazio
different factors in empowerment, compliance and behavioral change models. The final framework included: Knowledge base and educational content, Motivational factors such as Pointing systems and Virtual mentoring systems, Peer Pressure and Measurement tools such as Questionnaires and Compliance reports. Usability Study Usability of this game framework is measured by questionnaires. Some of the questions are Likert-scale while some others are comment based. Usability questions are categorized as: Efficiency which reflects the goal of the game in affecting compliance; Satisfaction which indicates how much fun the game has been; Learnability that shows how easy learning the game was; and Memorability which refers to the ease of memorizing game features; and finally Error that measures the rate of error in the game’s functionality (Fig. 4):
Fig. 4. Mean usability score based on different categories
3.2 Game Pilot Study for Adolescent Diabetic Drivers The pilot study, titled, Testing for Hypoglycemia While Driving, is an educational game aimed at adolescent diabetic drivers with T1D. This study was a required first step (Phase 1) in developing a larger (Phase 2) and more intensive intervention within the context of a computer-assisted blood glucose awareness education program. As such, the resources required to develop and implement a large-scale intervention to change behaviors was beyond the scope of what is outlined here, i.e., in Phase 1. The purpose of the study, however, is to change adolescents’ and parents’ attitudes about hypoglycemia and driving, as well as to change their knowledge of perceived susceptibility, perceived seriousness and threat, perceived benefits to taking action, and barriers to taking action. Finally, the ultimate intent is to decrease the risk of hypoglycemia while driving among adolescents with diabetes.
Healthcare Game Design: Behavioral Modeling of Serious Gaming Design
341
Phase 1: Gaming Pilot Design and Testing Stage In Phase 1, the authors modified and adapted the Cox intervention to a brief educational intervention to be used on a laptop computer [19][20]. In this phase, gaming scenarios were designed and developed, along with the HBM attitudinal and behavioral measures. The theoretical underpinning of game design was HBM, because it addresses the understanding that: (1) a person will take a health-related action if they feel that a negative health condition, such as low blood sugar, can be avoided, (2) a person will have a positive expectation that adhering to the recommended action will avoid a negative health condition, and (3) a person believes that they can take a recommended health action with a successful outcome. Four learning scenarios were developed during the analysis and design stages of the project (Fig. 5). For example, Scenario 1 allowed the participant to make choices prior to and during the drive to school. Scenario 2 allowed the participant to make choices prior to and during the drive home. The theoretical grounding for the multimedia design and testing of Phase 1 was the Multimedia Development Model (MDM) [21], which was derived from the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) instructional model. As reflected in ADDIE, MDM provided an iterative process that allowed the designers to re-visit each development stage during production. An important aspect of the model is its non-linear framework that allowed a repeatedly review and revise of the product. During the analysis phase, plans for content creation were reviewed. During the design phase, flowcharts and storyboards were developed using prewritten scenarios. Formative reviews of each asset and it’s functionality within each scenario helped to ensure the quality and standard of the design during production. HBM guided information presentation (susceptibility/severity of condition, benefits of behavioral change) and ultimate performance (behavioral cues) as the foundation for this design. During the analysis and design phases, graphical, textual, audio, and narrative assets were created. Once created, assets were incorporated into
Fig. 5. Scenario Two - Phase 1 of Educational Gaming for Adolescent Drivers with Diabetes
342
H. Kharrazi, A. Faiola, and J. Defazio
the design and reviewed for appearance, appropriateness, functionality, and fit. The final phase of the design cycle implemented usability testing on several subjects to ensure functionality and flawless execution of the design. Usability Testing Adolescent participants were recruited and the education game modules were tested for content clarity and usability through a usability and user experience evaluation of the product. After the usability findings were collected and analyzed the appropriate changes were made. The following figure (Fig. 6) depicts the average score for the participants’ responses to the usability questions (Likert-scale questions):
5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 1 Q
2 Q
3 Q
4 Q
5 Q
6 Q
7 Q
8 Q
9 Q
0 1 Q
1 1 Q
2 1 Q
3 1 Q
4 1 Q
5 1 Q
6 1 Q
7 1 Q
8 1 Q
9 1 Q
0 2 Q
Axis Title
Fig. 6. Average score of responses to usability questions
Phase 2: Gaming Pilot Testing Stage In Phase 2, researchers will administer the gaming modules to 150 adolescents ages 15 -18 and one of their parents. Information on behavior, attitude, and knowledge acquisition will be collected and analyzed to better predict the likelihood that adolescents and their parents will take action concerning the existing health condition.
4 Conclusion Interactive media, including serious games, are becoming an inescapable part of our everyday life. With the emerging digital gaming culture, serious gaming will become an increasingly vital part of healthcare education for upcoming generations. Advanced interactive user interfaces now provide new opportunities for serious games that can be used not only to educate younger patients about their disease, but also to empower them for a positive change in their behavior both now and into full adulthood. Chronic care constitutes a large slice of national healthcare expenses. Serious gaming can be an economical solution to educating, motivating, tracking, and empowering chronic patients for long term management of diabetes and prevention of complications. Serious games can lower not only the cost of chronic diseases but also
Healthcare Game Design: Behavioral Modeling of Serious Gaming Design
343
the additional costs associated with side effects. Moreover, serious games aimed at health education do not involve large expenses as minimum healthcare staff is required to interact with the patient. In an ideal game, the game’s behavioral model can be integrated into both the patient’s personal health record and the hospital/doctor’s health record system. The health record integration can help the system to reward the patient in the game based on actual improved health results, and it can also help the physician/healthcare staff to track the patient’s compliance to treatment; including other age groups such as adults and elderly patients. Artificial intelligence agents can be used to alert both the patient and the physician if the health status deteriorates. Moreover, different behavioral change models should be experimented with to find the most suitable model for an interactive patient empowerment approach. Managing hypoglycemia in adolescents is critical due to their lack of insight into their disease and the lower compliance rate to treatment. Behavioral models have been used to change the behavior of the patients toward their disease and to improve their adherence to treatment. In this article, Planned Behavioral Model and Health Belief Model were the underlying behavioral models. Two studies were conducted: one to increase compliance to treatment in T1D and another to reduce severe driving consequences of hypoglycemia ion T1D. Both studies showed significant results in usability measures of effectiveness and satisfaction. In summary, advancements in gaming technology, social and health informatics, and interaction design principles and practices, have all provided new knowledge and approaches to facilitating patient education and patient empowerment to achieve higher compliance [22][6].
Acknowledgement Portions of this chapter are used with permission from Dr. Donald Orr, Adolescent Medicine, Indiana University, School of Medicine, Indianapolis, Indiana.
References 1. Hannon, T., Rao, G., Arslanian, S.: Childhood Obesity and Type 2 Diabetes Mellitus. Pediatrics 116(2), 473–480 (2005) 2. Baranowski, T., Archimage, I.: Escape from Diab (Accessed, 2008) 3. Lee, C.: New Video Games Not Just for Fun. Washington Post (October 2006) 4. Hayes, B., Aspray, W.: The Informatics of Diabetes: A Research Agenda for the Socially and Institutionally Sensitive Use of Information Technology to Improve Healthcare. MIT Press, Cambridge (2009) (in press) 5. Lustria, M.: Can interactivity make a difference? Effects of interactivity on the comprehension of and attitudes toward online health content. Journal of the American Society for Information Science and Technology 58(6), 766–776 (2007) 6. Kharrazi, H., Watters, C., Oore, S.: Improving behavioral stages in children by adaptive applications. Journal on Information Technology in Healthcare 6(1) (2008)
344
H. Kharrazi, A. Faiola, and J. Defazio
7. Winn, B.: The Heart of Serious Game Design. In: Annual meeting of the International Communication Association, San Francisco, CA (2008), http://www.allacademic.com/meta/p170925_index.html 8. Hochbaum, G.: Health Belief Model: Why People Seek Diagnostic X-rays. Public Health Reports 71, 377–380 (1956) 9. Hochbaum, G.M., Sorenson, J.R., Lorig, K.: Theory in Health Education Practice. Health Education Quarterly 19(3), 293–313 (1992) 10. Ajzen, I.: The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes 2(50), 179–211 (1991) 11. Ajzen, I.: From Intentions to Actions: A Theory of Planned Behavior, 11–39 (1985) 12. Funnell, M.M., Anderson, R.M., Arnold, M.S.: Empowerment: a winning model for diabetes care. Practical Diabetol 10, 15–18 (1991) 13. Wolf, M.: The Medium of the Video Game. University of Austin Press, Austin, USA (2001) 14. Blackman, S.: Serious games and less! ACM SIGGRAPH Computer Graphics 39(1), 12– 16 (2005) 15. Thompson, D.J., Baranowski, T., Buday, R., Baranowski, J., Juliano, M., Frazior, M., Wilsdon, J., Jago, R.: In Pursuit of Change: Youth Response to Intensive Goal Setting Embedded in a Serious Video Game. Journal of Diabetes Science and Technology 1(6), 907–917 (2007) 16. Ye, Z.: Genres as a Tool for Understanding and Analyzing User Experience in Games. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, Vienna, Austria, pp. 773–774 (2004) 17. Dyck, J., Pinelle, D., Brown, B., Gutwin, C.: Learning from Games: HCI Design Innovations in Entertainment Software. In: Graphic Interface, Halifax, pp. 159–169 (2003) 18. Aoki, N., Ohta, S., Okada, T., Oishi, M., Fukui, T.: INSULOT: A Cellular Phone-Based Edutainment Learning Tool for Children with Type 1 Diabetes. Diabetes Care 28(3), 760 (2005) 19. Cox, D.J., Gonder-Frederick, L.A., Kovatchev, B.P., Julian, D.M., Clarke, W.L.: Progressive Hypoglycemia’s Impact on Driving Simulation Performance. Diabetes Care 23, 163–170 (2000) 20. Cox, D.J., Penberthy, J.K., Zrebiec, J., Weinger, K., Aikens, J.E., Frier, B.: Diabetes and Driving Mishaps: Frequency and Correlations from a Multinational Survey. Diabetes Care 26, 2329–2334 (2003) 21. Defazio, J.: An Innovative Approach for Developing Multimedia Learning Modules. In: ISECON 2001, Cincinnati, vol. 18, p. 34 (2001) 22. Faiola, A.: Designing Humane Technologies: A Potential Framework for HumanComputer Interaction Design. The International Journal of the Humanities 2(3), 1877–1886 (2006)
Analyzing Human Behaviors in an Interactive Art Installation Takashi Kiriyama and Masahiko Sato Graduate School of Film and New Media, Tokyo University of the Arts 2-5-1 Shinko, Naka-ku, Yokohama 231-0001, Japan {kiriyama,sato}@gsfnm.jp
Abstract. Arithmetik Garden is an interactive art installation designed to perform arithmetic operations by using the body. Analysis of data collected during its exhibitions shows that viewers behave differently than optimal solutions generated by computer. There is also an indication that viewer’s emotional changes can be detected by monitoring interactions. Keywords: Interaction, behavior, emotion.
1 Introduction Arithmetik Garden is an interactive art installation designed to provide the viewer with an experience of performing arithmetic operations by using the body. The viewer feels as if he or she became a number that is transformed when it goes through gates. There are eight gates in the installation, including the entrance gate, the exit gate marked with 73, and arithmetic gates of 5, 8, 3, 7, 4, and 2. The viewer picks up a card at the entrance and hangs it around the neck. An initial number -8, -1, 2, 4, 5, 7, 8, 36, 87, or 91 is printed on the surface of each card.
Starting from the initial number, the viewer tries to make the number equal to 73 by going through the arithmetic gates. For instance, if the viewer starts with 2 and goes through the 7 gate, the current number will become 2 7 14. One can walk along a path such as 2
14
22
11
77
73
to arrive at the goal number 73, where A
op
B
means that the current number A becomes B by going through the gate . The 2 gate works only with even numbers. If the viewer becomes uncertain about the current number, he or she can go to the equation monitor to see the current number and the path walked so far.
Fig. 2. Cards with initial numbers
Fig. 3. Equation monitor
The viewer can successfully leave the exit gate if the current number becomes 73. Outside the exit gate, an equation of the path is printed on a piece of paper such as 2 7 8 2 7 4 73 .
Analyzing Human Behaviors in an Interactive Art Installation
347
The state transition diagram mounted on the wall shows all paths leading to 73 (Figure 5). The viewer can reflect on the experience by tracing the path in the diagram, while discovering other possible paths. Arithmetik Garden was created in 2007 and first exhibited at Mori Art Museum, followed by the second exhibition at NTT Inter-Communication Center [ICC] in 2008, both in Tokyo, Japan. Over the two exhibitions, a total of 72,000 visitors experienced Arithmetik Garden. Some visitors reflected on their experience such that they were getting close to the goal but at some point they got lost. Other visitors mentioned to the enlightening moment in which they found a path to the goal. Such changes are of interest for interaction design because they make the experience engaging. We analyzed event logs and video recording to know how visitors walked in Arithmetik Garden and when their behaviors changed. In the rest of this paper, we discuss the methods and results of analysis.
2 Installation Arithmetik Garden employs RFID technology. When the viewer goes through a gate, an RFID sensor inside the gate reads the tag embedded in the card. Each event of passing through a gate is sent to the server and stored in the database. The server maintains the current number and the path of each viewer. The underlying mechanism of this installation is hidden from the user. People walk through gates without being bothered by menus and buttons. In fact, they do not touch anything from the entrance to the exit, because all sensing is done wirelessly.
Fig. 4. System layout
348
T. Kiriyama and M. Sato
Fig. 5. State diagram
Analyzing Human Behaviors in an Interactive Art Installation
349
3 Human Behaviors 3.1 Intuitive Decision During the exhibitions of Arithmetik Garden, we collected paths walked by viewers. Each record consists of an initial number and a sequence of arithmetic operations with timestamps. The records are represented in an XML format after exported from the SQL database. Once represented in XML, we can use XQuery and XSLT for data analysis. They are convenient to look for patterns in a large amount of data [1]. By comparing paths taken by viewers against the shortest paths calculated by computer, we discovered gaps between human decisions and logical optimality. For instance, if the current number is 10, there are two shortest paths to the goal; 10
15
11
77
73 and 10
6
11
77
73.
But among 6,259 cases in which the current number became 10, the largest number of people (2,694) chose the ×7 gate as the next (Fig.6). In fact, going to the ×7 gate takes the viewer away from the goal from distance 4 to distance 6. Here, distance d of a number N to the goal 73 is defined as the minimum number of steps needed to go to 73. Number 10 is of distance 4 to the goal because the two shortest paths to 73 shown above contain 4 steps.
Fig. 6. Selections from 10
350
T. Kiriyama and M. Sato
We suppose that the 7 gate appears to be most promising for the viewer whose current number is 10, as 10 7 70 is numerically close to 73. It seems that people have chosen this gate by intuition. Table 1 shows average time spent to choose the next gate after the current number became 10. It took 15.5 seconds in average to decide to go through 7, which was one of the two transitions that took the shortest time. This fact suggests that people intuitively chose the 7 gate because the resulting number was close to 73. Table 1. Number of people and average time for transitions from 10
gate
transition
number
time (sec)
7
10
70
2,694
15.50
3
10
30
1,175
17.18
5
10
15
1,079
15.14
8
10
18
482
24.45
4
10
6
448
21.48
2
10
5
381
21.28
6,259
17.22
total
3.2 Sticking to a Pattern Sometimes people are occupied in an immediate goal and do not look for alternatives. For instance, 65 is only one step away to the goal. One can make the number 73 by going through the 8 gate. Of a total 8,097 cases in which the current number was 65, 6,646 people went to 73. However, 1,123 people went through the 5 gate instead, such that; 65
70.
It means that in 14% of the total 8,097 cases, people missed the goal. Among the 1,123 people, 552 (49% of 1,123) came from the 5 gate as in; 60
65
70.
Moreover, 234 people (21% of 1,123) went through the +5 gate three times in a row such as;
Analyzing Human Behaviors in an Interactive Art Installation
60
65
70
351
75.
Fig. 7 illustrates these transitions. The pattern of repeating to go through the 5 gate can be explained that people stick to a simple pattern of adding 5 to the current number. Once people started to follow this pattern, they tend to forget about looking for alternatives.
Fig. 7. Sticking to a pattern of adding 5
3.3 Emotion Before the exhibition of the Arithmetik Garden, we did a public testing at university. We videotaped 340 people to analyze how they behave in the installation. By watching video, we found that people started to run near the end of the path since they became certain about how to make the goal. Some visitors were clearly seen an emotional change on the face [2]. It is also found in data that the intervals from a gate to the next became shorter near the end of path. We located such changes of intervals in data and reviewed the video of that moment. The video recording indicated that people were indeed delighted at knowing how to reach the goal. We believe that by monitoring data we can detect emotional changes It may be useful for making interactive systems respond to emotional changes of people [3]. Although initial numbers are 4 to 6 steps away from the goal, people spend 8 to 12 steps in average to reach the goal. We assume that people need to understand the situation and the task by exploring the space. Once the idea of embodiment of transformation becomes clear, they can start concentrating on calculation. If it is true, we may be able to detect another type of emotional change from exploration to concentration by watching the tendency of selecting gates.
352
T. Kiriyama and M. Sato
4 Conclusions We found that event data collected during exhibitions were extremely useful to study human behaviors in Arithmetik Garden. The data indicate that paths taken by humans were different from computer-generated optimal solutions. People tend to prefer to a seemingly promising path such as 10
70
over the shortest paths to the goal. The fact that people in average spent only a short duration of time on the transition to 70 indicates that this selection was made intuitively. Another finding is that people stick an immediate goal. While increasing the current number to 73 by going through the +5 gate multiple times, people tend to miss a solution such as 60
65
73.
It was also found that people started to run when they became certain about the way to the goal. Such a change can be detected by monitoring intervals between gates. Monitoring the user behavior will be useful to design interactive systems that respond to user’s emotional changes. Future research includes applying data analysis techniques developed in this study to other interactive art installations for better understanding of human behaviors.
Acknowledgements The exhibition of Arithmetik Garden was supported by Hayao Nakayama Foundation for Science & Technology and Culture, Dai Nippon Printing Co.,Ltd., Takaya Corporation, and Total Interior Sugahara Co.,Ltd. Data gathering in Arithmetik Garden was made possible by the corporation of Mori Art Museum and NTT InterCommunication Center [ICC]. We thank Masasuke Yasumoto, Masaya Ishikawa, Tomomi Kito, and Yoshikazu Fujita for their contribution in developing Arithmetik Garden.
References 1. Kiriyama, T.: Representing User Interaction Data of the Arithmetik Garden. In: Asian Topic Maps Summit 2007 (2007) 2. Kiriyama, T., Sato, M.: Observing Human Behaviors in an Interactive Art Installation. In: Desmet, P.M.A., Tzvetanova, S.A., Hekkert, P., Justice, L. (eds.) Proceedings from the 6th Conference on Design & Emotion (2008) 3. Norman, D.A.: Emotional Design. Basic Books (2004)
The Effects of Quest Types and Gaming Motivations on Players’ Knowledge Acquisitions in an Online Role-Playing Game Environment Jiunde Lee and Chih-Yi Chao Institute of Communication Studies National Chiao Tung University
Abstract. The study explores how the design of quest types and different players’ gaming motivations might affect knowledge acquisitions in an online role-playing game environment. An experiment was conducted to collect data. The results showed that “immersion motivation” had the most significant influence on knowledge acquisitions. The bounty-collection quest significantly affected the procedure knowledge of subjects with high immersion motivation, whereas the fed express quest affected declarative knowledge of subjects with high immersion motivation. Keywords: Online Role-Playing Games, Quest Types, Gaming Motivations, Declarative Knowledge, Procedural Knowledge.
1 Introduction As internet access has become more widespread, the computer game market has seen tremendous growth within the last few years. The explosion of computer games as a mainstream media form combined with captivating interactive technologies is viewed as a potential way of providing solutions for problems faced by divisions of instructions, corporations, governments, and public interest groups [22]. Such assumptions are largely based on the concept of computer games as a means of solving problems. In order to complete game quests, players with various gaming motivations become actively involved in the process of problem solving to make decisions, apply tactics, and construct necessary knowledge. Players’ gaming motivations and quest types appear to be the critical factors which engage players in this process. This study aims to explore how gaming motivations and quest types might affect players’ knowledge acquisitions in terms of declarative and procedural knowledge.
of challenges and quests (problem solving). According to Sawyer [22], the core concept of gameplay is actually a problem-solving process. Such a problem-based environment aims to promote dialogical interchange, argumentative decision making and reflective thinking [18]. Scholars [8, 15] have proposed that computer games could benefit the development of peoples’ perceptive and cognitive abilities such as inductive discovery, parallel processing, and spatial representation. Three issues are particularly worth considering within a game-based context: gaming motivations, quest types (problem spaces), and knowledge acquisitions. 2.1 Gaming Motivations One of the most cited papers about players’ gaming styles is Bartle’s 1996 study of player classification [2]. By using two coordinate axes - Acting / Interacting and Players / World, Bartle categorized game players into four gaming styles: Killer, Achiever, Socialiser, and Explorer. Yee [26], however, doubted the appropriateness of applying such a classification to define players, since it is quite common that one player might demonstrate more than one gaming style during gameplay. In fact, Bartle [2] himself also did not completely rule out the possibility that these four gaming styles might overlap with each other to a certain degree. Consistent with Malone’s [14] intrinsic motivations of playing games, Yee [27] suggested that players’ gaming motivations might be a better way to distinguish their different behavioral tendencies in games. There are three types of gaming motivations: Achievement, Social, and Immersion. Achievement motivation includes advancement, mechanics, and competition attributes. Social motivation includes socializing, relationship, and teamwork attributes. Immersion motivation includes discovery, role-playing, customization, and escapism attributes. In gameplay, players possess three types of gaming motivations simultaneously with one probably predominating the others. The present study adopts Yee’s viewpoint to observe how players’ gaming motivations might affect their knowledge acquisitions in different quest types. 2.2 Quest Types Newell and Simon [17] noted that a “problem space” is an organized unity made up of a set of operators and problem representation activities. As mentioned above, a computer game can be viewed as a problem space. By trial and error and hypothesis testing, players constantly search for solutions and construct knowledge of goals, rules, and concepts (operators) through this inductive discovery process [9]. Among the studies of investigating game quests [6, 12, 13, 23, 24, 25], Aarseth ‘s [1] and Dickey’s [4] studies particularly focus on quest design and have been frequently cited by scholars. Aarseth divided computer quests into three main formats: place-oriented, time-oriented, and object-oriented. In the place-oriented format, players have to find a path to the destinations; the time-oriented format usually fixes the time limit for the completion of missions; and the object-oriented format will set concrete targets for players to acquire, for instance, special gear from a monster NPC. These three quest formats work collaboratively to create a highly entertaining quest. Dickey [4] developed a more detailed category. Computer quests are defined as bounty quest, fed express quest, collection quest, escort quest, goodwill quest, and
The Effects of Quest Types and Gaming Motivations on Players’ Knowledge Acquisitions 355
messenger quest. Bounty and collection quests are quite similar in terms of goals and missions. Players in both quests have to contact NPCs for information then collect certain numbers of specific things by exploration, or defeat opponents. Escort quests ask players to protect a designate NPC to safely move around different game zones. The exciting part of this quest type is the conflicts which arise during the journey. The main purpose of goodwill quests is to enhance social interactions among players with different experience levels. Messenger quests are simply designed to let players interact with the game-controlled objects, such as NPCs, and learn the necessary information in order to complete quests. The present study adopts Dickey’s quest category to filter appropriate quest types for the experiment need. 2.3 Knowledge Acquisition Ever since Papert’s Logo [19] successfully encouraged meaningful knowledge creation through hands-on activities, computer games have been proposed by scholars [5, 20] as a possible constructivist tool to facilitate knowledge formation. Dickey [4] noted that quest types might affect different types of knowledge acquisitions in an online role-playing game environment. For instance, collection quests can help to strengthen players’ declarative knowledge of a game; bounty quests and escort quests set players in a more complicated situation in which they have to explore new territories, applying knowledge of game rules and concepts, and framing tactics to fight against enemies. Both types of quests highly demand procedural knowledge.
Fig. 1. Study conceptual model
356
J. Lee and C.-Y. Chao
Declarative knowledge and procedural knowledge are termed respectively by Jonassen, Bessner, and Yacci’s designations [10]: knowing that and knowing how. The knowledge types knowing that and knowing how are sometimes also correspondingly termed schema and schemata [21]. Essential to a schema [3] for an entity or an event is a collection of information or discrete concepts which help us to identify that entity or event. Schemata are packets of schema that are composed of the interrelationships among schemas. In other words, declarative knowledge is what a player receives the information for a set of facts relevant to the execution of a particular skill during a quest. These facts represent general procedures to generate behaviors. They are interpreted and stored in the player’s declarative memory in the form of statements. Procedural knowledge is acquired after the factual knowledge has been translated into a series of procedures that can be applied automatically without other interpretive activities; tuning steps follow to help players apply this knowledge more appropriately. In sum, this study mainly tries to explore how players’ gaming motivations and quest types might affect their knowledge acquisitions in terms of declarative knowledge and procedural knowledge within an online role-playing game environment.
3 Methodology 3.1 Research Questions Three research questions were generated accordingly to test out study hypotheses. Q1. How might the player’s gaming motivations affect his/her knowledge acquisition in terms of declarative knowledge and procedural knowledge? Q2. How might the game quest types affect the player’s knowledge acquisition in terms of declarative knowledge and procedural knowledge? Q3. How might the interaction effects between the player’s gaming motivations and game quest types affect player’s knowledge acquisition in terms of declarative knowledge and procedural knowledge? 3.2 Study Design The experiment environment, Fulade Online (http://cb.fulade.com.tw), was chosen to be based on two reasons. First, one of the primary goals of the present study was to find out how different quest types and the gaming motivations might affect players’ knowledge acquisitions. Thus, the content of this game environment should be as unfamiliar as possible to most players to increase the possibility of showing signs of change in subjects’ knowledge acquisitions. The author first excluded the top ten most popular online RPG games from the poll results (March 2007~March 2008) of Gamer (www.gamer.com.tw), the biggest online game social networking service (SNS) site in Taiwan, as well as Game world of Yahoo Taiwan. Three coders were invited to review three selected online RPG games. The intercoders reliability was calculated (Cronbach’s α = .90) which fits Fleiss’ [7] agreement level of “excellent”. Fulade Online was considered as the most appropriate experiment environment for its eight beginner-level quests closely fit with the operational definitions of Bounty-collection quest and Fed express quest by this study.
The Effects of Quest Types and Gaming Motivations on Players’ Knowledge Acquisitions 357
A total of 64 eligible subjects participated in this experiment. Upon arrival, subjects were given a Gaming Motivation Questionnaire adopted from Yee [27] to identify their gaming motivations in terms of achievement, social, and immersion. Then, the researcher randomly assigned subjects into the Bounty-collection quest group and the Fed express quest group. Each group included approximate numbers of participants. After completing the required 4 quests of each group, subjects moved on to answer the Declarative Knowledge Test developed based on the framework of Ju and Wagner [11] and the Procedural Knowledge Test developed based on the framework of McClure, Sonak and Suen [16]. The average time required to complete the experiment was two hours.
4 Results The study results show that subjects’ “immersion motivation” has the most significant influence on knowledge acquisition. With higher “immersion motivation”, players appear to acquire higher declarative knowledge and higher procedural knowledge. Meanwhile, interactions between “immersion motivation” and “social motivation” decisively affect players’ declarative knowledge. Interestingly, players with low “social motivation” and high “immersion motivation” hold better declarative knowledge than players with high “social motivation” and high “immersion motivation”. In addition, the study results are consistent with Ju and Wagner [11] suggestion in which the computer games are able to facilitate the knowledge construction through the problem-solving tasks such as game quests. In order to accomplish the assigned quests, players have to identify problematic issues, find available resources, explore new game spaces, and apply or adjust techniques as well as strategies. As a result, players can easily form the procedural knowledge through the process of problem solving. In turn, their procedural knowledge becomes the base from which they can continually strengthen and reinforce their declarative knowledge to the end of the game.
References 1. Aarseth, E.: From Hunt the Wumpus to Everquest: Introduction to Quest Theory. In: Kishino, F., Kitamura, Y., Kato, H., Nagata, N. (eds.) ICEC 2005. LNCS, vol. 3711, pp. 317–328. Springer, Heidelberg (2005) 2. Bartle, R.: Hearts, Clubs, Diamonds, Spades: Plaers Who Suit MUDs (1996), http://www.brandeis.edu/pubs/jove/HTML/v1/bartle.html (Retrieved November 20, 2007) 3. Bartlett, F.: Remembering: A Study in Experimental and Social Psychology. Cambridge University Press, Cambridge (1932) 4. Dickey, D.: Game Design and Learning: A Conjectural Analysis of How Massively Multiple Online Role-Playing Games(MMORPGs) Foster Intrinsic Motivation. Education Technology Research Development 55, 253–273 (2006) 5. Di Blas, N., Paolini, P., Poggi, C.: 3D Worlds for Edutainment: Educational, Relational and Organizational Principles. In: Pervasive Computing and Communications Workshops, Third IEEE International Conference (2005) 6. Eladhari, M., Lindley, C.: Story Construction and Expressive Agents in Virtual Game Wolrds. In: The Other Players conference, Center for Computer Games Research, IT University of Copenhagen, Denmark (2004) 7. Fleiss, J.L.: Statistical Methods for Rates and Proportions. John Wiley, New York (1981)
358
J. Lee and C.-Y. Chao
8. Greenfield, P.M.: Video Games and Cognitive Skills. In: Baughman, S.S., Clagett, P.D. (eds.) Video Games and Human Development: A Research Agenda for the 80’s, Cambridge (Mass.), Harvard Graduate School of Education, pp. 19–24 (1983) 9. Greenfield, P.M., Camaioni, L., Ercolani, P., Weiss, L., Lauber, B.A., Perucchini, P.: Cognitive Socialization by Computer Games in Two Cultures: Inductive Discovery or Mastery of An Iconic Code. Journal of Applied Developmental Psychology 15, 59–85 (1994) 10. Jonassen, D.H., Bessner, K., Yacci, M.: Structural Knowledge: Techniques for Representing, Conveying, and Acquiring Structural Knowledge. Erlbaum, Hillsdale (1993) 11. Ju, E., Wagner, C.: Personal Computer Adventure games: Their Structure, Principles, and Applicability for Training. ACM SIGMIS Database 28(2), 78–92 (1997) 12. Juul, J.: Half-Real: Video Games Between Real Rules and Fictional Worlds. The MIT Press, London (2005) 13. Kucklich, J.: Perspectives of Computer Game Philology. Game Studies 3(1) (2003), http://www.gamestudies.org/0301/kucklich/ (Retrieved December 11, 2007) 14. Malone, T.W.: What Makes Things Fun to Learn: Heuristics for Designing Instructional Computer Games. In: Proceedings of the 3rd ACM SIGSMALL. Symposium and the First SIGPC Symposium on Small Systems, pp. 162–169 (1980) 15. Loftus, G.R., Loftus, E.F.: Mind at Play: The Psychology of Video Games. Basic Books, New York (1983) 16. McClure, J.R., Sonak, B., Suen, H.K.: Concept Map Assessment of Classroom Learning: Reliability, Validity, and Logistical Practicality. Journal of Research in Science Teaching 36(4), 475–492 (1999) 17. Newell, A., Simon, H.: Human Problem Solving. Prentice-Hall, Englewood Cliffs (1972) 18. Nurmi, S., Lainema, T.: Problem-based Learning in The Business Context – Can Simulation Games Improve Problem Solving? In: Proceedings of the International Conference on Computers in Education (ICCE) 2004, Melbourne, pp. 227–235 (2004) 19. Papert, S.: Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, New York (1980) 20. Rieber, L.P.: Seriously Considering Play: Designing Interactive Learning Environments Based On the Blending of Microworlds, Simulations, and Games. Educational Technology Research & Development 44(2), 43–58 (1996) 21. Rumelhart, D.E., Ortony, A.: The Representation of Knowledge in Memory. In: Anderson, R.C., Spiro, R.J., Montague, W.E. (eds.) Schooling and the Acquisition of Knowledge, pp. 99–136. Erlbaum, Hillsdale (1977) 22. Sawyer, B.: Emergent Use of Interactive Games for Solving Problems Is Serious Effort. In: 2004 Game Developers Conference - The Serious Games Summit, San Jose, CA, March 22-23, 2004, pp. 22–23 (2004) 23. Tosca, S.: The Quest Problem in Computer Games. In: Proceedings of the First International Conference on Technologies for Interactive Digital Storytelling and Entertainment, Darmstadt, Germany (2003) 24. Tronstad, R.: Semiotic and Nonsemiotic MUD Performance. In: Proceedings of 1st Conference on Computational Semiotics for Games and New Media, Amsterdam, The Netherlands (2001) 25. Wibroe, M., Nygaard, K., Bøgh Andersen, P.: Games and Stories. In: Qvortrup, L. (ed.) Virtual Interaction. Springer, London (2001) 26. Yee, N.: Facets: 5 Motivation Factors for Why People Play MMORPG’s (2002), http://www.nickyee.com/facets/home.html (Retrieved December, 12, 2007) 27. Yee, N.: Motivations for Play in Online Games. CyberPsychology and Behavior 9(6), 772–775 (2006)
Self-movement Feeling Generation in Sports Watching with Screen Movement via Pan-Tilt Steerable Projector Hiroshi Noguchi1,2, Kei Yoshinaka2, Taketoshi Mori2, and Tomomasa Sato2 1
2
CREST, Japan Science and Technology Agency Graduate School of Information Science and Technology, Univesity of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656 Japan {noguchi,yoshinak,tmori,tomo}@ics.t.u-tokyo.ac.jp
Abstract. This paper describes system displays video frame of sports broadcasting with screen movement by an active projector. Since the screen moves as the camera operator controls their camera in the stadium by automatic detection of background movement from video frames, the watchers are force to move their heads in home environment. The head movement helps the watchers to feel self-movement as if they ware located in the stadium. The experiments demonstrate that the head movement when the watchers track the moving screen with their eyes and heads generates sense of self-movement and active feeling about players. Keywords: Active Projector, Sense of Self-Movement, Screen Movement, Sports Watching.
watching on television contains no interaction. The user only watches on television and does not contact to television. Therefore, control factors for perception are unavailable. Since the system does not contain no computer graphics image, realism factors is unrelated. The place on watching in home environments is usually fixed. Distraction factors are difficult to provide. Rather than these factors, generation of self-movement feeling is easy to realize. Examples of self-movements in sports watching are head movement and body twist. The spectators for such sports that need large play areas as football and basketball move their gazes and heads as the players and balls move in the large stadium. On the other hand, in watching on television, a camera operator moves their camera to keep the players and the balls center in the video frame. The spectators on television do not need to move their head and body. They cannot perceive self-movement. If the system can provide sense of self-movement to the watchers, they can feel presence on watching in the stadium. Our research aim is construction of display system that stimulates perception of self-movement in watching sports in television. Target video image is TV program about sports broadcasting. In video image, there are various scenes such as scene switching and zooming. Since our goal is generation of sense about self-movement, we treat with only tracking scene about the balls and the players including overall of field.
2 Generation of Self-movement Sense with Screen Movement There are two methods to generate movement of watcher’s head and body. One method is that the system moves the watcher's body directly in accordance with scenes on TV program. The other is that the system moves gaze area of the watcher. The former requires large devices for user movement. It is unsuitable for home use. We utilize the latter method. If the user watches TV program on large display, the user seems to move his/her head. However, as above mentioned, in TV program, the players and balls are usually on center in the frame. The user does not need to watch sports with head movement. For head movement, the system is required to move the gaze area. In watching television, gaze area is equivalent of screen area. The screen area of the display system is needed. The movement of gaze area also generates distraction factors for presence. The narrow gaze area and movement create attention of the watchers. The attention helps the users to feel involvement and immersion in watching. There are 3 types of approaches for screen movement. • Head Mount Display (HMD) One approach is that the system covers the user’s field of view with special display device such as HMD and the system provides screen with head position of the user. This method’s merit is that the system can generate screen movement of all directions and the degree of immersion is high. Cost and constriction of the special devices for the users are demerits. The measurement about head position is needed. It also forces the users to wear special sensors for head measurement. In watching sports, the spectators
Self-movement Feeling Generation in Sports Watching with Screen Movement
361
do not watch all directions but the front of them. The field of all directions is not needed. This approach is not suitable for home use. • Large Display Another approach is that the system equips large display such as 4K display and high-resolution projector. The system displays only clipped area of whole screen and moves the clipped area with video images. This method has the merits; easy implementation of screen movement, fast moving and resizing of screen. The method also contains the problems. The large display requires large area for equipment. The movable area on screen movement is limited to display size. • Active Projector The other method is the system physically moves screen of the display device. While the TV monitors are too heavy to move on the wall, the projectors are easy enough to move their bodies. The pan and tilt movement of the projector body realizes screen movement on large area. The movable projector has demerits; projected image is distorted, screen area tends to be small and speed of screen movement is limited. These methods contain merits and demerits. We utilize active projector for realization of screen movement. This is because 1) the active projector covers larger area than large display and 2) the screen includes no frame [2]. Once the user notices the frame of the display, the user finds out limits of screen movement. This prevents immersion and decreases degree of sense on presence. Display system without aware of screen frame is important. The several research projects [2][3][4][5] utilize the active projectors. The main usage of the projector is display of static image everywhere. In the addition to the disply of static image, the display system with screen movmenet is required to synchornize displaying the image and moving the screen. This is interesting topic about active projector.
3 Display System Implementation We have been utilized the pan-tilt streerable projector for informational support of the occupants in the room [6]. Performance of the projector is shown in Table 1. The projector consists of traditional projector, tilt motor and pan motor. The projector is controllable with DMX communication. Pan and tilt angles are measurable with equipped rotary encoders. The projector’s focus and zoom are also controllable. In usage of screen movement, change of distance between the projector and projection plane is small. In the paper, we do not control focus and zoom parameter. Configuration of the projection system is illustrated in Fig. 1. The display system projects video images by the following proceduce. Table 1. Specification of pan-tilt steerable projector Range (degree) Pan Tilt
-180 to 180 -120 to 120
Maximum Speed (degree/sec.) 80 80
Measurable Angle Resolution (degree) 0.09 0.05
362
H. Noguchi et al. video source background detector
display target pos.
pan and tilt control
calculated result
control data image distortor 画像補正部 pan and tilt control
image frame フレーム画像 テクスチャ化 texturization
target pos. image
pan and tilt
3D 次元 renderer レンダラ
target pos.
Offline phase
Online phase
Fig. 1. Display System Diagram
3.1 Detection of Image Background Movement Ideally speaking, position of moved screen is desired to be equal to the field of the spectator’s view in the stadium. However, the information related to position of the watching view has disappeared in video image of TV program. Fortunately, if switching of camera does not occur, the camera operators control their cameras to track the players, the ball or something to focus. It is assumed that this camera operation is similar to movement of eyeshot in watching sports on spectator’s seat. Therefore, screen control with the parameter of estimated camera movement from the video images is suitable to generate sense of presence in watching sports on the display system. Video images that track the players or balls in the playfield consist of large area of moving background and small area of tracking target objects. Background movement is generated with camera movement. Thus, camera operation is estimated from detection of background image movement. For detection of background movement, we utilize optical flow of the video images. The display system detects feature points in each image converted from video frame and calculates optical flows from Bouguet’s method [7],
Fig. 2. Detected Background Motion: White line means detected optical flows of feature points. Large allow in the bottom represents 5-times vector of detected background motion.
Self-movement Feeling Generation in Sports Watching with Screen Movement
363
which is the method that generates optical flow hierarchically from harsh image to high resolution image. Vectors of calculated optical flows are quantized in units of length and angle. The mode of the quantized vectors is regarded as camera movement (Fig. 2). 3.2 Planning of Projector Control Screen movement parameters are easily calculated from detected background movement. Displacement value of screen (dxb, dyb) in single video frame is calculated by given parameters: screen size (Wd, Hd), screen resolution (Wr, Hr) and movement of background (dxb, dyb). (1) The position at the video frame is calculated from integration of screen movement vectors. The needed size of screen movement in total video frames is also found. The system can calculate appropriate screen size from calculated total area of movement and projectable plane size. The maximum speed of projector pan-tilt is also found with distances of screen movements in frames. The system decides whether the video image can display by specification of the active projector. The system calculates pan and tilt value of the active projector in each frame from needed screen position above calculated. The active projector is regarded as a robot system that includes 2 degrees of rotation freedom and 1 degree of liner freedom, which is a kind of virtual liner joint between optical center and projection plane (Fig. 3). In the robot system, holomorphic tilt and pan values are found by inverse kinematics method.
Fig. 3. Freedom of the pan-tilt projector
3.3 Image Correction Projected image on plane is distorted as long as projection ray’s direction does not equal to normal direction of the plane. Distorted image decreases sense of presence. Mismatch between target screen position and calculated position arises from difference between the optical center of the projector and the center of rotation movement. The projectors contain the opposite characteristics of the cameras (Fig. 4). The camera calibration techniques resolve these problems. In fact, intrinsic parameters are calculated with the projected grid images and measured position. Because of non-linearity with offset between the optical center and the center of rotation, extrinsic parameters are difficult to calculate. We give the parameters from blueprint of the projector. The lens of the projector contains no deformation. Distortion calibration is not needed.
364
H. Noguchi et al.
Measurement in Pattern Projection at Known Postures
rendering plane camera
screen plane
Intrin sic Paramete r Calculation
rendering image
corrected image projection
Fig. 4. Projector Calibration and Image Correction
The system generates distorted images with 3D rendering technique. The system renders the 3D scene where optical center of the projector is virtual camera and the polygon whose texture is target image locates at the target position (Fig. 4). 3.4 Synchronization of Video Images, Projection Image Generation and Pan-Tilt Control In online projection phase, in order to watch without feeling of strangeness, accurate synchronization of video images, distortion image generation and pan-tilt control of the projector. There are several delays in the system; delay of projector control, delay of video image transmission and duration of computation for distortion. The transmitting delay from distorted video frames to the projector is the longest. The system makes delays of pan-tilt control in the time duration of transmission for synchronization. The values of controlled pan and tilt do not always match with target values because of motor performance, control noise and control delay. Before online generation of screen movement, the system controls the projector with calculated route of the screen and measures pan and tilt angles from the projector. The parameters for projection image distortion are decided from measured pan-tilt values. If the system detects mismatch between target pan-tilt value and measured value, the system modify the parameter at the image distortion phase. This mechanism covers as high speed as the projector cannot chase the target screen and absorbs vibration movement of pan-tilt.
4 Experiments about the Display System We evaluate that the display system make the watchers feel self-movement. Total 14 subjects watch the 3 kinds of videos. Experimental room design is shown in Fig. 5. The prepared videos are approx. 15 second-length videos about soccer, valley ball and video game. The screen size is equivalent of 50-inch display television. The video features are shown in Table 2. The video scenes of the soccer video are shown in Fig. 6. The video game is not a sport, but in the video game, background except the leading character moves. We assumed that the screen movement makes the watcher feel something except presence with video that includes background movement. In order to investigate relationship between head movement and sense of self-movement, the subjects located in 1, 2 and 2.3 [m] (directly below the projector) front of the screen. The experiment protocol is the following. Firstly, the subject watches the video with no movement of the screen. Secondly, he/she watches the same video with movement of the screen. After watching, the subject writes the relative score 1(bad) to 5 (good)
Self-movement Feeling Generation in Sports Watching with Screen Movement 2.325m
365
x
2.51m
Active Projector
1m
1m
Active Projector 1.9m(Height)
z
Projection Plane
: Watching Points
Fig. 5. Experimental Setup Table 2. Features of the video images in experiments Image Source Soccer Valley Ball Video Game
Movement Range (W x H [m]) 1.3 x 0.1 1.0 x 0.0 2.3 x 1.1
Maximum Speed in Width (m/s) 0.2 0.6 0.4
Maximum Speed in Height (m/s) 0.1 0.0 0.5
Fig. 6. Experiment Scenes in Soccer Video
against the video watching in no screen movement. The questionnaire items are 1) visibility of scenes, 2) self-movement feeling and 3) active feeling about target players and characters. Result about visibility of scenes is shown in Fig. 7. The subject scores are low rather than fixed screen. In the screen movement, it is difficult to watch the scenes. Scores of the video about valley ball that includes the highest speed screen movement are low. This means that fast movement decreases the degree of visibility.
366
H. Noguchi et al. relationship between videos and scores Valley Ball 12
12
10
10
6 4 2
Score
1
2
3
Score
4
4 3.5
8 6 4
3 2.5 2 1.5
2 0
5
4.5
Score
8
0
5
Video Game Number Of People
Number Of People
Number Of People
Soccer
1
1
2
3
Score
4
0.5
5
0
2.3m
Soccer
Valley Ball
Video Game
Fig. 7. Result about Visibility of Scenes
Result about self-movement feeling is illustrated in Fig. 8. As expected, the sense of self-movement is higher in screen movement rather than fixed screen. The scores are related to distances of subject’s locations. The subject in the location near the screen should move their head widely. The subject feels high degree of self-movement. This shows degree of head movement with screen movements are tightly related to sense of self-movement. 5
Video Game
Valley Ball
4 3.5
Score
Number Of People
Number Of People
Number Of People
Soccer
relationship between distances and scores
4.5
3 2.5 2 1.5 1 0.5
Score
Score
Score 2.3m
0
1m
2m
2.3m
Fig. 8. Result about Self-Movement Feeling
Result about active feeling about target players and characters is shown in Fig. 9.Active feeling is also higher in screen movement rather than fixed screen. On the contrary of the self-movement feeling, active feeling is not related to distances. This feeling arises from only screen movement without the subject’s head movement. This effect appears in video game. The implemented system has capability of increasing the degree of self-movement feeling and active feeling for something except sports. relationship between videos and scores Valley Ball
Score
Video Game
Score
Number Of People
Number Of People
Number Of People
Soccer
Score
Score 2.3m
Soccer
Valley Ball
Fig. 9. Result of Active feeling about Target Players and Characters
Video Game
Self-movement Feeling Generation in Sports Watching with Screen Movement
367
In all experiments, in the location directly below the projector, although the subjects watch the video at the similar view of the camera operator, the scores are not dramatically different from scores in the other locations. This means accuracy about gaze movement is not necessary for sense of self-movement and active feeling.
5 Conclusion We constructed the display system including the active projector moves the screen with video frames for the watchers to feel self-movement. The system displays the sports scenes of TV programs with screen movement by detection of background movement in video frames. The experiments demonstrate the system enhance the watchers’ perception of self-movement and active feeling about the players rather than televisions, whose displays are fixed. The system currently cannot treat with the video including zoom and switching of scenes. The system needs offline measurement about pan-tilt value of the projector. Resolution about these problems is a future task. The current system only generates sense of self-movement. It is so far from perception of presence. We will challenge to generate other factors about presence in watching sports.
References 1. Witmer, S., Singer, M.: Measuring presence in virtual environment: A prensece questionnaire. Presence: Teleoperation and Virtual Environment 7, 225–240 (1998) 2. Pinhanez, C., Podlaseck, M.: To frame or not to frame: The role and design of frameless display in ubiquitous applications. In: Beigl, M., Intille, S.S., Rekimoto, J., Tokuda, H. (eds.) UbiComp 2005. LNCS, vol. 3660, pp. 340–357. Springer, Heidelberg (2005) 3. Pinhanez, C.: The everywhere displays projector: A device to create ubiquitous graphical interfaces. In: Abowd, G.D., Brumitt, B., Shafer, S. (eds.) UbiComp 2001. LNCS, vol. 2201, pp. 315–332. Springer, Heidelberg (2001) 4. Yang, R., Welch, G.: Automatic and continuous projector display surface estimation using everyday imagery. In: WSCG, pp. 328–335 (2001) 5. Raskar, R., Baar, J., Beardsley, P., Willwacher, S., Rao, S., Forlines, C.: ilamps: Geometrically aware and selfconfiguring projectors. In: SIGGRAPH, pp. 809–818 (2003) 6. Noguchi, H., Mori, T., Sato, T.: Attentive Information Support with Massive Embedded Sensors in Room. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4551, pp. 883–892. Springer, Heidelberg (2007) 7. Bouguet, J.: Pyramidal implementation of the lucas kanade feature tracker: description of the algorithm. Technical report, OpenCV Document (2000)
Design of Interactive Emotional Sound Edutainment System Myunjin Park and Kyujung Kim School of Media, Soongsil Univeristy 1-1 Sangdo-1-dong, Dongjak-gu, Seoul, Korea {cooljin,kyu}@ssu.ac.kr
Abstract. This paper introduces an emotional sound edutainment system for children to learn basic musical composition called as musical education sound interactive game (MESIG) employing a new type of user interface. Developed interactive game interface provides children to enjoy the game, so that they learn how to compose musical notes with touching the tangible objectives instead of using ordinary input devices. This way on experiencing and playing the computer games has been evolved to use the body and hands’ movement so as to interact with the game in virtual environment, which brings out interest for the children and their learning capability becomes more effectively improved. This system introduced in this paper requires a single camera and carries out skin color model tracking function to detect hand gesture as input device for playing the game. This computer vision technique based on image processing makes possible to operate an expressive interactive musical education system. To exploit the effectiveness, evaluation and analysis works are accomplished upon the realization of sound edutainment game. Keywords: Interactive games, edutainment, skin color model, computer vision.
Design of Interactive Emotional Sound Edutainment System
369
The developments of interactive and participatory environments that combine the physical and virtual have brought as a natural continuation to the computer game industries. This development area not only restricts to entertainment domain but indirectly creates a learning environment for children. A critical review of examples of immersive virtual reality worlds created for children, with particular attention given to the role and nature of interactivity, is attempted. Interactivity is examined in relation to learning, play, narrative, and to characteristics inherent in virtual reality, such as immersion, presence, and the creation of illusion [3]. Usual virtual reality games try to make the user interface transparent or invisible, where user does not realize the existing of interface to immerse into the games. The conventional method, the player has to use special devices such as gloves or head-mounted displays [4][5]. This motivates us to study a simple manner of children to interact with interesting game contents in interactive digital environment without any bulky device. The interactive digital environment is referring as a system which adapts to the user’s action and allows varied degrees of freedom to control over the time or space [6]. The study of the user interaction in virtual environment based on activity theory framework [7] is analyzed and decision to design activity contents to generate larger motivating in game. In this paper, we introduced a tangible input to interact with the virtual environment. Human motion interacts with the virtual environment to gain the children attention and interest to play the games. This interactive games interface able to develop and gain children attention, interest, and learning how to compose simple musical notes throughout their gaming time experience. The proposed MESIG System is described in section 2 including the explanation of the details of how to track user’s body and hand movement and the process for generation and rendering snowflakes and musical notes object by using DirectX. In the last subsection of section 2, the way how to perform the collision detection is described. Section 3 shows the experimental results. The conclusion is summaries in section 4.
2 Proposed MESIG System The proposed MESIG system provides emotional playing activity game to user, such that appropriate sounds relevant to action of user’s body and hand and snow flake movements. The functions of the proposed system can be categorized into three actions. As the first, the image of snow falling is provided to the passengers drawing their attractions if there is no one in front of camera. As the second action, when the specific user stands at the camera, all the snowflakes turns their shape into musical notes having their own colors such that the user feels like to touch. As the last action, by tracking user movement, when the user’s hand touches the musical notes, the collided snow flake is disappeared and the corresponding sound classified by its color is emitted. The procedure for processing of this system can be summarized in Figure 1 as shown below. The overall system determines whether or not there is a person in front of screen. Towards this, the image captured by camera should be changed a binary image using a background subtraction method and recognized user’s body and hand using a
370
M. Park and K. Kim
Camera
Projector
All Object
Input Image Background Subtraction
Each Snow Object Coordinate Compute
Object Image Using Component Labeling and Threshold
Render Show Object
User Object Image
Each Object Projector Coordinate to Screen Coordinate
Camera Coordinate to Screen Coordinate
Each Snow Object Move
Each Note Object Move
Object Test Snow Object
Yes
Note Object
No Change Texture From Snow to Note
Collision Detection No Yes Output Sound
Delete Object
Fig. 1. Overall structure of the proposed system MESIG
component labeling method. As soon as the existence is confirmed, all of the snowflakes subsequently changed into musical notes. Then, utilizing the position information about regions associated with formerly extracted hand and face, the collision detection is performed if the user’s hand touches the projected musical notes. Whenever the collision is detected, corresponding musical note becomes disappeared with emitting the sound, at sequel, the new object is appeared. Surely, repeating this process provides the interactive environment to the user controlled by tracking and moving user’s hand, so that the snowflake is rendered at the instant of sensing the touch of snowflake with the help of DirectX engine. 2.1 How to Track User’s Movement Since the user described as a kind of interface, when the user touches snow flake, its shape turns out to be musical notes, and relevant sound is generated. Here the importance is how to track the user movement. Towards this, this system uses background subtraction technique in order to segment out objects of interest from captured image in real-time. Objects are generated using background subtraction and shadow detection based on color difference [8]. From the objects images, we can change from snow flake objects to musical nodes and the musical is disappeared. We used threshold value of RGB
Design of Interactive Emotional Sound Edutainment System
371
color to subtract the background and foreground images. Figure 2 shows the silhouettes pixels extraction based on RGB color. The object of interest generation uses two methods. First, we use the distance in order to separate an object pixel and a background pixel, and the relevant equation for the distance is the following;
d (i, j ) = cr (i, j ) − cb (i, j )
(1)
where c represents the norm of a vector c. Second, we use difference to separate shadow can be expressed by (2) where • is the dot product operator.
distance Background Image C (R , G , B ) b
b
b
Run-time Image C (R , G , B ) r
r
r
r
b
difference
Fig. 2. Overview of objects pixels extraction of (i,j)th pixel color
We define a background image whose (i,j)th pixel color is denoted by the vector Cb(i,j) or Cb(Rb, Gb, Bb), and the run-time image whose (i,j)th pixel color is denoted by the vector Cr(i,j) or Cr(Rr, Gr, Br). The distance is computed by using intensity difference between the run-time and background pixels. If the intensity difference is very large, then the pixel should be a silhouette pixel. If the intensity difference is very small, then the pixel must be a background pixel. Shadow pixel is determined by performing pixel determination to exclude silhouette and background pixel as described above. The value difference is the angle between the vectors Cr and Cb in the RGB color domain and hence is a measure of the color difference between the run-time and background pixels in Figure 3. After background subtraction, the extracted object is shown in Figure 3. The noise could be appeared due to musical notes and snowflakes when real-time captured image is extracted as shown in Figure 3(b)-(c) based on background subtraction. In order to discriminate the noise and the person, each component is classified into the pure noise and the person by comparing those with the threshold value. Here, it is easy to determine whether the component is
372
M. Park and K. Kim
(a) Background Image
(d) Object Image A
(b) Capture Image A
(c) Capture Image B
(e) Object Image B
(f) Result of Labeling
Fig. 3. MESIG of user tracking
the noise or the person since the component corresponding to the person has more large number of pixels. In the proposed system, the threshold is fixed to 1200, so that the component is recognized into the person if the number of pixels is more than 1200, otherwise the noise. Figure 3(f) is the result after the discrimination and the suppression of noise from the subtraction image shown in Figure 3(e), and the shaded component is recognized as the person. 2.2 How to Perform Generation and Rendering Snow Flake Object Using DirectX This section utilizes DirectX as tool for generating realistic snowflakes or musical notes. It is well-known that DirectX provides comfort API to control the functions of high performed hardware. Thus, more powerful multimedia applications can be easily developed. In order to project realistic snowflakes or musical notes to the user, the eight steps are processed as shown in Figure 4 by using DirectX. First, the user sets up the total number of snowflakes. If the number of snowflakes is not enough as determined, residual snowflakes are generated by random selection of the number of snowflakes in random. After performing this step, as the second step, all the snowflakes has its own projection coordinate, and its magnitude and moving path are determined one by one. Each snow flake object follows its own moving path. As the third step, the presence of the user is testified. At fourth step, all the snowflakes turn their shapes into musical notes. Then, the collision detection is processed whether there is any touch between hand or face and musical notes in fifth step. If there is any collision, the corresponding musical note emits a sound and a color in sixth step. Here, each musical note can have 7 colors or sounds. The collided
Design of Interactive Emotional Sound Edutainment System
1
373
Initial Object Set 8
2
Moving Objects
3
User Object Test
4
Change Texture From Snow to Note
5
Collision Test
Moving Objects
Elimination Snow Object
7
Output Sound
6
Collision
No Collision
Fig. 4. Processing steps for the proposed MESIG system
musical note becomes disappeared in seventh step, whereas the object without collision keeps moving its position in eighth step. After the sound note is eliminated, the number of snowflakes will be decreased by the number of the disappeared. These 1 to 8 steps are repeated until all the snowflakes are disappeared. Figure 4 shows these processing steps for generation, movement, change and elimination of snowflakes. 2.3 How to Perform Collision Detection It is important to detect the event that snowflakes collide with user movement. Since the user performs interaction by using body or hand movement, this interface can provides very interesting situation. In order to perform the collision detection, the functions for tracking the user’s hand or face explained in Sec. 3 and the localization of projected snow flake explained in Sec. 4 should be performed. However each snow flake has its own coordinate, it hard to perform the collision detection [8] by only using the location of projected snowflakes. In other words, the coordinate oriented from the camera tracking the user’s hand or face is totally different from that from projector for snowflakes, it is necessary to use the common coordinate. This common coordinate can be updated by transforming the coordinate of projector into that of screen. After all, the collision detection is performed by this coordinate transformation. Since the coordinate of screen can be calculated by the projected region, it can easily test whether the collision happens or not. To make these coordinate changes, the keystone correction technique can be generally utilized for providing the corrected image wherever the image is projected from the projector with considering relative positions between the camera and the projector. Here, in order to detect the collision, the following equations (3) and (4) are used to transform the coordinates of the camera and the projector into the common screen coordinate. First of all, the relationship between the coordinate of projector and that of camera is designated as in LHS of equation (3) with whose corresponding transformation matrix T . And secondary, the relationship between the coordinate of
374
M. Park and K. Kim
screen and that of camera is defined as in RHS of equation (4) by using C . Using these relations, the relationship between the coordinate of screen and that of projector can be expressed in equation (4), whose relevant transform matrix defined as P
⎛ x proj ⎞ ⎛ xworld ⎞ ⎜ ⎟ ⎜ ⎟ C × T × ⎜ y proj ⎟ ≡ ⎜ yworld ⎟ , P = C −1 × T ⎜ 1 ⎟ ⎜ 1 ⎟ ⎝ ⎠ ⎝ ⎠
(4)
−1
Utilizing equations (3) and (4), the coordinate change from camera to screen can be accomplished, and Figure 4 visually depicts these coordinate transformations step-bystep. From the results of these transformations, the collision detection is performed by matching the coordinate of snow flake with that of either user’s hand or face.
3 Experimental Results and Analysis In this section, we present result of the proposed game MESIG. The proposed game required a camera (LG – color CCD camera), a projector, screen and a personal computer (Intel® Xeon™ CPU 3.20GHz, 2.56GB RAM, Windows XP OS) to execute the MESIG. The display screen will be change when the real-time camera of the system detect human or hand skin color. Then the musical notes display will replace the snowflakes display. Figure 5 (a) shows the overall system setup of MESIG which is projected to a wide screen; (b) shows the initial display of snowflakes when no player detected in real-time camera. The display screen environment will change from snowflakes to musical notes when a player face or hand skin color is detected. The musical notes have 7 basic elements of sound notes which are represented in rainbow colors. When player hand touches on a particular note, the sound notes will be generated. Figure 6 shows the result of MESIG with different players captured from the normal digital camera. The system shows the interactive game environment by detect the player skin color. When the player’s hand touch on the musical notes, a sound will be generated based on the notes representation. This MESIG will motivate the children interest and creativity to compose musical notes. The sound notes only response to one time hand touch of the musical notes.1 This analysis is accomplished for 30 children as the examination group whose age ranges from 4 to 7. The pre-exam is conducted before the children play, then after the post-exam is executed to analyze the degree of effectiveness. In order to determine how the educational interest and achievement is improved, the basic questionnaires are assigned to the children, which are including the preference and the skill about 1
www.cooljin.net/conf08/iadis_elearing
Design of Interactive Emotional Sound Edutainment System
(a) Overall system
375
(b) Initial display of snow flakes
Fig. 5. Proposed MESIG system environment
(a) Initial display of snow flakes
(b) Player near to the screen
(c) Player1 touch the notes
(d) Player2 touch the notes
Fig. 6. Experimental result shows the player plays the MESIG using hand to touch the musical notes to generate a sound notes
computer games. For fair evaluation, the corresponding questionnaires are selected from the pool made by the people who have the major on child education. Experimental results are subsequently analyzed by conducting SPSS program. Among those, firstly, the sound educational achievement is exploited by evaluating T-value a priori. At second, after the end of executing game, to concrete the inference a priori made, the posteriori T-value evaluation process is conducted at sequel. Significance level of all the experiments is fixed, whose corresponding value is less than 0.05. Table 1 shows the results of evaluation before and after play of the sound edutainment game.
376
M. Park and K. Kim Table 1. Level of interest for sound edutainment game
Table 2. Achievement of sound perception education Subject
Evaluation
Sound
preexamination postexamination
perception
Num. of Average Student (N)
Standard Deviation
30
11.47
3.42
30
13.84
2.65
T-value
P-value
5.567
0.01
As shown in Table 1, before the game play, the level of interest is not so high, but after the game play the level of interest is positively improved. In Table 1, among examination group, it can conclude that the number of children having high level of interest is increased from 27% up to 43%. Besides the increment of interest, the capability of sound perception is also enhanced. The level of achievement corresponding to the children in examination group is improved by 2.37 points. According to t=5.567 and p=0.01. From the statistical point of view, the results in Table 2 can be said to be meaningful because p < 0.05. And the results show that the sound perception education via the interactive game is quite effective to the children.
4 Conclusion This paper introduced an interactive game for children to learn basic musical composition. The proposed system is an interactive game named as musical education sound interactive game (MESIG), which can be treated as a new type of user interface. Developed system is designated an interactive game interface for children to enjoy and play the games while learning how to compose musical notes using tangible input instead of ordinary input devices. Thus, the proposed MESIG system provides emotional playing activity game to user, such that appropriate sounds corresponding to action of user’s hand and snow flake movements. The functions of the proposed system can be categorized into three actions. To verify the effectiveness of the proposed game for sound perception edutainment, the fair statistical analysis is conducted. All the results show the improvement of sound perception capability as well as the increment of interest. This indicates that the interactive game controlled by body-motion can strongly motivate the reason for learning, so that this new way of
Design of Interactive Emotional Sound Edutainment System
377
education via edutainment in a form of interactive game increases the level of learning interest and improves the capability of sound perception more effectively. Acknowledgments. This work was supported by Soongsil Research Fund.
References 1. Madej, K.: Towards digital narrative for children: from education to entertainment, a historical perspective. Comp. in Entertainment (CIE) 1(1) (2003) 2. Stromberg, H., et al.: A group game played in interactive virtual space: design and evaluation. Symposium on Designing Interactive Systems (2002) 3. Roussou, M.: Learning by doing and learning through play: an exploration of interactivity in virtual environments for children. Computer in Entertainment (CIE) 2(1) (2004) 4. Bobick, A., et al.: Perceptual user interfaces: The KidsRoom. Communication, ACM 43(3), 60–61 (2000) 5. Pausch, R., et al.: First Steps Toward Storytelling in Virtual Reality. In: Computer Graphics Proceedings, ACM SIGGRAPH. Annual Conference Series, pp. 193–203 (1996) 6. Talin: Real interactivity in interactive entertainment. In: Dodsworth Jr., C. (ed.) Digital Illusion: Entertaining the Future with High Technology. Addison-Wesley, Reading (1998) 7. Nardi, B.A.: Context and Consciousness. In: Activity Theory and Human-Computer Interaction. MIT Press, Cambridge (1996) 8. Kong, G., et al.: A real time system for robust 3d voxel reconstruction of human motion. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition., vol. 2, pp. 714–720 (2000) 9. Sukthankar, R., et al.: Smarter Presentations. Exploiting Homogra-phy in Camera-Projector Systems. In: Proceedings of International Conference on Computer Vision, vol. 1, pp. 247–253 (2001)
Understanding Online Game Addiction: Connection between Presence and Flow SungBok Park1 and Ha Sung Hwang2,*
*
1 Hanyang University, visiting professor, Graduate School of Journalism and Mass Communication 17 Haengdang-dong, Seongdong-gu, Seoul, South Korea [email protected] 2 Dongguk University, assistant professor, dept. of communication studies 26, 3 Phil-dong, Chung-gu, Seoul, South Korea [email protected]
Abstract. Addictive behavior in online gaming has been an important research topic since it has been one of the most popular activities in entertaining for younger people in Korea. However, despite the growing popularity of online games, empirical studies about the effects of immersion to the online game behavior are relatively rare. By applying two psychological concepts –presence and flow-the present study investigates how different types of immersion affect on online game addiction. Results show that both presence and flow play significant roles in online game addiction, however, flow mediates the relationship between presence and online game addiction. Based on these findings, implications and suggestions for future studies are discussed. Keywords: Presence, Flow, Online Game, Addiction, Virtual Reality.
Understanding Online Game Addiction: Connection between Presence and Flow
379
studies. Hence, the present studies argue that immersion into online games is after all a result of experiences of presence and this feeling of presence may be correlated to addictive behavior in online games. On the other hand, Choi and Kim [2] found that people continue to play online games if they have optimal experience commonly know as the concept of flow. In addition, previous research found that flow state positively related with the addiction to online game. Some scholars have insisted that presence can be defined as a special type of flow experience that occurs during teleoperations. For example, Fontaine [3] states that flow experience produces peaks of involvement that seem to be similar to the vividness of presence. Although previous research suggested that the two concepts-presence and flow—share similarities such as the immersive component and intensive feelings of involvement, it is not well understood relationships between these two concepts and how experiences of presence and flow are related to online games addictions. Therefore, this study aims to examine whether presence and flow are related in the context of online game. Moreover, the study explores the impact of the experience of presence and flow on online game entertainment and addiction.
2 Theoretical Background: Social Presence 2.1 Presence and Online Game The concept of presence has become central to theorizing about the advanced humancomputer interaction such as virtual reality systems, as well as traditional media such as television, film and books. Researchers have begun to realize that the feeling of presence is at the heart of all mediated communication environments, because presence is at the heart of human’s desire to use media to move beyond the limits of body and sensory channels[4]. After an extensive review of presence-related concepts and their explications, Lombard and his colleagues define presence as “the perceptional illusion of non-mediation” [5,6]. This illusion of nonmediation refers to a phenomenon in which “…a person fails to perceive or acknowledge that the existence of a medium in his or her communication environment and responds as he or she would if the medium were not there.” Therefore, presence describes a state of consciousness that gives the impression of being physically present in a mediated world. Commonly, presence is defined as the sensation of ‘being there’ in a mediated environment Since presence is primarily a subjective sensation, it has been argued that ‘subjective report is the essential basic measurement’. Indeed, the majority of studies measure presence through post-test questionnaires and rating scales, which have the advantage that they do not disrupt the media experience and are easy to administer. For instance, Slater et al [7] asked participants, after exposing them to a Virtual reality systems, to answer three questions on Likert scales (1-7) that served as an indication of presence: (1) their sense of ‘being there’ in the computer-generated world, (2) the extent to which there were times when the experience of the computergenerated world became the dominant reality for the participants, thereby forgetting about the ‘real world’ outside, and (3) whether they remembered the computergenerated world as ‘something they had seen’ or as ‘somewhere they had visited’.
380
S. Park and H.S. Hwang
According to Slater et al.[7], the ‘experiencing-as-a-place’ is central to understanding presence in Virtual environments: people are there, they respond to what is there and they remember it as a place. As such, presence has received substantial attention from the virtual reality programs, it is becoming increasingly relevant to online game environment. Presence has been recently been identified as a potentially important variable in online game research because it may affect use and a variety of outcomes of exposure, ranging from enjoyment to aggression. Although few studies have examined the relationship between the experience of presence and online game use, Tambrini et al. [8] found that playing a game created a strong sense of presence than observing a game, presumably due to the additional interactivity. Even though many technological features of online games are expected to contribute to the sense of “being there”, Bracken and Skalski [9] suggest that image quality impact both the level and types of presence dimensions experienced by video game players. In addition, previous studies provide compelling evidence for the mediating effect presence in the context of e-commerce [10], human-robot interaction [11] and entertainment games [12]. Especially in the context of entertainment games, the enjoyment and evaluation of video games are mediated by feelings of presence during game playing [12,13]. Consequently, the above research findings indicate that the experience of online games relatively complies with the psychological state of “being there” in a mediated virtual environment. Following this logic, are game users who are more immersed into online games more likely to become addicted? Few studies, however, concern with this topic, the investigation is worthwhile of intense exploration. 2.2 Flow and Online Game Flow theory can be referred to as the ‘psychology of optimal experience,’ which in recent years has been applied to the Internet behavior by some research. Studies suggest that cyberspace behavior has been reported as highly correlated with flow experience. As in his early definition, Csikszentmihalyi [14] sees "the holistic experience that people feel when they act with total involvement" (p.36) Further, he states (p.3) that flow experience are those optimal and enjoyable experience in which we feel “in control of our actions, masters of our own fate…we feel a sense of exhilaration, a deep sense of enjoyment” [15]. As a result, flow experience is more an emotional state during the process of the user’s activity. Csikszentmihalyi [14] originally identified four flow components: control, attention, curiosity, and intrinsic interest. To supplement Caikszentmihalyi's concept of flow experience, Trevino and Webster characterized four dimensions of flow [16]. According to them, within the human-computer interaction experience, flow incorporates the extent to which (1) the user perceives a sense of control over the computer interaction, (2) the user perceives that his or her attention is focused on the interaction, (3) the user’s curiosity is aroused during the interaction, and (4) the user finds the interaction intrinsically interesting. In Hoffman and Novak [17] flow is defined in terms of the experience of flow (intrinsic enjoyment, loss of self-consciousness), behavioral properties of the flow activity (seamless sequence of responses facilitated by interactivity with the
Understanding Online Game Addiction: Connection between Presence and Flow
381
computer and self-reinforcement), and its antecedents (skill/challenge balance, focused attention, and telepresence). Drawing upon this suggestion, Ghani and Deshpande [18] further argue that two important components-enjoyment and concentration- leads to diminished sense of time during the particular activity. It seems that while in the flow state, a user experiences a sense of happiness, accompanied by a feeling of an exploratory distortion in time perception, which often occurs in the absence of time pressure when conducting a specific activity that brings in positive feedback. As a result, the time allocation for the activity increases. However, similar to the distortion in time perception, during activities users may experience another perception--distortion of the sense of space. Such perception is related to the concept of presence as Heeter [19] uses the term "(tele) presence" to describe the sense of presence when an individual is physically far away from the scene. Indeed, some scholars insist that flow and presence shares similarities because these are terms to describe immersive component and intense feeling of involvement [3]. It is believed that flow experience produces peak of involvement that seem to be similar to the vividness of presence. Similar to presence, some studies argue that flow is significantly related to online game behavior since in online games, continuous scoring, promotion, immediate feedback, and achievement of self satisfaction have become the channels for upgrading individual self-esteem of the Internet users. Therefore, the experience described by flow state such as clear objective and immediate feedback, challenge encounter and adequate skill, combination, sense of control, curiosity, loss of self consciousness, purposeful experience, and inner interests are the states which can be experienced and accomplished by online games. Indeed, in study of Taiwan college students’ Internet behavior, Hwang [20] found that the college students actually experience flow state when using the Internet. Also Choi and Kim [2] found that people continue to play online games if they have optimal experience and flow state had an impact on user’s loyalty. In addition, motivations for online gaming influence users’ flow state and online game addiction. Kim and Park [21] identified seven needs for online game in Korean generation: escaping, leisure, community, character/compensation, satisfaction, entertainment, and pass time. Among these factors, they also found that the need for entertainment has an effect on immersion, while the need for community has a significant impact on addictive behavior in online game. In sum, evidence from flow studies has shown that enjoyment is of key value to generate optimal flow. Though the state of flow is temporal and highly subjective, it is suspected that people who enjoy flow experience during an activity may develop a tendency to repeat the activity and repetition of a particular activity may eventually develop into a tendency toward addiction. Therefore, flow experiences may play a key role in activating addiction through repetition of online game activities.
3 Research Questions Based on the claims and findings examined above, this study explores whether presence and flow are related in the context of online game especially focusing on the
382
S. Park and H.S. Hwang
relationships with entertainment and moreover, explores the impact of the experience of presence and flow on online game entertainment and addiction. Therefore, the following research questions are proposed. RQ1. How the experience of flow and a sense of presence are related online game entertainment? RQ2. How the experience of flow and presence predict online game addiction?
4 Method 4.1 Participants The Participants for this study were Korean college students. The total number of the sample was 133, with 78 males and 55 females. The average participant was 22 years old and the distribution of academic level was freshman (43.3%, N=57), sophomore (24.1%, N=31), junior (21.1%, N=27), and senior (8.3%, N=11). 4.2 Measures The survey questionnaires were distributed to participants and measure includes presence, flow, entertainment, and addictive behavior. Presence. Presence was measured with presence scale by selection from previous studies [5,6,8,22]. The scale represents three dimensions: arrival (the feeling of being present in a mediated environment, a sense of ‘being there’ in the virtual environment), departure (the feeling of no longer being present in the physical environment, my body was in this room, but my mind was inside the world created by online game), and perceived realism (extent to which the virtual environment became the dominant reality, extent to which the virtual world seems to be real). Six items were used and responses were scored on 5-point scales ranging from strongly disagree to strongly agree. The presence index was reliable (Cronbach’s alpha=.89). Flow. Participants’ flow state while playing online game was measured by the scale developed by Choi et al.[23]. The measurement includes 5 items such as attention of focus, sense of potential control, loss of self-consciousness, sense of ecstasy, and time distortion. Participants received that definition of flow with a short description at the beginning of questionnaire. Responses were scored on 5-point scales ranging from strongly disagree to strongly agree. The flow index was reliable (Cronbach’s alpha=.90). Online Game Addiction. Online game addiction was measured with the scale developed by Kim and Park [21]. This measurement includes 5 items to test addictive online game behavior for Korean people [21]. The items include ‘how often do you fear that life without the online game would be boring, empty, and joyless?, ‘how often do you try to cut down the amount of time you spend online game and fail?, ‘how often do you find yourself anticipating when you will go on-line again?, ‘how often do your grades or school work suffer because of the amount of time you spend online game?’, ‘how often do you lose sleep due to late-night gaming?’ Responses were scored on 5-point scales ranging from strongly disagree to strongly agree. The index was reliable (Cronbach’s alpha=.88).
Understanding Online Game Addiction: Connection between Presence and Flow
383
Entertainment. Online game entertainment was measured with two items. These are ‘how much you enjoy the game?’, ‘To what extent the online game entertains you? Responses were scored on 5-point scales ranging from not at all to very much. 4.3 Analysis Person correlations were computed for the relationships among presence, flow, and entertainment. Then regression was conducted to see the predictors of online game addiction. To investigate further how these two concepts--presence and flow--predict online game addiction, a path analysis was conducted using Amos.
5 Results First, to examine the relationships among presence, flow, and entertainment Pearson correlations were conducted. The findings suggest that presence(r=.18, p<.05) and flow (r=.13, p<.05). were significantly correlated with online game entertainment. However the findings indicated that presence has a more positive relationship with online game entertainment than flow does. Korean college student in the sample reported that the more they experience of being there in the online game environment, they feel more that the online game entertained and made them enjoyable. Next, to investigate the predictive power of presence and flow in online game addiction, multiple regressions were conducted. Due to multicollinearity, stepwise regression was used to identify that best predictor among the independent variables. The result indicates that both presence (β=.23, p<.01) and flow (β=.19, p<.01) had an impact on online game addiction (see table 1). This means that the more online game users feel sense of presence and experience of optimal state, the more they are addicted to online gaming. To investigate further how the concept of presence and flow predict online game addiction, a path analysis was conducted using Amos. The result shows that flow mediates the relationship between presence end online game addiction (see figure 1). The model fit that data. (df=1, p=.11, GFI=.98, RMSEA=.07). This result indicates that feeling of presence facilitates the occurrence of flow and flow in turn seems to enhance online game addiction. When analyzing the data in another way, the result shows that flow does mediate the relationship between presence and online game addiction. Table 1. Stepwise Regression: Predictors of Online game addiction Model
Fig. 1. Path model off relationships among presence and flow and online game addiction
6 Discussion The present study aims to examine whether presence and flow are related in the context of online game and moreover, explores the impact of the experience of presence and flow on online game entertainment and addiction. Within our investigation, presence and flow are correlated in relation to online game addiction, and flow mediates the connection between presence and online game addiction. Thus, the experience of being there in which immersed in a virtual reality facilitates the occurrence of mental state operation in which a person is highly involved in playing gaming. What following is a discussion of the implications of the present study with respect to online game activity in virtual environments. First, this study provides empirical evidence supporting the anticipated powerful effect of experience of flow in online games. More importantly, it was found that online game players experience high level of flow state and their addiction to online games can be explained by the flow experience. This finding seems to be consistent with previous research [9] and thus, the study confirms that flow might play an important role of addiction to online games. Second, the present study shows that there are significant mediating effects of presence between flow and online game addiction. Having experience of sense of presence applies significantly to the experience of flow and addictive behavior to online game activity. Theoretically, this result is in line with previous literature about the mediating effects model of presence in human-computer interaction especially, with respect to entertainment games. These findings, however, call for further investigation to elaborate the mediating role of presence in the context of online games. As a final remark, we would like to provide some suggestions for future studies. A limitation of the present study is that this was conducted with Korean college students as the convenience sample, thus future studies need to examine the effects of presence and flow on online game addiction with more diverse topics in other countries in order to validate and corroborate the present findings. Another interesting extension of the present study would be to examine individual difference in terms of feeling of presence and flow in relation to online game addiction. Furthermore, since this study merely employed the survey research, it might not reveal the authentic Internet situation. In the future, researchers could conduct field studies or field experiments to examine the finding obtained in this research.
Understanding Online Game Addiction: Connection between Presence and Flow
385
References 1. Korea Game Industry Agency: 2007 Korean games white book. Korea Game Industry Agent: South Korea (2007) 2. Choi, D., Kim, J.: Why people continue to play online games: In search of critical design factors to increase customers loyalty to online contents. Cyberpsychology & Behavior 7, 11–24 (2004) 3. Fontaine, G.: The experience of a sense of presence in intercultural and international encounters. Presence: Teleoperators and Virtual Environment 1(4) (1993) 4. Biocca, F., Kim, T., Levy, M.R.: The vision of virtual reality. LEA’s communication series, p. 401. Lawrence Erlbaum Associates Inc., Hillsdale (1995) 5. Lombard, M., Ditton, T.B.: At the Heart of it all: The conpcet of presence. Journal of Computer-Mediated Communication 3(2) (1997), http://www.ascusc.org.jcmc/vo13/issue2/ 6. Lombard, M., Reich, R.D., Grabe, M.E., Bracken, C., Ditton, T.B.: Presence and Television: The role of screen size. Human Communication Research 26(1) (2000) 7. Slater, M., Usoh, M., Steed, A.: Depth of presence in virtual environments. Presence: Teleoperators and Virtual Environment 3, 130–144 (1994) 8. Tamborini, R., Eastin, M., Skalski, P., Lachlan, K., Fediuk, T., Brady, R.: Violent virtual video games. Journal of Broadcasting and Electronic Media 48(3), 335–357 (2004) 9. Bracken, C., Skalski, P.: Presence and video games: The impact of image quality and skill level. In: The 8th Annual International Workshop on Presence, Clevland, Ohio, USA, August 24-26 (2006) 10. Friedman, T.: Making sense of software: Computer games and interactive texuality. In: Jones, S.G. (ed.) Cybersociety: Computer-mediated communication and community, pp. 73–89. Sage, London (1995) 11. Lee, K.M., Park, H.: Can a robot be perceived as a developing creature? Effects of robot’s long term cognitive developments on its social presence and people’s social responses toward it. Human Communication Research 31(4), 538–563 (2005) 12. Lee, K.M., Jin, S., Park, S.: Kang.: Effects of narrative on feelings of presence in computer/video games. In: The Annual Conference of the International Communication Association, New York, USA (May 2005) 13. Lee, K.M., Peng, W., Jin, S., Yan: Can robots manifest personality? An empirical test of personality recognition, social response, and social presence in human-robot interaction. Journal of Communication 56, 722–754 (2006) 14. Csilkszentmihalyi, M.: Beyond Boredom and Anxiety. Jossey-Bass., Sanfrancisco (1975) 15. Csilkszentmihalyi, M.: The psychology of optimal experience. Harper& Row, New York (1990) 16. Trevino, L.K., Webster, J.: Flow in computer-mediated communication: Electronic mail and voice mail evaluation and impacts. Communication Research 19, 539–573 (1992) 17. Hoffman, D.L., Novak, T.P.: Marketing in Hypermedia Computer-Mediated Environments: Conceptual Foundations. Journal of Marketing 60, 50–68 (1996) 18. Ghani, J.A., Deshpande, S.P.: Task characteristics and the experience of optimal flow in human-computer interaction. Journal of Psychology, 381–391 (1994) 19. Heeter, C.: Being There: The Subjective experience of presence. Presence: Teleoperators and Virtual Environments 1(2), 262–271 (1992) 20. Hwang, C.: Internet usage of Taiwan’s college students: The flow theory perspective. Master’s thesis. National Chicago Tung University, Taiwan (2000)
386
S. Park and H.S. Hwang
21. Kim, Y., Park, S.: A Study on the online game use influences in game flow and addiction: Focusing on the uses and gratifications approach. Korean Journal of Journalism and Communication Studies 50(5), 355–377 (2007) 22. Kim, T., Biocca, F.: Telepresence via television: Two dimensions of telepresence may have different connections to memory and persuasion. Journal of Computer Mediated Communication 3(2) (1997) 23. Choi, D., Kim, H., Kim, J.: A cognitive and emotional strategy for computer game design. Journal of MIS Research 10, 165–187 (2000)
The Experience of Presence in 3D Web Environment: An Analysis of Korean Second Life SungBok Park1, Ha Sung Hwang2,∗, and Myungil Choi3 1
Hanyang University, visiting professor, Graduate School of Journalism & Mass Communication 17 Haengdang-dong, Seongdong-gu, Seoul, South Korea [email protected] 2 Dongguk University, assistant professor, Dept. of Communication Studies 26, 3 Phil-dong, Chung-gu, Seoul, South Korea [email protected] 3 Namseoul University, full-time lecturer, Dept. of Advertising & Public Relation 21 Mae ju-ri, Seconghwan-eup, Seobuk-gu, Chonan, South Korea [email protected]
Abstract. Second Life is a 3D virtual web environment that has aspects of visualization and sense of presence, as well as text and audio interaction. The whole research aims to explore the sense of presence experienced by online users in Korean Second Life assessing with what ways online users who engage in Korean Second Life perceive a sense of presence. This research leads us to understanding possible factors creating the sense of presence in Korean Second Life as well as in 3D web environments of the near future. Instead of providing specific response from research subjects who have experienced Second Life, this paper presents the theoretical backgrounds to speculate on Second Life constructs and features for enhancing users’ feeling of presence. Keywords: Presence, 3D Web environment, Second Life, Virtual place, Avatar, Emoticons.
1 Introduction The ways of communication through digital media, especially the Internet, have been changing drastically with 3D virtual technology on the Internet. Second Life is the best known of this 3D online virtual world completely created and owned by its residents. Second Life has several characteristics differentiating it from other existing online environments. Especially, the combination of all conceivable types of real-time interaction and digital embodiment found in Korean cultural variation may create various virtual human interactions unlike anything else online. Thus, this study aims to explore the sense of presence experienced by online users in Korean Second Life assessing with what ways online users who engage in Korean Second Life perceive a ∗
sense of presence. This study leads us to understanding possible factors creating the sense of presence in Korean Second Life as well as in 3D web environments of the near future.
2 Overview of Second Life Bing accessible via the Internet, Second Life is an online virtual world initiated by Linden Lab that launched on June 23, 2003. It creates a highly visual, spatial, and auditory chat environment. It is also an excellent example of the current trend toward graphical, interactive domains on the Internet. Thus it is an online 3D virtual world or multi-user virtual environment (MUVE). It provides an immersive environment within witch users can create digital characters, and interact with people from around the world. Residents can explore, socialize, meet other people, participate in individual and group activities, and create and trade virtual property and services with one another, or travel throughout the world, which residents refer to as the grid which is a 3D virtual technology platform. As of April, 2008, Second Life has a total number of over 13 million registered members, geographically dispersed around the country. The most recent estimates put the number of Korean users in Second Life at about 200,000.
3 Presence in SL Environment Advances of web environments in immersive, interactive technology, combined with its increasing availability and quality, have resulted in a practical concern with the manner in which people interact with technologically mediated 3D virtual web environments and online games such as Second Life. Terms of presence or virtual presence are used interchangeably to describe the extent to which people perceive that they are actually present in an artificially created environment. Many attempts have been made to define such an experience - the feeling of being in an environment that is virtually created - and identify its determinants. In the following sub-sections, we will describe the concept of presence and illustrate influential factors affecting users’ presence in a technologically mediated environment, Second Life. 3.1 Overview of Presence First introduced as “Telepresence” by Minsky [1], the term presence is investigated by researchers in a diverse set of disciplines, such as communications, psychology, computer science, engineering, and others. Both terms refer to user’s subjective experience of a medium as a sense of “being there.” This notion of presence is concerned with the subjective feeling of existence within a mediated environment [2,3,4,5]. According to Lombard and Ditton [6], presence is considered as a multidimensional concept. They identified six different conceptualizations of presence by reviewing a diverse set of literatures. These conceptualizations include, for instance, (1) presence as social richness (e.g., the extent which a medium is perceived as
The Experience of Presence in 3D Web Environment
389
sociable, warm, sensitive, personal), (2) presence as realism (e.g., the extent to which a medium can produce seemingly accurate representations of objects, events, and people), (3) presence as transportation (the feeling of “you are there”, “It is here”, and/or “we are together” , and (4) immersion (the extent of perceptual and psychological immersion in a mediated environment). Lombard and Ditton [6] combined all of these conceptualizations into a single conceptual definition of presence as the perceptual illusion of nonmediation. An illusion of nonmediation occurs when an individual is willing to overlook or fails to perceive the existence of a medium in his/her communication environment and responds as he/her would if the medium were not there [7]. This illusion leads to the subjective experience of being in environment, even when individual physically inhabit another. There has been relatively research conducted to investigate factors that contribute to a sense of presence [e.g., 5,6,7,8]. Studies have suggested characteristics of a medium (e.g., dimensionality, interactivity, image quality, and number of sensory outputs) and characteristics of the medium user (personality type, prior experience with the medium, willingness to suspend disbelief) are important factors that contribute a sense of presence. For instance, Steuer [5] argues that the vividness and interactivity found in a medium help to evoke presence, and that identifying these attributes in different media will allow a user to predict the type of presence he/she will experience. Witmer and Singer [8] identify involvement and immersion as necessary psychological states for the experience of presence, and indicate that they are critical determinants of both the learning and performance outcomes from media use. As such, presence is determined by formal features of a medium and characteristics of the medium user [6]. 3.2 Sense of Virtual Place from Virtual Constructs in Second Life There does not exist a specific territoriality in cyberspace, rather there exists only perceived spatiality or territoriality that can be technologically mapped in computer networks enabling a person to perceive that everything exists and is located as in offline space. Rather than examining the Internet as a place for locality, scholars on the specific technical and communications features of the Internet often like to stress its ‘placeless’ nature [9]. Due to the conceptual characteristics of cyberspace placelessness, web environments seem to have no territoriality. Community studies show that a sense of place plays a crucial role in forming and sustaining community attachments and community life [10]. Some scholars also identify an evident sense of place in online communities [11,12,13]. Thus, attachment to specific virtual place in Second Life may, then, be considered to be an important factor in forming and sustaining online users’ feeling of presence. Second Life environment, as the best example to illustrate the implications of territoriality and spatiality of cyberspace, give tacit evidence to this assumption. Only a few scholars suggest that the geographical elements of physical environments play a key role in virtual environments [13]. Such may be because cyberspace is a computer-generated conceptual space with no necessary geographical analogue. However, it seems that Second Life designers understand the purposes and effects of virtual landscapes. They make systematic efforts to increase the number of choices available to Second Life members or participants.
390
S. Park, H.S. Hwang, and M. Choi
Fig. 1. Dokdo Island and Korean traditional ancient ships
Currently, text-based systems are giving way to virtual worlds equipped with more advanced 3D graphic interfaces. In Second Life, the metaphor of place and space has a highly visible presence, and produces hyper-real simulations that imitate physical place and space. Such a shift in virtual environments from text-based systems, such as MUDs or MOOs, to advanced 3D virtual systems, such as Second Life and various virtual worlds suggests that virtually constructed geographical landscapes may be essential for virtual environments. In addition, providing fascinating new social spaces that exist only in cyberspace, virtual environment systems can make valuable contributions to the understanding of the geographical structure of virtual space [14]. For instance, Second Life has a number of theme categories in which there are different advanced 3D graphical virtual landscapes and constructs for participants. Second Life is one of a number of commercially developed virtual reality (VR) systems that are publicly available on the web environments. Using avatars, Second Life provides a sense of geographic space, with freedom to move in different directions, and feelings of physicality [14]. In Second Life, users are able to own land and build homesteads, thereby constructing their own places for online social interaction. The world of Second Life provides the most persuasive example of how a virtually constructed place (as place is one of the essential aspects of a physical environment) plays an important role in web environments. In addition, virtual games and virtual environment systems remind us that although landscapes are virtually constructed, they are patterned on geographical landscapes in the physical world. In more detail, all virtual constructs including weapons, artifacts, landscapes, and even most human actions in Second Life are patterned after those in the physical world. This indicates that the virtual environment itself is conceptually constructed on the basis of the physical environment which may provide enhanced feeling of presence.
The Experience of Presence in 3D Web Environment
391
Fig. 2. The scenery around the South of Gate in Seoul
Second Life comprises recognizable elements like buildings with floors and walls, towns with homes and streets, or landscapes with mountains, boulders, meadows and blue sky backgrounds. Many scenes recreate offline life public meeting places like bars, art galleries, parks and plazas. Second Life can create fantastical creatures of all kinds. Thus, we emphasize the importance of such spatial constructions in the continuity of a feeling of presence in Second Life, such as mediated proximity and belonging which may give us a sense of presence when we are in Second Life. 3.3 Digital Embodiment of Virtual Human: Avatar In virtual environments, users are generally represented by virtual, or digital, embodiments, commonly referred to as avatars. As a virtual representation of a human being, avatars are graphical icons used to represent online participants, usually in chat rooms with visual interfaces. The use of avatars in cyberspace provides a new mode for rich personal interaction and communication, for the avatars provide the ability to communicate nonverbally. Many sophisticated 3-D avatars in Second Life are animated and change shape according to what the user is ‘doing’, such as walking or sitting. Avatars’ appearance includes the way a person dresses, cuts his or her hair or decorates his or her body. In Second Life one can choose his or her online representation, although this choice may depend on the software used and the world visited. The uses of avatars can facilitates feelings of “being there” or “being together” and re-introduce spatial elements and nonverbal cues usually deficient in traditional CMC environments [15], so that people tend to regard them as realistic representations in cyberspace. In their early get-acquainted period, online community members are likely to engage in making their online identity solid and positive in cyberspace through various, such as the
392
S. Park, H.S. Hwang, and M. Choi
Fig. 3. Example avatar of a Korean user
manipulation and use of textual graphics, the application of other technologies that capture image from audio and visual to the Internet interface, and most importantly, through digital embodiments like avatars. An avatar is as a virtual representation of human being in cyberspace representing as an online participant in a virtual world. An avatar functions as a virtual body. A visitor uses the avatar to get around the cyber world. When people meet virtually, their presence is partially indicated through their avatars’ presence. Visitors interact through the body movements and gestures of the avatars. In a graphical virtual environment, the avatar looks like a human, though not necessarily the human it represents. An observer responds to the other avatar similar to the way they would behave towards another person’s physical presence. Visitors move their avatars together when they are speaking with one another and separate them when the conversation ends. They change the avatar’s facial expression to indicate emotion. They manipulate the avatar’s limbs in order to make gestures. Vilhjalmsson [16], in an analysis of the conversation transcripts, shows significant improvement in the overall conversational process and significantly fewer messages spent on channel maintenance in the avatar groups. The avatars also significantly improved the users’ perception of each others’ efforts. Green [17] articulates that being digital of body in cyberspace is one means of embodiment formalized by practices of representation systems. Taylor [18] also admires avatars as digital embodiments of ourselves that can provide the aspects of presence, social integration, and communication. As digital bodies, avatars disclose something about a person in cyberspace. They are public signals of who online participants are. They also shape and realize how users internally experience their selves in cyberspace. Consequently, the digital embodiment and avatar of virtual humans is to give the user as sense of presence and as the representation of one self and other self in cyberspace. As elements in Second Life, avatars—including their appearance, behaviors and controls—condition the expectations and behaviors of Second Life participants or visitors. Based on extensive databases, geographical information systems,
The Experience of Presence in 3D Web Environment
393
multi-media and virtual reality modeling, in 3D graphical virtual worlds users can “step into and move around” within these cities, contribute to discussions on different kinds of forums, and run daily errands [19, p.154]. Such virtual avatar worlds are the expression of the desire of humans to create in the digital environment a world as rich as the offline reality in which we live. Virtual human, computer-generated entity that looks and acts like people and engage in conversation in virtual environments have become prominent in the study of communication. The idea of virtual human envisions future computer systems that are social actors rather than tools. Techniques from artificial intelligence, computer animation, and human–computer interaction are increasingly converging in the field of embodied conversational virtual agents. Such agents are envisioned to have similar properties to humans in face-to-face communication, including the ability to generate simultaneous verbal and non-verbal behaviors. 3.4 Emotional Indicators: Emoticons As people continue to interact and maintain relationships in cyberspace, they gradually move toward deeper areas of their mutual interpersonal relationships through verbal indicators, such as the emoting function and emoticons, in order to compensate for the limits of social context and the lack of nonverbal cues which may enhance a feeling of presence in virtual environments. In addition, CMC media are shifting away from primarily text-based interaction to more audio and visual-based interaction. In Web environment and in the new multimedia possibilities of the Internet, text, sound, video and image produce a wider range of digitized symbolic cues and these compensate for emotions and presence assumed to be lacking in online interaction. There are ways to make up for the lack of physical presence in the CMC environment. Using emotional indicators, such as emoting, emoticon, and avatar, can create a sense of presence in which we can identify with the other person. For instance, emoticons (emotional icons) are used to compensate for the inability to convey voice inflections, facial expressions, and bodily gestures in written communication. Emulating a facial expression and expressing a feeling that supplements the message, emoticons are used most in meet and greet situations where one wants to appear approachable. Some researchers illustrate how emoticons are useful for making up for the lack of nonverbal context cues in virtual environments [20,21,22,23]. Emoticons may indicate the degree of intimacy in a personal relationship. Utz [24] argues that the more emoticons a person use, the more friendships that person builds. Baym [20] also states that community members in cyberspace create expression through emoticons and abbreviations. The combination of multiple features of CMC technology and using emotional indicators in online interaction can be a powerful element for enhancing sense of presence in Second Life.
4 Overview of Research Design This research used a qualitative analysis of the user experience in Second Life in 4 undergraduate classes. As one of course assignments, total number of students who participant in this research for a week was 214. The process to evaluate the student
394
S. Park, H.S. Hwang, and M. Choi
experience in Second Life is inductive method using grounded theory and cultural theory as a foundation. Every student was asked a series of open-ended questions designed to gather their views about and experience with Second Life activities. The central question posed to the students for this paper was “in what ways and to what extent do you feel a sense of presence to Second Life?” The generated data were categorized and refined into common themes.
5 Conclusion Second Life is social spaces formed and maintained by CMC technologies, human interaction, and the individual’s interpretations. Today’s advanced technology provides a user with a unique experience that one has left real world and is now “present” in the virtual environment. This notion of presence in the virtual world has been central to both VR researchers and designers since it is introduced [1]. There have been many attempts to examine what presence is, how it occurs, and what cases and effect are. Yet, the benefits of presence, either to enhance interactive VR design or human performance, have not been clearly discovered. Given that, this paper suggests several interposed factors that may facilitate the perceived presence of users interacting with technologically mediated environments such as Second Life. Because our web environment is still undergoing constant development, future web environment is continuing in the direction of more complicate and enhanced GUI providing an enhanced sense of presence.
References 1. Minsky, M.: Telepresence. Omni 3, 45–51 (1980) 2. Heeter, C.: Being there: The subjective experience of presence. Presence: Teleoperators and Virtual Environments 1(2), 262–271 (1992) 3. Kim, Biocca: Telepresence via Television: Two dimensions of Telepresence May Have Different Connections to Memory and Persuasion. Journal of Computer-Mediated Communication 3(2) (1997), http://www.ascusc.org.jcmc/vo3/issue2/ 4. Sheridan, T.B.: Musings on telepresence and virtual presence. Presence: Teleoperators and Virtual Environments 1(1), 120–126 (1992) 5. Steuer, J.: Defining virtual reality: Dimensions determining telepresence. Journal of Communication 42(4), 73–93 (1992) 6. Lombard, M., Ditton, T.B.: At the Heart of it all: The conpcet of presence. Journal of Computer-mediated communication 3(2) (1997), http://www.ascusc.org.jcmc/vo13/issue2/ 7. Lombard, M., Reich, R.D., Grabe, M.E., Bracken, C., Ditton, T.B.: Presence and Television: The role of screen size. Human Communication Research 26(1) (2000) 8. Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments 7, 225–240 (1998) 9. Rheingold, H.: The virtual community: Homesteading on the electronic frontier, revised eds. The MIT Press, Cambridge (2000) 10. McMillan, D.W., Chavis, D.M.: Sense of community: A definition and theory. Journal of Community Psychology 14(1), 6–23 (1986)
The Experience of Presence in 3D Web Environment
395
11. Baym, K.N.: The emergence of on-line community. In: Jones, S.G. (ed.) Cybersociety 2.0: Revisiting Computer-Mediated Communication and Community, pp. 35–68. SAGE, Thousand Oaks (1998) 12. Baym, K.N.: Tune in, Log on: Soaps, Fandom, and Online community. SAGE Publications, Thousand Oaks (2000) 13. Gotved, S.: Spatial dimensions in online communities. Space & Culture 5(4), 405–414 (2002) 14. Dodge, M.: Explorations in Alpha World: The geography of 3-D virtual worlds on the Internet. In: The RGS-IBG Annual Conference for Virtual Reality in GeographyWorkshop and Special Session, Leicester, England, January 4-7 (1998) 15. Mantovani, G.: The Psychological construction of the Internet: From information foraging to social gathering to cultural mediation. CyberPsychology & Behavior 4(1), 47–56 (2001) 16. Vilhjalmsson, H.H.: Avatar augmented online conversation. Doctoral dissertation. Program in Media Arts and Sciences at the Massachusetts Institutes of Technology (2003) 17. Green, N.: Beyond Being Digital: Representation and virtual corporeality. In: Holmes, D. (ed.) Virtual politics: Identity & Community in Cyberspace, pp. 59–78. SAGE, London (1997) 18. Taylor, T.L.: Living Digitally: Embodiment in Virtual Worlds. In: Schroeder, R. (ed.) The social life of avatars: Presence and interaction in shared virtual environments, Springer, London (2002), http://www.itu.dk/~tltaylor/papers/Taylor-Living Digitally.pdf (Retrieved on January 20, 2009) 19. Ridell, S.: The web as a space for local agency. Communications 27, 147–169 (2002) 20. December, J.: Units of analysis for Internet communication. Journal of Communication 46, 14–38 (1996) 21. Talamo, A., Ligorio, M.: Identity in the cyberspace: The social construction of identity through on-line virtual interaction. In: 1st Dialogical Self conference, Nijmegen, June 2326 (2000) 22. Wolf, A.: Emotional expression online: Gender differnces in emoticon use. CyberPsychology & Behavior 3, 827–833 (2000) 23. Utz, S.: Social information processing in MUDs: The development of friendships in virtual worlds. Journal of Online Behavior 1 (2000), http://www.behaviro.net/JOB/v1n1/utz.html
Influence of Real-World Ten-Pin Bowling Experience on Performance during First-Time Nintendo Wii Bowling Practice Kirsten A. Peters User Centric, Inc. 2 Trans Am Plaza Dr, Ste 100 Oakbrook Terrace, IL 60181 USA [email protected]
Abstract. In order to understand if proficiency in a real-world activity influences performance or movement characteristics in a simulation of that activity, six expert real-world ten-pin bowlers, ten novice real-world ten-pin bowlers and eight expert Wii bowlers completed 3 games of Nintendo Wii Sports’ bowling game. Two values were recorded for each throw: the (score) and the total range of motion (ROM) for the participant’s throwing arm (using two-dimensional motion capture). Averages across the first five trials were compared to averages across the last five trials. From the first five to last five trials, there were significant increases in both the mean score and mean ROM values, when collapsing data across experience level. While there was a significant main effect of experience level on the overall ROM values, differences between each experience group’s ROM values were not detected. A larger sample size is necessary to confirm if real-world ten-pin bowling experience influences score and ROM during first-time Wii bowling. Keywords: Motor program, motor learning, motion capture, gesture-based interaction, video games, transfer of skill, real-world vs. simulation.
Influence of Real-World Ten-Pin Bowling Experience
397
activity would impact performance or the arm movements used when simulating that same activity using the Nintendo Wii. Nintendo's Wii Sports bowling game was chosen as the primary stimulus because it is closely modeled after the rules of the real-world game of ten-pin bowling and the outcome is wholly dependent on user input. Novice and expert ten-pin bowlers participated in the current study, as well as a control group of expert Wii bowlers. Performance was measured using the score of the first throw. The total range of motion (ROM) of the dominant bowling arm during the first throw was measured using a two-dimensional motion capture system. The primary goal of this exploratory study was to clarify the relationship between experience with a real-world activity, performance simulating the real-world activity, and arm movements used to simulate the real-world activity. Specifically, the study examined if either group’s score or ROM values changed during a first time Wii bowling practice period.
2 Background 2.1 Ten-Pin Bowling In the modern American sport of ten-pin bowling, players score points by rolling a weighted ball down a lane and knocking over pins. While real-world bowling technique varies from athlete to athlete, most players use a similar arm swing movement and a four- or five-step approach technique. In one throw, a bowler receives kinesthetic, visual, and auditory feedback. Based on the outcome of a delivery, the bowler can choose to use the same movement or modify the movement in the next frame [2]. In ten-pin bowling, repetition is key. The more precisely a bowler can duplicate the same series of motions each time they throw the ball, the higher his or her score will be. It can take years to master proper bowling technique with consistent precision and accuracy. While the amount of time and experience needed to become an advanced bowler varies, technique is likely to be fairly consistent and well established once a bowler reaches this level. 2.2 Nintendo’s Wii Sports Bowling Game The Wii Sports video game, which is a part of every Wii console, includes a bowling game. The game (see Fig. 1) uses the same rules and scoring conventions as realworld ten-pin bowling and the virtual environment is modeled after the environment and playing area seen at a real bowling alley. In Wii Sports bowling, players hold the remote and move their arm like they would in the real-world game of ten-pin bowling. When a player is ready to roll the ball, he or she holds down the trigger button located near their index finger and then raises the remote to their chest. While holding down the trigger button, the player swings as if he or she was rolling a real bowling ball [3]. When the player is ready to release the ball, he or she releases the trigger button at the bottom of their swing.
398
K.A. Peters
Fig. 1. The Nintendo Wii Remote video game controller (left) and a screen view of Nintendo’s Wii Sports bowling game (right)
2.3 Motor Schema Theory and Transfer of Skill Over time, a real-world bowler learns the series of body motions necessary to produce high scores. Schmidt’s motor schema theory attempts to explain how motor skills, such as those used to develop proper bowling technique, are generally learned [4]. Learning motor skills involves acquiring schemata which define the “relationships of the information involved in the production and evaluation of motor responses" [5]. These rules are learned through experience and used by the motor programs to produce new actions [6]. Schema theory describes an overall mental construct, a “generalized motor program”, which uses the schemata to determine specific actions. For example, over time, a bowler generates a ‘bowling motor program’. Motor learning is the process of improving motor skills where one utilizes a “set of processes that underlie the changes in a capability for movement” [5]. During the process of motor learning, a person’s motor movements will change as they develop the internal capability for the movement. While motor learning can be neither directly observed nor defined as a change in behavior, the processes taken together will lead to some particular product, state or change. One can infer the presence of motor learning based on the observed changes in motor behavior over time. When a bowler’s approach starts producing consistent results, they have learned what movements are necessary for their expert bowling motor program. If the tasks of throwing a ball in real-world bowling and Wii bowling are similar enough, skill may be easily transferred. The amount of transfer of skill between two similar tasks depends on the degree of similarity of the two tasks. As the tasks become more dissimilar, the amount of transfer, as measured by performance, can be expected to decrease [7]. It has yet to be determined whether a purely quantitative relationship exists between the degree of task similarity and subsequent change in the amount of transfer. According to the motor schema theory, if a task is modified in some way and this similar-yet-modified task is practiced routinely for a non-trivial length of time, it is possible that a separate motor program will eventually be created. When tasks are practiced in simulations prior to being actually performed, a positive transfer of skill has been consistently observed in previous studies [8, 9]. Unfortunately, little research has been done on human performance when skills are transferred from a real task environment to a virtual environment. However, there is some evidence that this transfer of skill can be positive in certain applications [8, 10].
Influence of Real-World Ten-Pin Bowling Experience
399
3 Experimental Goal and Hypotheses When a user is first exposed to a system, they choose a technique based on prior knowledge and motor schemas associated with the task requirements. Therefore, when players are told that a video game is a bowling simulation, they may assume the two activities to be similar and start the video game with a similar movement technique to one they have used for real bowling. While proper real-world bowling technique has been documented, proper (i.e., most effective) Wii bowling technique has yet to be determined. The nature of the Wii bowling simulation is such that it does not give users adequate kinesthetic feedback from the heavy ball. Players may start out using their general bowling motor program, but with practice, may modify their schema and arm movement behavior for Wii bowling. With experience, Wii bowlers may develop a modified arm swing movement that does not require the transfer of momentum to the ball from the full arm swing motion, but still achieves the goal of knocking down pins. Though it may be largely based on the original generalized motor program, another modified program may develop over time. Given the aforementioned topics, questions begin to arise. When expert real-world bowlers play Nintendo’s Wii Sports bowling game for the first time, does their proficiency at the real task help or hurt their performance at the simulated task compared to novice bowlers? Will either groups’ movement characteristics or performance abilities change during their first bowling session and, if so, how? How do each group’s movement characteristics and performance abilities compare to expert players of Nintendo’s Wii Sports bowling video game? The present study was developed in an attempt to examine these questions by observing the movement characteristics and performance abilities of novice and expert real-world bowlers over one experimental period of Wii bowling. Because they both had limited experience with the Wii bowling game, the expert real-world bowlers and novice real-world bowlers were expected to start the study with similar ROM and scores. However, because they would not be limited by the real-world bowling motor program, the novice real-world bowlers were expected to have a greater decrease in ROM and greater increase in score compared to the expert realworld bowlers. Therefore, the novice bowlers would end the experimental period with a smaller ROM and higher score than the expert real-world bowlers. As the control group, the expert Wii bowlers were expected to show no change in ROM or score over the experimental period. Their ROM was expected to be the smallest and their scores were expected to be the highest of the three groups.
4 Methodology 4.1 Participants A sampling method of convenience was used to recruit three groups of participants: Expert real-world bowlers, novice real-world bowlers, and expert Wii bowlers. Table 1 shows a summary of the participant characteristics.
400
K.A. Peters Table 1. Summary of group characteristics
Group
Real-world bowling experience Mean score Frequency of per game playing
Wii bowling experience Mean score Frequency of per game playing
> 160
≥ 1x/week
< 100
< 2x/month
< 100
< 2x/month
< 100
< 2x/month
NA
NA
>160
≥ 3x/month
The group of expert Wii bowlers was used as a control group. The data collected from this group was treated as a baseline comparison for the data collected from the other two groups. The real-world ten-pin bowling experience from this control group was not taken into consideration, as their Wii bowling technique (i.e., a unique schema) has already been established regardless of their ten-pin bowling experience. 4.2 Procedure The experimental tasks were performed on the Nintendo Wii. Participants used the Wii Remote to interface with the video game system. Participants were given a 6 ½ foot by 3 foot playing area within which to move. This allowed them to play the game with free and natural movement while allowing us to record their movements. Two reflective markers for the motion capture system were attached to Velcro strips and placed around participants’ right humerus. The approximate length of the participant’s humerus was measured and the two strips were placed at the first and third quartiles of the upper arm (see Fig. 2). Participants were then given a short orientation to become familiar with the Nintendo Wii interface and control of the Wii Remote within the Wii bowling game. Regardless of group assignment, all participants performed the same experimental task. They were asked to play the Wii bowling game with the goal of achieving the highest score. Participants completed three games of ten frames each, for a total of 30 trials. A trial was defined as a full frame which may have included a second throw, as per standard ten-pin bowling rules. Participants were required to stand during the trials and allowed to walk and lean forward as long as they stayed within the designated area needed for the motion capture system to function properly. 4.3 Data Collection Two values were recorded for each throw: the number of pins knocked over (score) and the total range of motion (ROM) for the participant’s arm. The arm movement was recorded with a motion capture video system and later translated into a ROM value represented as an angle. Motion capture is the process of digitally recording body movements and translating them into a two- or three-dimensional model [11]. A 2D motion-capture video system was used in this study. The system consisted of a digital video camera,
Influence of Real-World Ten-Pin Bowling Experience
401
passive reflective markers placed on a participant’s upper right arm, a light source pointed at the participant (to increase the amount of light reflected by the markers), and a computer running the digital video capture and Peak Motus software package [12]. The camera faced participants laterally and recorded the projection of each reflective marker at 60 frames per second. Accounting for the position and orientation of the camera, the software used the digitized video to transform these projections into a 2D position of each marker. The motion capture system was configured to allow for the capture of the entire arm swing motion of each throw. 4.4 Data Analysis In a game of real-world bowling, the goal of the first throw in a frame is always the same: knock over as many pins as possible. The conditions during the second throw of a frame are highly variable and entirely dependent on the result of the first throw. Therefore, our primary analysis focused only on the data collected during that first throw. However, participants were asked to complete both throws of every frame. A statistical analysis of the score data was conducted for all three participant groups. Since the change in performance over the experimental period was the primary variable of interest, the scores of the first throws from the first five and last five trials were included in the analysis. In order to calculate the ROM joint angles, the digital video data needed to be digitized by the motion capture software [12]. In this process, for each first throw, the software located the position of the centroid of each reflective marker from each frame of the video such that the participants’ arm motion was converted to a series of x-y coordinates. Maximum amount of shoulder flexion and hyperextension were deduced and combined to create the total ROM for each swing. Similar to the performance score analysis, only ROM data from the first throws of the first five and last five trials were analyzed for each participant group.
5 Results 5.1 Wii Bowling Score A 2 x 3 (trial period by experience level) split plot Analysis of Variance (ANOVA) was conducted on the score data where trial period (first five trials and last five trials) was the within-subjects variable and experience level (expert real-world bowlers, novice real-world bowlers and expert Wii bowlers) was the between-subjects variable. The purpose of this analysis was to determine if experience level influenced how much a participant’s score increased from the first five trials to the last five trials. A paired t-test was also conducted on the mean scores of the first and last five trials for each group. Figure 2 shows the average score for each group during the first five and last five trials. The main effect of trial period was found to be significant (F1, 21 = 4.89, p < .05). Overall, the scores from the last five trials (M = 8.23) were significantly higher than the scores from the first five trials (M = 7.83). There was no main effect found for experience level. This means that overall no one group scored better than another
402
K.A. Peters
Average Scores 9.0 8.65 8.5
Score
8.28
8.20
8.0 7.92 7.72 7.5
Expert bow lers
7.43
Novice bow lers Expert Wii-bow lers
7.0 First 5 Trials
Trial period
Last 5 Trials
Fig. 2. Average scores of each group for the first and last five trials of the session
group. Furthermore, we found no interaction between the trial period and experience level. From the first five to last five trials, no one group’s scores increased significantly more than the other groups’ scores. 5.2 Range of Motion A 2 x 3 (trial period by experience level) split plot ANOVA was conducted on the ROM data where trial period (first five trials and last five trials) was the withinsubjects variable and experience level (expert real-world bowlers, novice real-world bowlers and expert Wii bowlers) was the between-subjects variable. Similar to the score data, the purpose of this analysis was to determine if experience level influenced how much a participant’s ROM changed between the first five trials and the last five trials. Figure 3 shows the mean ROM for each group during the first five and last five trials. The ROM results show a significant main effect of trial period (F1, 21 = 17.43, p < .001). Overall, participants’ ROM increased significantly from the first five trials (M = 157.76°) to the last five trials (M = 180.19°). Post-hoc t-tests showed a significant increase in ROM for the expert real-world bowlers, (t5 = 3.094, p < .05) and novice bowlers (t9 = 4.19, p < .005) but not for the expert Wii bowlers.
Average ROM Values
225
209.95
ROM (degrees)
205 190.84 185
176.40 165
162.61 154.83 Expert bow lers
145 140.26
Novice bow lers Expert Wii-bow lers
125 First 5 Trials
Trial period
Last 5 Trials
Fig. 3. Average ROM values of each group for the first and last five trials of the session
Influence of Real-World Ten-Pin Bowling Experience
403
There was also a significant main effect of experience level (F2, 21 = 3.89, p < .05). Post hoc comparisons showed no significant differences between the individual experience levels, although there was a non-significant trend for expert real-world bowlers to have a larger ROM than novices (p = .055) and expert Wii bowlers (p = .074). The results also revealed no interaction between the trial period and experience level. This means that the three groups did not differ in the amount of ROM increase. 5.3 Correlation The Pearson product-moment correlation coefficient was calculated between the score and ROM variables to determine if there was a relationship between the increase in ROM values and the increase in score. This analysis did not reveal a significant correlation between the mean ROM and mean scores. There were also no significant correlations between the mean ROM and mean scores for each individual group.
6 Discussion The purpose of this study was to determine if proficiency in ten-pin bowling influenced performance abilities or movement characteristics when playing simulated Wii bowling. In general, the data did not support the original hypotheses that novice real-world bowlers would have a greater decrease in ROM and greater increase in score than the expert real-world bowlers. In several cases, test statistics approached significance, suggesting that the lack of support for some hypotheses may be at least partially due to the small number of observations analyzed in this study. 6.1 Wii Bowling Score While the overall Wii bowling scores increased significantly from the first five to the last five trials, there was no significant main effect for experience level nor was there a significant interaction between experience level and trial period. These findings indicate that novice real-world bowlers did not have a greater increase in score than expert real-world bowlers. An observed range effect may have prevented the detection of a relationship between experience level, trial period and Wii-bowling scores. While ten-pin bowling scores can range from 0-10 pins per trial, participants’ average scores only ranged from 7.43 to 8.65. 6.2 Range of Motion The significant increase in ROM during the session is contrary to the original hypothesis. The training period for this study only included one frame of Wii bowling. It is possible that participant’s bodies were not fully warmed up when they started their first trial and the overall increase in ROM was related to the loosening of their joints and increased activity of their muscles. The fact that the expert real-world bowlers had the largest overall mean ROM suggests these participants may have applied their established real-world bowling motor program to play Wii bowling, which requires a large arm swing movement to
404
K.A. Peters
propel the weighted ball down the lane. The significant increase in ROM for these participants can then be explained by the warm-up theory described above. The novice bowlers should not have such developed real-world bowling motor programs. However, the expert real-world bowlers maintained a bowling motor program for a reason: they were experts at real-world bowling. The non-significant trend for novice bowlers to have a smaller ROM than the expert real-world bowlers may be a result of novice bowlers not having a real world bowling motor program to define how large of a movement to use during Wii bowling. The significant main effect of experience level on ROM points toward a confirmation of Schmidt’s motor schema theory and the existence of generalized motor programs [4]. With a larger sample size, it may have been possible to determine if experience in real-world bowling influenced the increase in ROM over the experimental period. 6.3 Correlation While the ROM data was collected for this study using a 2D motion capture system, real-world and Wii bowling can involve 3D arm and body movements. For example, many expert real-world bowlers twist their wrist right before releasing the ball to include a spin. No correlation between ROM and score data was discovered. A likely explanation for the lack of correlation between ROM and score data is that data collection was solely focused on the 2D arm swing movement at the shoulder. The influence of any other body part and the overall body motion in all three dimensions was ignored. Additionally, as discussed before, the increase in ROM may not be related directly to the score performance because participants’ bodies may have been warming up throughout the thirty trials. As the joints were loosened and the muscle activity increased, the ROM at the shoulder may have increased because their arm muscles were able to stretch further. This may be another reason why no correlation was found between the overall mean ROM and overall score.
7 Conclusion Based on the overall increase in score, participants playing Wii bowling for the first time were able to improve their performance in just three games. These results indicate that the Wii bowling game was designed to allow for quick improvement, which is often not the case in real-world bowling. This function may be part of a strategy to motivate new users to continue playing Wii bowling and other games on the Nintendo Wii. Motion capture systems are often used by video game designers when creating realistic character movements. However, motion capture and video games do not often come together for academic research purposes. The motion capture system used in this study focused on capturing the amount of flexion and hyperextension of participants’ arm movements at the shoulder. While this 2D movement is a major component of the bowling arm swing motion, we believe that future studies attempting to understand the bowling arm movement should utilize a 3D system and
Influence of Real-World Ten-Pin Bowling Experience
405
help examine the role of the more subtle arm movements and their impact on performance. Further research is also needed to explore the non-significant trends observed during this study. A longer trial period should be used as the thirty trials may not have been enough to observe the true change in ROM and scores associated with learning to play Wii bowling, or the differences between the groups. Additionally, a larger sample size is necessary to determine if real-world ten-pin bowling experience has an influence on performance and movement during an initial Wii bowling practice.
References 1. Nintendo, What is Wii?, http://wii.nintendo.com/whatiswii_index.jsp 2. Bellisimo, L.: The bowler’s manual, 4th edn. Prentice-Hall, Inc., Englewood Cliffs, NJ (1982) 3. Nintendo of America. Wii Sports [Video game software and instruction booklet]. Nintendo of America, Redmond, W.A (2006) 4. Schmidt, R.A.: A schema theory of discrete motor skill learning. Psychological Review 82, 255–260 (1975) 5. Lee, T., Sherwood, D.: Schema Theory: critical review and implications for the role of cognition in a new theory of motor learning. Research Quarterly for Exercise and Sport 74(4), 376–382 (2003) 6. Newell, K.M.: Motor skill acquisition. Annual Review of Psychology 42, 213–237 (1991) 7. Schmidt, R.A., Young, D.E.: Transfer of movement control in motor skill learning. In: Cormier, S.M., Hagman, J.D. (eds.) Transfer of learning: Contemporary Research and applications, pp. 47–79. Academic Press, Inc., London (1987) 8. Kenyon, R.V., Afenya, M.B.: Training in virtual and real environments. Annals of Biomedical Engineering 23(4), 445–455 (1995) 9. Torkington, J., Smith, S.G., Rees, B.I., Darzi, A.: Skill transfer from virtual reality to a real laparoscopic task. Surgical Endoscopy 15(10), 1076–1079 (2001) 10. Bürki-Cohen, J., Boothe, E.M., Soja, N.N., DiSario, R., Go, T., Longridge, T.: Simulator fidelity – the effect of platform motion. In: Proceedings of the International Conference on Flight Simulation – The Next Decade, Royal Aeronautical Society, London, UK (2000) 11. Furniss, M.: Motion Capture. MIT Communications Forum (2004), http://web.mit.edu/comm-forum/papers/furniss.html 12. Peak Motus [Computer software]. Vicon Motion Systems, Oxford, UK
Emotionally Adapted Games – An Example of a First Person Shooter Timo Saari1, Marko Turpeinen2, Kai Kuikkaniemi2, Ilkka Kosunen2, and Niklas Ravaja3 1 Temple University, 1801 N. Broad Street, Philadelphia, PA, USA, and Center for Knowledge and Innovation Research (CKIR), Helsinki School of Economics, Finland and Helsinki Institute for Information Technology (HIIT), Finland [email protected] 2 Helsinki Institute for Information Technology (HIIT), Finland {Marko.turpeinen,kai.kuikkaniemi,ilkka.kosunen}@hiit.fi 3 Center for Knowledge and Innovation Research (CKIR), Helsinki School of Economics, Finland [email protected]
Abstract. This paper discusses a specific customization technology – Psychological Customization - which enables the customization of information presented on a computer-based system in real-time and its application to manipulating emotions when playing computer games. The possibilities of customizing different elements of games to manipulate emotions are presented and a definition of emotionally adaptive games is given. A psychophysiologically adaptive game is discussed as an example of emotionally adapted games. Keywords: Customization, adaptive systems, psychological effects, emotion, games, emotionally adapted games, psychophysiological measurement, Psychological Customization.
1 Introduction Emotions or emotion-related variables (e.g., competitiveness) play a critical role in gaming behavior [1, 2]. People seek, and are eager to pay for, games that elicit positive emotional experiences and enjoyment; however, an enjoyable game may not elicit only positive emotions but possibly also negative ones (e.g., anger, fear). Thus, one of the major goals for video game designers is to elicit optimal emotional responses or response patterns. It is possible to build systems which either automatically or semi-automatically adapt games to create optimal emotional responses and response patterns. Such customization systems entail the manipulation of the game per player to elicit specific emotional responses or to create desired response patterns. We propose a system called Psychological Customization to be used in the adaptation of games for manipulating emotional states of players.
Emotionally Adapted Games – An Example of a First Person Shooter
407
Psychological Customization entails the customization of transient (i.e. short-term) user experiences (i.e. psychological effects) when interacting with media- and communication technologies. Experience-based customization entails the automatic or semi-automatic adaptation of information per user, task and context in an intelligent way with information technology. A subset of Psychological Customization is to vary the form of information (modality for instance) per user profile, task and context, which may systematically manipulate (approach, avoid, modify intensity, frequency and duration, create combinations, create links to behavior) different psychological effects. Psychological effects can be considered transient states, such as emotion, mood, types of cognition, learning, flow, presence, involvement and enjoyment. [e.g. 3, 13] Psychological Customization works on the principle of target experiences which can be set by using the system either by providers of a service or by users. Target experiences are different types of transient psychological states that have varying durations, frequencies, intensities, combinations, and motivational and action tendencies as well as a linked stimulus class which facilitates a particular state. The system is set up to either approach or avoid a certain target experience within the other parameters of customization such as altering the intensity, duration or frequency of a certain effect or creating simultaneous combinations or links to probable behavior of different target experiences. [3] Psychological Customization can infer customer needs via user models and various feedback loops observing customer behavior and responses. Psychological Customization also provides different adaptations of presenting information to different customers based on the customer interacting with the configuration settings of the product or service. The basic functioning of the system is based on a classic control theory model, the biocybernetic loop. It defines two kinds of control loops in complex and adaptive systems that can be established: negative (avoid an undesirable standard) and positive (approach a desirable standard) loops of feedback [e.g. 4, 5]. Target experiences are then controlled by this type of reasoning in the system based on real-time feedback from user responses and/or based on ready-made design-rule databases. Emotion. Although various definitions of emotions have been proposed, the most general definition is that emotions are biologically based action dispositions that have an important role in the determination of behavior [e.g., 6 ]. It is generally agreed that emotions comprise three components: subjective experience (e.g., feeling joyous), expressive behavior (e.g., smiling), and physiological activation [e.g., sympathetic arousal, 7]. There are two main competing views of emotions. Proponents of the basic distinct emotions argue that emotions, such as anger, fear, sadness, happiness, disgust, and surprise, are present from birth, have distinct adaptive value, and differ in important aspects, such as appraisal, antecedent events, behavioral response, physiology, etc. [8]. In contrast, according to a dimensional theory of emotion, emotions are fundamentally similar in most respects, differing only in terms of one or more dimensions. Proponents of the dimensional view have suggested that all emotions can be located in a two-dimensional space, as coordinates of valence and arousal [or bodily activation; e.g., 6, 9]. The valence dimension reflects the degree to which an affective experience is negative (unpleasant) or positive (pleasant). The arousal
408
T. Saari et al.
dimension indicates the level of activation associated with the emotional experience, and ranges from very excited or energized at one extreme to very calm or sleepy at the other. Other theorists have, however, suggested that the two main, orthogonal dimensions of emotional experience are negative activation (NA) and positive activation (PA) that represent a 45° rotation of the valence and arousal axes [10]. The NA axis extends from highly arousing negative emotion (e.g., fear) on one end to low-arousal positive emotion (e.g., pleasant relaxation) on the other, while the PA axis extends from highly arousing positive emotion (e.g., joy) to low-arousal negative emotion (e.g., depressed affect). The self-report NA and PA dimensions have been suggested to represent the subjective components of the BIS and BAS, respectively [e.g., 10, 11]. We adopt the latter definition of emotion. On the NA axis we call the high arousal negative emotion anxiety and stress while the low arousal emotion can be termed as pleasant relaxation. On the PA axis we see the high arousal emotion as joy and the low arousal emotion as depression. In our studies we have successfully used both psychophysiological measurements and self-report to index emotional processes, also when playing computer games. In computer games, there is a dynamic flow of events and action, games potentially eliciting a multitude of different emotions varying across time. A serious limitation of prior game studies is that they have used tonic, rather than phasic, psychophysiological measures. Tonic measures (e.g., the mean physiological value during the game minus pre-game baseline) do not enable the examination of the varying emotions elicited by different instantaneous game events. Given that psychophysiological measurements can be performed continuously with a high temporal resolution, it is possible to quantify phasic responses to instantaneous game events (e.g., by comparing the local pre-event baseline to physiological activity immediately following event onset). [12] It is then evident that psychophysiological measurement of emotional states when playing a computer game is both feasible and fruitful in providing an account of some aspects of the moment-to-moment experience of the user [13]. Naturally, psychophysiology could be extended to function as a feedback loop into the gaming engine making real-time adaptation of the game relative to the emotional state or mood of the user a possibility.
2 Emotionally Adapted Games Emotionally adapted gaming can be seen as based on gaming templates which are parts of the meta-narrative of the game. Hence, a basic approach to an element to be adapted inside a game is a psychologically validated template which creates a particular psychological effect. A broad view of templates may be that the whole game consists of a database of psychologically validated templates that are dynamically presented by the gaming engine in sequences during gameplay. A limited view entails that a smaller collection of templates is used. The element of psychological evaluation means that the selected psychological influence (such an emotional response) of the template on a particular type of user is sufficiently predictable. These psychologically evaluated templates may consist of i) manipulating
Emotionally Adapted Games – An Example of a First Person Shooter
409
the substance of a game, such as story line (initiating events, new characters etc.) and manipulating the situations specifically related to the character of the player (such as putting the character into sudden and dangerous situations inside the game) and ii) manipulating the form or way of presentation of the game (such as visual elements, shapes, colors, types of objects, sound effects, background music, level of interactivity and feedback etc.). The difficulty level of the game may also be continuously automatically be adjusted, thereby keeping the skills and challenges in balance, which results in a maintenance of an optimal emotional experience and possibly also a flow-state. [14] Why and when then to manipulate emotion in gaming on the basis of avoiding or approaching a specific emotional state? First, there are the transient basic emotional effects of games that are dependent of the phase of the game or some specific events. These are emotions such as happiness, satisfaction, sadness, dissatisfaction, anger, aggression, fear and anxiousness. These emotions are the basis of narrative experiences, i.e. being afraid of the enemy in a shooting game, feeling aggression and wishing to destroy the enemy and feeling satisfaction, even happiness, when the enemy has been destroyed. Emotional regulation systems in these instances most naturally may focus on manipulating the event structures, such as characters, their roles, events that take place and other features of the narrative gaming experience. [14] Second, there are possibilities for emotional management, especially in the case of managing arousal, alertness and excitation. Also, one may wish to manage negative emotions, such as sadness, dissatisfaction, disappointment, anger, aggression, fear and anxiousness. The case for managing these emotions is twofold. On the one hand, one may see that these emotions could be eliminated altogether in the gaming experience. This can happen via either eliminating, if possible, the emergence of such an emotion in the game. For example, one can make a deliberately happy game with level-playing monkeys in a far away island throwing barrels at obstacles and gathering points. This would include minimum negative emotions. Or, in a game where negative emotion is a basic part of the game, one may wish to limit the intensity, duration or frequency of the emotions via manipulating gaming events and gaming elements so that sadness or fear are at their minimum levels, or that gaming events do not lead to sadness at all. [14] Similarly, managing level of arousal or the intensity, duration and frequency of select negative emotions may be quite feasible in the case of children as a form of parental control. On the other hand, one may wish to maximize arousal, alertness and excitation, perhaps even anger, fear and aggression for hardcore gamers. Third, there are possibilities related to the avoidance of certain types of emotions that are typically indicative of a poor gaming experience. Inactivity, idleness, passivity, tiredness, boredom, dullness, helplessness as well as a totally neutral experience may be indicating that there is some fundamental problem in the usergame interaction. This could be due to poor gaming skills of the user vs. the difficult challenges of the game or some other factors, such as the user is stuck in an adventure game for too long and can not proceed without finding a magic key to enter the next level or so. When a gaming engine detects these emotions in the user, it may adapt its behavior to offer the user more choices of selecting the difficulty level of the game or offer the user some clues as to how to go forward in the game. The game can also adapt its level of difficulty to the player’s skill level. [e.g. 14]
410
T. Saari et al.
Fourth, it is also possible to create different combinations of emotional states (satisfied and angry) or emotional states and other psychological states (pleasant relaxation and efficient information processing) or emotional states and behavior (using specific motivational and action tendencies). [3] All of these possibilities may be relevant. However, the elimination or minimization of certain emotions may be specifically feasible in the case of indicated overly poor gaming experience in which the game may adapt its behavior to assist the user. It should be noted that events in games may change quickly and produce complex situations and hence complex emotions that may change rapidly. Consequently, one should better integrate these approaches into the genre or type of the game, such as driving simulator, first person shooter, sports game such as golf, or an adventure game, or a level-playing game for children. [14]
3 Example: Psychophysiologically Adaptive First-Person Shooter Game We will now present a basic system schematic of an emotionally adapted game in Figure 1. The process of a typical gaming engine is depicted on the left-hand side of the diagram. The engine continuously monitors user input, which is typically collected using a keyboard, a joystick, or other game controllers. This input data is then processed and transferred to the layer that handles the game’s internal logical state, and the user input may influence the game state. After the logical state of the game is defined the system alters the actions of the synthetic agents in the game world. For example, these include the actions of computer-controlled non-player characters. The complexity of this AI layer varies greatly depending on the game. Based on the game state and the determined actions of the synthetic agents, the physics engine determines the kinetic movements of different objects within the world. Finally, the game world is synthesized for the player by rendering the graphical elements and producing and controlling the audio elements within the game. [see 14] The proposed emotional regulation can be implemented as a middleware system that runs parallel to the actual game engine. The input processing layer of the game engine can receive a data flow of captured and pre-processed sensor data. The realtime signal processing may consist of different forms of amplifying, filtering and feature selection on the psychophysiological signals. This data flow may directly influence the state of the game world, or it can be used by the emotional regulation sub-module of the emotion feedback engine. This module consists of the rules of emotional balancing for different player profile types and gamer-related explicitly set preferences controlled by the “emotion knob”. In addition, it contains a collection of design rules for narrative constructions and game object presentation within the game world. The emotional regulation module also receives input from the game engine’s logical layer to make selections related to desired emotional balance and narrative structures within the game. [14] The outputs of emotional regulation engine may then be applied to various different levels of the actions of the game engine: i) the logical state of the world may be re-directed, ii) the actions of the synthetic agents may be controlled, iii) the
Emotionally Adapted Games – An Example of a First Person Shooter
411
kinetics of the game may be altered and iv) the rendering of the game world may be changed. First two options are more relevant to high-level and story-related structures of the game, whereas the last two are more directly related to the selection of presentation of objects within the virtual environment. [e.g. 14]
Main game engine Input processing layer
processing player’s input data flows in real-time
Game logic layer
Emotion feedback middleware engine Data acquisition
collecting and pre-processing biofeedback sensor data
Emotional regulation
determining the state of the game world
balancing emotions in the game for different player profile types
Artificial Intelligence layer
story-based design rules to control the game content
Narrative rules
determining how synthetic agents act in the world
Presentation rules
Kinetics layer enforcing the world’s physics laws on game objects
database of presentation rules for different game objects
Emotion knob
Synthesizing layer
rendering displayed images and producing game audio
player-set content preferences
Fig. 1. Emotional adaptation system design for games. Adapted from [14].
With our system design for games it is possible for the game designer as well for the user to set desired emotional targets to be approached or avoided. The system uses both positive and negative feedback loops to determine the ideal adaptations case-bycase for game play for various emotional effects to be realized and managed. Indeed, to implement and evaluate some of the ideas presented, we have explored novel technical solutions and tested different kinds of psychophysiological adaptations that can be implemented. EMOShooter is a prototype platform for psychophysiologically adaptive 3D first-person shooter (FPS) gaming. It is built on top of open-source graphics engine (OGRE 3D) and physics engine (ODE). In this experimental platform we have the possibility to modify practically any game world element, player avatar, avatar outlook, or control parameter. EMOShooter is a simple psychophysiologically adaptive game and hence a part of our emotionally adapted games definition. The system uses psychophysiological signals to influence the ease of use of the controls of the game hence affecting game play difficulty and game play experience. The system does not have target experiences systematically implemented at this moment nor does it have an emotion knob to tune the system. However, the EMOShooter game is a valuable example of one type of emotionally adapted games in demonstrating one feasible link between real-time emotional state measurement with psychophysiology and the game play. The goal of the EMOShooter game is to kill cube-like enemies either with sniper or machine gun. We have been testing various adaptation patterns with EMOShooter by primarily EDA and respiration as psychophysiological signals in our adaptive
412
T. Saari et al.
feedback system regards how these signals can be meaningfully connected to the actual game play via adapting game controls. Adaptation of game controls includes changes in rate of fire, recoil, movement speed and shaking. If a player is aroused this will be reflected in EDA and respiration signals which in turn will make rate of fire and movement slower and will make the aim shaky. Hence, for a highly aroused player the game becomes more difficult. For a mildly aroused or calm player the controls become more efficient and easy to use hence facilitating performance at game play. Game events are mostly arousing. The amount of cubes to shoot, their approach and firing on the user, the amount of health left after being hit and the sound effects all are geared to drive up arousal in the game. The player’s task is to be calm as indexed by psychophysiological signals to be able to operate the controls more efficiently. In our tests of the game we have collected also EMG data to infer the valence dimension of emotion during game play. In addition to the psychophysiological signals we have collected data from the players using behavioral game logging, video capture, interviews and questionnaires. During our tests we noticed that proper calibration and base lining of the psychophysiological signals is very important for the adaptations to work. We also noticed that having robust stimuli in the game is crucial for the adaptations to work because in many cases the stimulus functioned as a trigger in adaptation. The psychophysiological signals used are calibrated by using dynamic range (basically a variation of dynamic signal normalization algorithm), which has a memory buffer of a few seconds (depending on signal). Dynamic range is easy to use and effective calibration mechanism, and relative change seems to be more practical than absolute values in this kind of gaming. According to our early analysis, there are three key issues in designing psychophysiologically adaptive games i) understanding the meaningful emotionally adaptive gaming patterns, ii) implementation of adaptation algorithms and signal processing, and iii) purposeful use of sensors in the game context [15]. The design patterns used in emotionally adaptive gaming must be meaningful and enjoyable for the player, and the utilization of signals must also obey the overall goal of the game. In order to achieve the goal player should find the right rhythm or balance of playing the game and control of psychophysiological responses and signals. Signals should be analyzed as close to real-time as possible in psychophysiologically adaptive gaming in order to keep the feedback loop in pace with the game adaptations and game events. We have used time-series analysis with short sample windows. In practice, ECG, EEG and EMG always require extensive data processing, but EDA and respiration can be almost used as such to create the adaptation signal. This implies that not all psychophysiological signals are equally open to be used as real-time inputs into an adaptive game at least in this stage of signal processing hardware and software development. Usability of psychophysiological recording devices remains quite poor. Respiration, HR [heart rate] and EDA are probably the easiest to implement. Also in case of emotional adaptation the design of the game may include the physical design of the sensors, e.g. “Detective hat” for EEG sensors or “Sniper-gloves” for EDA sensors. Hence, the sensors could be designed as part of the game story rather than presented as cumbersome and invasive laboratory-originated equipment.
Emotionally Adapted Games – An Example of a First Person Shooter
413
In future versions of EMOShooter we may also employ the system design of emotionally adapted games including setting of explicit experiential targets and their parameters for gaming sessions and the emotion control knob.
4 Discussion Gaming, as we have presented it in this article, is perhaps one of the most promising application areas of Psychological Customization. We see that both casual and hardcore gamers could benefit from the use of our system and entirely new types of games can be created. Psychological Customization would enable game designers to use our tools both when developing the game and testing it with users in a rapid manner as well as part of the final product. From the user’s point of view, using various types of control knobs of emotion or other experiences enables them to customize and have more control over their gaming experiences. Good games are composed of delicate synthesis of the components creating a pleasant game balance and challenge for players. Introducing emotional adaptation increases the complexity of game design tasks involved. However, regards the economics of game development our system would not induce a dramatic cost. The system automatically establishes gaming patterns and structures which would fill a target experience and its parameters. Our system could be a modular toolset that can be adapted to various types of gaming platforms and gaming engines. The emotional tuning knob could be integrated into existing game controls including sliders for level of graphic violence in the game, for instance. Of course, development work is needed to create an easy-to-use game adaptation interface for users to set their preferences for game play. There are several challenges for Psychological Customization systems in games. We see three main areas which are critical: i) measurement of experience, ii) quality of reasoning in an adaptive system and iii) acceptability by users. The measurement of experience in the first place is a key challenge. However, we have presented an approach to concentrate on those aspects of experience, such as emotion, which are perhaps better defined than experiential states in general. Despite this focus, there is still disagreement in theorizing, operationalizing and measuring emotions. The quality of reasoning in an adaptive system is often a bottle-neck in performance. If a closed system is produced with fixed design-rules it would inevitably encounter situations, stimuli and users which would challenge the systems fixed rules and produce errors in adaptations. While there are several deep and sophisticated technical and mathematical approaches to this problem including different ways of machine learning and reasoning (which are beyond the scope of this article), we propose a higher-level solution possibility. Our answer is to rely on the co-evolutionary potential of our system design with changing user models and emergent design-rules. Of course, this approach needs working algorithms and techniques to form the necessary metadata and other data structures to be processed by the system. User acceptance of invasive psychophysiological measurement as input to the game is critical. Hardcore gamers may be more suspect to accept new peripheral
414
T. Saari et al.
devices linking them to game than gaming novices or casual gamers. However, the culture of connecting one’s body to a game is already evolving. Think of Wii as an example with a controller tied to one’s wrist, constantly touching the skin. It would not be unimaginable to think of psychophysiological sensors embedded in similar controls as people are more used to “semi-invasive” gaming controls beyond the use of a mouse and keyboard. The solution here could be to design sensors as embedded into essential existing or new types of gaming peripherals. A driving wheel with EDA and ECG sensors or driving gloves with similar sensors with added blood pressure, muscle tension and finger movement sensors could be used as easily acceptable controls of a driving simulator, for example. In our tests of the psychophysiologically adaptive game as a first prototype of emotionally adapted games we have been able to produce meaningful gaming patterns and game adaptations. We argue that our approach to emotionally adapted games is novel and creates new opportunities for designing games. We feel that our approach may result in a new type of enabling technological platform focused on the customization of gaming experiences. This new enabling technology platform can facilitate the development new types of games but can also be used with existing types of games and gaming platforms.
References 1. Grodal, T.: Video games and the pleasures of control. In: Zillmann, D., Vorderer, P. (eds.) Media entertainment: The psychology of its appeal, pp. 197–212. Lawrence Erlbaum Associates, Mahwah (2000) 2. Vorderer, P., Hartmann, T., Klimmt, C.: Explaining the enjoyment of playing video games: The role of competition. In: Marinelli, D. (ed.) Proceedings of the 2nd International Conference on Entertainment Computing (ICEC 2003), Pittsburgh, pp. 1–8. ACM Press, New York (2003) 3. Saari, T., Turpeinen, M., Ravaja, N.: Technological and Psychological Fundamentals of Psychological Customization Systems – An Example of Emotionally Adapted Games (A manuscript submitted for publication) 4. Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology 40, 187–195 (1995) 5. Wiener, N.: Cybernetics: Control and communication in the animal and the machine, 2nd edn. MIT Press, Cambridge (1948) 6. Lang, P.J.: The emotion probe. Studies of motivation and attention. American Psychologist 50, 372–385 (1995) 7. Scherer, K.R.: Neuroscience projections to current debates in emotion psychology. Cognition and Emotion 7, 1–41 (1993) 8. Ekman, P.: An argument for basic emotions. Cognition and Emotion 6, 169–200 (1992) 9. Larsen, R.J., Diener, E.: Promises and problems with the circumplex model of emotion. In: Clark, M. (ed.) Review of personality and social psychology, vol. 13, pp. 25–59. Sage, Newbury Park (1992) 10. Watson, D., Wiese, D., Vaidya, J., Tellegen, A.: The two general activation systems of affect: Structural findings, evolutionary considerations, and psychobiological evidence. Journal of Personality and Social Psychology 76, 820–838 (1999)
Emotionally Adapted Games – An Example of a First Person Shooter
415
11. Gray, J.A.: The neuropsychology of temperament. In: Strelau, J., Angleitner, A. (eds.) Explorations in temperament: International perspectives on theory and measurement, pp. 105–128. Plenum Press, New York (1991) 12. Ravaja, N., Laarni, J., Saari, T., Kallinen, K., Salminen, M.: Phasic Psychophysiological Responses to Video Game Events: New Criterion Variables for Game Design. In: Proceedings of HCI 2005, Las Vegas, NV, USA (2005) 13. Saari, T., Turpeinen, M.: Towards Psychological Customization of Information for Individuals and Social Groups. In: Karat, J., Blom, J., Karat, M.-C. (eds.) Personalization of User Experiences for eCommerce. Kluwer, Dordrecht (2004) 14. Saari, T., Ravaja, N., Turpeinen, M., Kallinen, K.: Emotional Regulation System for Emotionally Adapted Games. In: Proceedings of FuturePlay 2005 conference, Michigan State University, USA, October 13-15 (2005) 15. Kuikkaniemi, K., Laitinen, T., Kosunen, I.: Designing emotionally adaptive gaming. In: Proceedings of EHTI 2008: The First Finnish Symposium on Emotions and HumanTechnology Interaction, University of Tampere, Tampere, Finland, pp. 13–18 (2008)
DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities Tatsushi Takeuchi1, Koichiro Watanabe1, Tomoo Inoue2, and Ken-ichi Okada1 2
1 Graduate School of Science and Technology, Keio University Graduate School of Library, Information and Media Studies, University of Tsukuba {takeuchi,watanabe,okada}@mos.ics.keio.ac.jp, [email protected]
Abstract. This paper describes a system called DiamondTheater that supports creative activities in theater using a tabletop tangible interface. This system is used to aid in the planning of certain aspects of theater production, such as actor positioning, and sound and lighting cues. Without having some considerable experience in production, it is difficult to create a mental picture of an actual production. Thus, a miniature theatrical stage is reproduced on the tabletop surface to facilitate the user’s creation of such a picture and the sharing of ideas. Users collaboratively construct a stage by placing small dolls to represent actors and many kinds of miniature stage sets. In addition, the system allows users to reproduce other aspects of theater production such as sound and lighting. We performed a user study of this system and demonstrated that DiamondTheater appears to effectively assist the user’s activity in theater production design. Keywords: collaborative work, theater, tabletop tangible interfaces.
DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities
417
on the process of designing and arranging theater productions for actual public performance. By constructing a miniature theatrical stage on a tabletop surface, DiamondTheater helps users imagine and design theater productions, assisting with aspects of theatrical productions such as lighting and sound, as well as the interplay between actors and other theatrical elements. Each user can work while sharing the entire image of the production with the other users.
2 Creative Activity in Theater Theater as an artistic production is a branch of the performing arts and is usually created by utilizing a combination of diverse elements, such as stage lighting, sound effects, special effects, and stage sets. Staff members work according to their roles. A general example of a work flow in theater leading up to a public performance is shown in Figure 1. The cue sheet in Figure 1 is a form or template that lists information about cues. A theatrical cue is the trigger for an action to be carried out at a specific time. An action could be, for example, a lighting or sound change. Usually information about execution, timing, sequence, and details of the production are also written on the cue sheet. The board operators and running crews operate the theater productions they are involved in during a performance while confirming the information written on the cue sheet dealing with their department.
Fig. 1. A simple example of the work process leading up to a public performance
3 Related Work Many researchers have studied systems that support a wide variety of creative activities, such as story creation, animation and drawing[3,4]. Although many systems supporting two-dimensional creative activities have been studied frequently, systems for three-dimensional creative activities, such as a dance and theater, have not been developed well. Little is known about systems to support activities for these branches
418
T. Takeuchi et al.
of the performing arts. Several researchers have developed systems for theater activities, almost all of which use computer graphics and virtual spaces to represent the theater stage[5,6]. These systems using virtual spaces help users imagine the actual stage used in theater. However, virtual images are somewhat lacking in their ability to represent reality. Moreover, these systems help users to understand basic theatrical concepts, but are not systems that can be used to support real-life work. Of the few systems that support real-life staff works activities in theater, Avatar[7] is an application for supporting the distributed collaborative production of theater shows. The authors use virtual reality to represent certain theater scenes. When designers cannot actually be co-located to collaborate with each other on the design, Avatar displays the virtual scene using computer vision at the remote locations to share the scene in real-time. In Avatar, small objects with AR markers are used as input devices to the virtual world that is being shared by remote directors who are participating in the design. The poses and positions of the objects are recognized by using a webcam to observe the markers. This idea would be useful to facilitate the collaboration of remote users in production design. However, it is probably better to use real mockups in the case of co-located collaboration because the designers can share the workspace. There are many commercially produced systems for designing theater productions. For example, Matrix3[8] is a system for audio show control, and wysiwyg[9] is used to plan, design, and program lighting productions. These systems assist user's theater production design; however, users who are inexperienced in theater would be not able to operate these systems effectively since almost all such systems are tools for professionals. Moreover, the interfaces of these systems are usually PC-based and use a GUI, and these are normally systems for personal use for that reason. Thus, such systems are not assumed to be used by multiple users. We are interested in a system that can support the real-life work of staff, especially co-located collaboration in theater. We focus on the pre-production process of theater regarding theater production design. When compared with people that have a wide range of experience, it is difficult for staff without much experience to picture the actual stage, which includes a wide variety of theater elements such as sound and lighting, when thinking about theater productions. Furthermore, a cue sheet is created after the discussion of the theater production, but it is very complicated to write out a cue sheet by hand or using a PC based on the theater productions that were decided regardless of whether the person has experience or not.
4 System Design Our objective is to use our system to solve some of the above-mentioned problems. The major objectives of our system are (1) to help users create a mental picture of the actual stage when designing and considering a plan for theater productions, (2) to provide users with opportunities to share the ideas of each user, and (3) to reduce the amount of effort required to create a cue sheet. This system helps users visualize the actual stage and theater productions. Users can collaboratively discuss the theater productions for actual live performance using this system. The interface is based on an interactive tabletop surface, on top of which users can reproduce a small stage. Our system uses MERL's DiamondTouch table[10]
DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities
419
to support face-to-face collaborative work, and allows users to collaboratively design theater productions while sharing ideas. Users can also easily reproduce theater productions that they conceive, such as lighting and sound, on the reproduced stage, and easily confirm the relationship between the actors and other elements. Because the theatrical stage is represented three-dimensionally on the DiamondTouch surface, it is easier to capture the entire image of the actual stage, including depth and height, than when using theaters with virtual spaces.[5,6,7] The theater production data that users design and finalize is saved as electronic data in an XML (Extensible Markup Language) file. A cue sheet is automatically created based on the theater production data, and the users can obtain cue sheets as needed. As a result, the system saves users the trouble of making cue sheets. At the present stage we intend for our system to be used by people who are inexperienced in theater, such as children and students. It is difficult for inexperienced people to design theater productions while comprehensively creating a mental picture of them. We believe that our approach is an effective way for such people to enhance their creativity and to support creative activities in theater.
5 Implementation 5.1 Overview and Architecture A system image of DiamondTheater is shown in Figure 2. This system consists of two types of displays: DiamondTouch for reproducing theater productions and the stage, and a semi-transmissive display for showing information about the script and theater productions that the user has selected. We represent a theatrical stage on top of the DiamondTouch table. This is placed on a large rack, as shown in Figure 2. Placing a doll as a surrogate for an actor and placing small stage sets, the system reproduces a stage on the surface as shown in Figure 2. Specific graphics are drawn on the tabletop using an overhead video projector. The stage and theater productions are graphically projected onto the table from above as shown in Figure 3. The DiamondTouch senses the position and movement of the dolls placed on the tabletop surface.
Fig. 2. System image in which users are thinking about the production collaboratively
420
T. Takeuchi et al.
Fig. 3. An operating panel and a theatrical stage projected onto the tabletop surface
Fig. 4. Production and columnar writing script (Japanese style) data projected onto a semitransmissive display
The DiamondTouch can identify the manipulator by recognizing an electric signal that is transmitted through users when they touch the DiamondTouch surface. Then we add a round metal plate to the bottom of each doll and wrap a copper wire around the doll to enable it to be recognized by the system. Using the metal bases enables the electric signal to be carried from the user's body through the metal and wire to the table. When the users place a doll on the tabletop surface, the metal and wire carries the electric signal from the user to the DiamondTouch, and the DiamondTouch recognizes the signal and identifies the manipulators. Moreover, in order to assess the differences among the dolls, we add different-sized metal plates at the bottom of each doll. Therefore, by calculating the size of the metal plate placed on the tabletop surface, the system can recognize which objects the user is manipulating. As a result, the system can distinguish between the positions to which the user may move an object
DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities
421
without the use of any tools other than the DiamondTouch table. If metal plates of different sizes are added to the dolls used, the system can theoretically distinguish between any numbers of dolls. The semi-transmissive display is used to display various additional information. This display is placed upright beside the rack as shown in Figure 2. A sample image of this display is shown in Figure 4. The semi-transmissive display shows data about the theater productions and script information. The data about the theater production that users have decided on at the time of using the tabletop appear at the bottom of the screen. Script information, such as the lines, stage direction, speakers, and page numbers, appear at the top of the screen. These data are projected onto the semitransmissive display from the rear projector. Users can use a script (tangible object) that is actually used in practice. Instant memos and ideas are often written into scripts during practice or during discussions between staff. These actions are important to support the staff's creative activities, and it is for that reason we use a real script. We use RFID technology to load data about the script into the system. We add RFID tags to the pages of the script to be able to identify particular pages when they are viewed. When the page of a real script is turned over, the RFID reader attached to the rack loads the data and reflects the information of the selected page and projects it onto the semi-transmissive display. 5.2 Operation Procedure Users mainly take three steps to determine theater productions with DiamondTheater: (1) Choose line or stage direction to coordinate the theater production, (2) design the theater production while actually reproducing it using the system, and (3) finalize and save the theater production. Users first select a line at which they want to think about the theater production. Using RFID technology, DiamondTheater can identify which page of the script the users are looking at. Every time a page is turned over by the users, the tag data is recognized by the RFID reader, and information about the script on the page is read. The system provides users with a button to select a particular line on the tabletop surface. Users can select a line simply by touching this button while watching the data shown on the semi-transmissive display. Users design the theater productions while reproducing them and participating in collaborative discussions. Each theater production can be determined in association with a single or multiple lines/stage directions. The results of the theater productions that have been designed are saved as electronic data in an XML file. If a user requires a cue sheet, it can be automatically and easily output using DiamondTheater functionality without the need for complicated work. 5.3 Reproduction of Theater Productions The system can reproduce three kinds of stage effects: one of sound and two of lighting. The two types of lighting that can be reproduced are global lighting (border light) and spotlight. The look of turning a spotlight onto a doll instead of an actor is shown
422
T. Takeuchi et al.
in Figure 5. Each lighting effect can be turned on or off using a simple interaction with the system. Therefore, by simply touching a finger to the tabletop surface, users can control the lighting. In addition, sound is also available via a simple interaction. When a user selects by touch a piece of music that he or she wants to play, the music will play from two separate speakers placed near the tabletop. The standing position of an actor on stage is represented using a small doll. An example of the dolls used is shown in Figure 5. The system can recognize the actor's position if a user places a doll on the tabletop surface. Based on the position that the user places the doll, an image which expresses the actor's position is graphically projected on the tabletop (A or B shown in Figure 3). The users consider and determine the actor's position. Then, when the doll is quickly touched twice to the tabletop surface at the designated position, the position data is determined and saved in an XML file.
Fig. 5. A spotlight turned on a doll used to represent an actor
5.4 Use of Production Data We have defined a new language to save it, called CueML (Cue sheet Markup Language), which is based on XML. CueML is the language used to save data about the theater productions and to produce the cue sheet. The data for each production is saved and related to a single or multiple line(s)/stage direction(s). A cue sheet for reference during the actual performance is automatically produced by the system using the CueML data. A variety of XML elements are defined using XML tags in CueML. We can save more detailed information of a theater production in addition to the data that can be saved using the current implementation of DiamondTheater. For example, we can save data regarding volume and a sound effect, intensity and color for lighting, and so on. The current version of DiamondTheater cannot, however, deal with such detailed information. However, we will be able to extend the functionality of DiamondTheater to include these functions in the future. The system allows users to refer to theater productions that other groups saved previously by reading in data from a previous or current CueML file. This allows users to design theater productions by referencing other ideas. This process will support users in getting new ideas when users come to a deadlock.
DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities
423
6 User Study 6.1 Overview We conducted a user study to clarify the effects of DiamondTheater on users. The objective of this study was to observe how the system would affect the users' theater production planning, and how users would interact with the system. Fourteen university participants aged 21 to 24 years were asked to participate in this experiment. The participants worked in pairs during the experiment. All of the participants were inexperienced in regards to theater activity, and were novice users of the interface that we developed. Participants were asked to freely and collaboratively design theater productions about a script that we prepared using the interface. We used another interface using a PC in order to compare DiamondTheater to a system without DiamondTheater. The participants received an explanation of how to use each system before starting the experiment. A script that we produced for two-page spread was used for this experiment. The participants were asked to decide on three types of theater productions without a time restriction using each system and the script. The three types were sound, lighting, and actor positioning. The experiment had the following two system setups: DiamondTheater, and a PC and dolls. y DiamondTheater: Participants were provided with two dolls associated with actors. The positions of these dolls when placed on tabletop surface were automatically identified by DiamondTheater. The user received an explanation of how to control and turn on and off the sound and stage lighting. After designing the theater productions, the participants were asked to save their data. y PC and dolls: This case was similar to the work usually undertaken by theater production staff. The sound was played using a PC. However, stage lighting was not available because it cannot be easily reproduced in the normal environment. Participants were provided with two dolls to help them create a mental picture of their idea, the same as was done with the DiamondTheater-based system. Participants were asked to write the results of their designs on the script by hand. y We wanted to understand how DiamondTheater would allow participants inexperienced in theater to create a mental picture of their actual theater productions. We observed the user interaction in each setup, and analyzed whether DiamondTheater supported user activities related to theater production design even if the users had little or no experience in theater. After designing theater productions, we asked participants whether they noticed any particular aspects of the system. 6.2 Results Through this experiment, we were able to confirm some practical advantages of DiamondTheater in the process of planning theater productions. Participants seemed to actively interact with DiamondTheater while communicating with each other. With both system setups, participants designed a theater production freely with their own ideas while showing each other their ideas. The considerable difference between the two setups was the number of interactions with each system. Participants clearly interacted with DiamondTheater more actively
424
T. Takeuchi et al.
compared to the setup using a PC and dolls. DiamondTheater was likely a more familiar and intuitive interface than the PC and dolls setup. The participants were able to use the system easily by touching the tabletop screen or using physical objects (dolls and a script). Therefore, the participants were able to quickly become accustomed to the operations when using DiamondTheater. With DiamondTheater, almost all of the participants proactively tried to reproduce their ideas for the theater productions as they came up with them. As a result, the participants could design theater productions and share specific images while having an actual mental picture of their ideas. On the other hand, participants often thought about the three kinds of theater productions separately when using the setup using a PC and dolls. Although the participants were able to confirm the sound and actor positions, reproducing the stage lighting was normally difficult. Participants were not able to create a comprehensive mental picture of their production idea due to this constraint. This is a problem that could possibly occur in the process of actually designing a theater production. In addition, when using the PC and dolls setup, the participants sometimes selected incorrect productions for certain scenes. We define an incorrect production as incongruous or not appropriate for a certain scene. For example: y All lighting is turned off when there are lines of dialog. y Only one spotlight is turned on when two actors are talking. y The distance between the standing positions of the actors is too distant when they are talking with each other. We counted the number of incorrect productions in each single line or stage direction based on the theater production data collected from participants. Therefore, if an incorrect production occurred for three consecutive lines, the number was three. As a result, we found only one incorrect production when the DiamondTheater setup was used. However, nine incorrect productions were found when the PC and dolls setup was used. Incorrect productions increased dramatically when DiamondTheater was not used. All of the incorrect productions were found to concern stage lighting. Incorrect productions occurred at stage direction (for example, "He said it with a grim expression"), where participants decided on theater productions without a light turned on the actor in such situations. If all of the lighting were turned off, the audience would be not able to know what the actor's expression was. The number of incorrect productions was dramatically reduced when using DiamondTheater than when not. This result suggests that DiamondTheater adequately supported the participant's activities. Judging from the experiment, the participants worked while actually confirming their ideas with DiamondTheater. This is why the participants would seem to be able to accurately understand the situation of the theater productions that they were thinking about. However, with the setup using the PC and dolls, the participants were not able to adequately create a mental picture of their production ideas, especially in regards to stage lighting. This is because the participants could not try out their lighting plans with such a system, which was not the case with DiamondTheater.
DiamondTheater: A System for Reproducing Theater and Supporting Creative Activities
425
7 Conclusion In this paper, we have presented a system called DiamondTheater. This system in its current form is for use by people who are inexperienced in theater. DiamondTheater provides users with the opportunity to reproduce productions on a miniature theatrical stage. With the ability to reproduce various productions as well as an actual theatrical stage, users can design productions efficiently with a clear image of the production and collaborate when conceiving ideas. Participants in our user study were more successful and efficient when using DiamondTheater at design tasks in spite of their lack of experience. Thus, DiamondTheater allows users to use free-thinking when working in order to develop good ideas for diverse theater productions, and supports real-life creative work activities using simple operations.
Acknowledgments We would like to acknowledge Mitsubishi Electric Research Laboratories for donating a DiamondTouch table.
References 1. Patten, J., Ishii, H.: Mechanical constraints as computational constraints in tabletop tangible interfaces. In: ACM CHI 2007, pp. 809–818 (April/May 2007) 2. Ishii, H., Ullmer, B.: Tangible bits: Towards seamless interfaces between people, bits, and atoms. In: ACM CHI 1997, March 1997, pp. 234–241 (1997) 3. Raffle, H., Vaucelle, C., Wang, R., Ishii, H.: Jabberstamp: embedding sound and voice in traditional drawings. In: The 6th international conference on IDC 2007, pp. 137–144 (2007) 4. Drori, T., Rinott, M.: Pixel materiali - a system for creating and understanding pixel animations. In: IDC 2007, pp. 157–160, June 6-8 (2007) 5. Lewis, M.: Bowen virtual theatre. In: ACM SIGGRAPH 2005 conference on Web graphics 6. Geigel, J., Schweppe, M.: Virtual theatre: a collaborative curriculum for artists and technologists. In: SIGGRAPH 2005 (2005) 7. Dompierre, C., Laurendeau, D.: Avatar: a virtual reality based tool for collaborative production of theater shows. In: CRV 2006, pp. 35–41 (June 2006) 8. Matrix3, http://www.meyersound.com/lcs/matrix3/ 9. Wysiwyg, http://www.cast-soft.com/cast/software/products.jsp? SUBCATID=2 10. Dietz, P., Leigh, D.: Diamondtouch: A multi-user touch technology. In: User Interface Software and Technology (UIST 2001), pp. 219–226 (2001)
New Health Information Systems (HIS) Quality-in-Use Model Based on the GQM Approach and HCI Principles Reem Al-Nanih1, Hana Al-Nuaim1, and Olga Ormandjieva2 1 Department of Computer Science, King Abdul Aziz University, Saudi Arabia Dept of Computer Science & Software Engineering, Concordia University, Canada [email protected], [email protected], [email protected] 2
Abstract. Human Computer Interaction (HCI) is concerned with the design, evaluation, and implementation of interactive computing systems for human use. HCI is important in Health Information Systems (HIS), because misunderstandings arising because of poorly designed interfaces may lead to medical errors. This paper proposes a Quality-in-Use Model for HIS user interfaces, which identifies HIS-specific quality-in-use goals based on HCI principles, such as Mental Model, Metaphor, Visibility, Affordance, and Feedback. The Goal Question Metric (GQM) method was applied to build the new quality-inuse model applicable to most HIS systems. The resulting quality model is tailored for use in the medical field and reflects the values and viewpoints of the various user groups affected (e.g. doctors and nurses). Its qualitative and quantitative feedback can play a constructive and instructive role in medical institutions such as hospitals, and improve user productivity, satisfaction, and performance. Keywords: Health Information System (HIS), Human Computer Interaction (HCI), Goal Question Metrics (GQM), Quality-in-Use.
and satisfaction. The proposed quality-in-use model was evaluated on the Phoenix Health Information System (PHIS) currently in use at the King Abdulaziz University Hospital (KAUH). The application of our HIS Quality-in-Use model to the PHIS and the analysis of the quality feedback helped us produce a system (PHIS2) designed to minimize human error, reduce user frustration, and help doctors − with little or no training − work in a more pleasant, efficient, and effective way. The rest of the paper is organized as follows: Related work on HIS evaluation and HIS quality modeling is reviewed in section 2. Section 3 introduces the HCI principles required to understand the quality-in-use model proposed in this paper. The model itself is described in section 4. The results of the empirical validation on a case study are summarized in section 5. The conclusions and directions for future work are outlined in section 6.
2 Motivation and Related Work Computers are an integral part of modern medicine and diagnosis. This is especially so in hospitals, as they constitute the core of all advanced medical investigation methods such as ultrasound, computerized tomography, and magnetic resonance imaging [2]. HIS users need interfaces tailored to their specific requirements to facilitate their work and to help them avoid misunderstandings which may lead to medical errors. This need is still underappreciated by software designers [2]. The above motivated the research reported in this paper. HCI and human factors have a significant role to play in increasing the quality of HIS, and consequently in reducing the number of errors, especially those associated with the use of medical devices [5]. Finding design problems in an existing system related to human factors can be a challenge, however, because flaws in user interfaces can be subtle [6]. To the best of the authors’ knowledge, very little has been published on the quality modeling of HIS user interfaces. There has been a study of a hospital's order-entry system which identified twenty-two ways in which the system caused patients to be administered the wrong medicine [7]. Most of the issues involved were related to usability problems; for example, at times, users had to review up to twenty screens to see all of a patient's medications. In [3], the role of evaluation in the design of HIS is emphasized, and a framework for considering evaluation methods ranging from controlled experimental approaches to naturalistic observational approaches applicable throughout a system’s entire development life cycle is proposed. The problem of defining a quality model for evaluating HIS design is addressed in [4]. The model, based on the ISO/IEC 9126 external and internal product design quality, has been interpreted to meet the requirements of some classes of typical HCI system applications, and exploits experience gained both in the field of medical informatics and in the assessment of software products. The values that result from weighing the quality characteristics according to evidence of their criticality constitute a quality profile which can be used both for evaluation and certification of HIS design and implementation from developers’ point of view. The research reported in this paper differs considerably from the related work in the view on quality of HIS user interfaces. Our objective is to build the HIS Qualityin-Use model from the user's view in terms of HCI principles, which encompass the richness in HIS functionality and represent a broader view than software usability.
New Health Information Systems (HIS) Quality-in-Use Model
4 HIS Quality-in-Use Model The objective of this section is to present the Quality-in-Use model tailored specifically to HIS user needs. The Goal Question Metric (GQM) approach [1] was applied to define the user needs for the HIS Quality-in-Use goals, and how these goals are further decomposed into more technical sub goals and the corresponding indicators based on HCI principles.
432
R. Al-Nanih, H. Al-Nuaim, and O. Ormandjieva
GQM. The GQM approach is based on the fact that, for an organization to measure in a purposeful way, it must first specify goals for itself and its projects, then it must trace those goals to the data that are intended to define them operationally, and, finally, it must provide a framework for interpreting the data with respect to those goals [1]. The result of applying the GQM approach is a description of a measurement model targeting a particular set of issues and a set of rules for the interpretation of the measurement data. A GQM model is a hierarchical structure (Figure 1) which starts with a goal. That goal is refined into several sub goals using appropriate questions which usually break the issue down into its quantitative or qualitative indicators. Each indicator is then refined into a measurement procedure and rules for data interpretation. Quality-in-Use. Quality-in-use is the user's view of the quality of an environment containing software and is a broader view than software usability. It is measured based on the results of using the software in the environment (here, the medical field), rather than on properties of the software itself [14]. The ISO/IEC 9126-4 International Standard describes a two-part model for software quality: a) internal quality and external quality, and b) quality-in-use. Quality-in-use is defined as the capability of the software product to enable specified users to achieve specified goals with effectiveness, productivity, safety, and satisfaction in specified contexts of use [14]. In ISO/IEC 9126-4, effectiveness is defined as the capability of the software product to enable users to achieve specified goals with accuracy and completeness; productivity is defined as the capability of the software product to enable users to expend appropriate amounts of resources, such as time, to complete the task or the user’s effort, in relation to the effectiveness achieved; safety is described as the capability of the software product to achieve acceptable levels of risk of harm to people, businesses, software, property, or the environment; and, finally, satisfaction is expressed in terms of the user’s response to interaction with the product, which includes attitudes towards the use of the product).
Fig. 1. Hierarchical Quality-in-Use model for HIS (User Performance subgoal)
HIS Quality-in-Use. The quality-in-use model proposed in this research is rooted in the ISO/IEC 9126-4 definition of quality-in-use [14] and is aimed at helping medical personnel execute work-related tasks, in terms of: i) reducing the medical errors by making the use of HIS more pleasant and easier to manipulate, and ii) increasing their performance, productivity and satisfaction by means of HCI principles. The GQM approach was applied to further decompose user performance, user productivity, and user
New Health Information Systems (HIS) Quality-in-Use Model
433
satisfaction sub goals into their corresponding indicators providing feedback on the achievement of the user sub goals, and measurement procedures for obtaining the data required for the feedback. The decomposition is motivated by the HCI principles in the specific context of use, namely, HIS in the medical field. Figures 1, 2, and 3 illustrate the above decomposition for user performance, productivity, and user satisfaction. Validation of the HIS Quality-in-Use model proposed in this paper consisted of demonstrating the improvement of quality-in-use characteristics user performance, productivity, and satisfaction on existing software, The Phoenix Health Information
Fig. 2. Hierarchical Quality-in-Use model for HIS (Productivity subgoal)
Fig. 3. Hierarchical Quality-in-Use model for HIS (User Satisfaction subgoal)
434
R. Al-Nanih, H. Al-Nuaim, and O. Ormandjieva
System (PHIS, see section 5.1), which, after applying the HCI principles, produced a new version of the system (PHIS2, see section 5.2).
5 Case Study An example of medical software is the Phoenix Health Information System (PHIS). The creation of the PHIS began in 1998 after a team from New Zealand and other countries met in Dubai to evaluate current health information systems. King Abdulaziz University Hospital (KAUH) started to use the PHIS in 2004 to replace the Oasis system. The quality-in-use quality criteria were formally evaluated on the PHIS by novice and expert users from the hospital to identify problems they were having with the system. The researcher concluded that all the problems found based on the tasks doctors commonly perform on a daily basis were rooted in a violation of the HCI principles Mental Model, Metaphor, Visibility, Affordance, and Feedback. 5.1 PHIS Identifying and selecting users. The PHIS is divided into three parts. Each part serves a different group of people in the hospital: administrators, nurses, and doctors. This study focuses on the PHIS subsystem used by doctors and the tasks they commonly carry out on a daily basis. The doctors at the KAUH are listed in three categories: interns, residents, and consultants. Interns use the system the most frequently. The characteristics of the interns are summarized in Table 1. Table 1. Characteristic of the Interns Type of Intern Work Place Time Using the PHIS Training Time Attended the training session
Experienced
Novice In various wards KAUH From 8:00 AM to 5:00 PM From 3 to 7 months From 3 to 5 days Two hour training session available for all new doctors 80% 90%
Collecting data and identifying problems. The two data collection methods used to gather information from experienced and novice users were interviews and observations. Responses and feedback from both groups were recorded: Experienced users: Ten experienced PHIS users were interviewed and observed. These interviews provided the researchers with a clear picture of the problems users encountered regarding learnability and ease of use. The interviews also identified the daily tasks most commonly performed by the doctors on the system, such as: i) review the result of a laboratory or radiology test and check a patient’s prescribed medication, ii) request a new test, and iii) book an appointment for an operation. Novice users: Ten novice PHIS users were interviewed and observed while they carried out a list of tasks on the PHIS to evaluate their ability to use the system without help (Table 2).
New Health Information Systems (HIS) Quality-in-Use Model
435
Table 2. Results of the Task Questionnaire
User 1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 User 9 User 10 Total
Task1 Check a laboratory result × 9 × × × 9 9 9 × × 4/10
Task2 Print a laboratory result × × × × × 9 × × × × 1/10
Task3 Request an order for radiology × 9 9 9 × × × 9 × × 4/10
Task4 Book an appointment for an operation × × × × × × × × × × 0/10
All the interviewees were found to be competent computer users. Five tasks were presented to the users in the questionnaire (Table 2): a check (9) means that the user performed the task successfully, while a cross (×) means that the user was unable to finish the task or to start the task. Forty percent of the users were able to complete tasks 1, 3, and 5, while the remaining users were no. Only 10% could complete task 2 and none of the users was able to complete task 4. Even though the users were proficient in terms of their computer skills and attended the training session, fewer than 50% were able to complete the tasks given to them. Table 3 lists the HIS tasks most commonly performed by the doctors, and the applicable HCI principles: Table 3. Tasks Given to the Novice Group Tasks Task 1.1 Check the last HB Task 1.2 Print the result Task 1.3 Compare CBC Tests? Task 1.4 Print HB from Compare Item Task 2 Order CBC Task 3 Booking Task 4 Check the radiology result Task 5 Print medication report
HCI Principles Mental Model Metaphor Metaphor Mental Model and Visibility Visibility and Feedback Affordance Mental Model, Visibility, and Feedback Mental Model and Feedback
Summary of HCI Problems in the PHIS. The results of interviewing the ten experienced users and the ten novice users indicated that the PHIS is not user-friendly and not easy to learn, and there are several interface design faults which need attention. The proposed quality indicators helped reveal the existing PHIS HCI problems, as outlined below (the problems are grouped by indicator for clarity): Mental Model • Lengthy steps required to reach a target window • Selecting a ward does not filter the teams • Team abbreviations are inconsistent from one ward to another
436
R. Al-Nanih, H. Al-Nuaim, and O. Ormandjieva
Metaphor • Use of the same icon for different functions, e.g.
and
Feedback • Function name of the window title displayed, instead of the window title • In the Profile tab: − there is a list of buttons using vague abbreviations (e.g. MAR/IVAR) − patient's demographic information is not displayed Visibility • Invisible order window Affordance • Diagnosis list not arranged alphabetically or by category • Doctors tried to click on a time text box without noticing the background color The obstacles and problems encountered by the doctors were solved based on the principles of HCI from a design point of view to minimize human error and user confusion, as well as to increase the efficiency and effectiveness. The result was PHIS2, a redesigned and improved version of the PHIS developed through a process that included pilot testing, redesigning, and redeveloping its user interface when it did not reflect HCI principles. 5.2 PHIS2 The following illustrates the improvement of the quality-in-use of PHIS2 by comparing the PHIS and PHIS2 quality indicators per sub goal: User Productivity improvement. After conducting the formal quality assessment of the PHIS and identifying the HCI principles that had been violated, all the recommendations from the findings were applied to improve user productivity with PHIS2. The productivity obstacles and problems encountered by the doctors were solved based on HCI principles from a design point of view to minimize human error caused by the poor quality user interface design of the PHIS. User Performance improvement. The ability of the user to complete a task successfully is used to evaluate the user’s performance. The performance measures are defined in Table 4. Table 4. Performance Measurement Performance measures Time needed by the novice user to perform task successfully How many times he/she chose the wrong icon How many times he/she made the wrong menu choice
Metric End Time- Start Time # of wrong icons selected # of wrong menu choices
Based on the technique suggested by Dumas and Redish [15], the performance of three experienced users was considered as the baseline for judging the time taken by participants. If their performance was considered excellent, more time was considered acceptable and much more time was considered unacceptable. Table 5 shows an example of the selected performance measures for the ‘Check the last HB’ task. Each
New Health Information Systems (HIS) Quality-in-Use Model
437
Table 5. Setting Criteria for Measuring Performance Measure Task 1.1: Check the last HB Time for task M= # of wrong menu choices I= # of wrong icon choices
Excellent < 1.5 M 1 1
Acceptable 1.5-2 M 2 -3 2 -3
Unacceptable >2 M More than 3 More than 3
task for each participant was measured and the data compared to that in Table 5, which classifies user performance as excellent, acceptable, and unacceptable. The results show that the performance of the novice group was excellent for more than 80% of the users for Tasks 1.3, 2, 4, and 5, and for 70%, 80%, and 60% of the users for Tasks 1.1, 1.2, and 3 respectively. For Task 1.4, performance was excellent for 50% of the users and acceptable for 50% of the users. In only one case was performance unacceptable, that is, for 20% of the users for Task 1.1. The data on the number of wrong menu choices show that performance was excellent for Task 4 and Task 5 for 100% of the users and for Task 1.1 for 90% of the users. The data on the number of wrong icons chosen show that performance was excellent for all tasks except Task 3 for 100% of the users, with performance for Task 3 being excellent for 80% of the users. It was interesting to compare the performance of the PHIS with the performance of PHIS2 by ten novice users performing four identical tasks. These tasks represent the Mental Model in Task 1.1, Metaphor in Task 1.2, Feedback and Visibility in Task 2, and Affordance in Task 3. The performance measure in the PHIS was accomplishment of the task, irrespective of the time taken and the number of attempts. Using PHIS2, the novices were able to accomplish their tasks without previous training, while using the PHIS, despite the in-house training, fewer than 40% were able accomplish their tasks. From the user performance evaluation, it is clear that the novice users were able to perform all the tasks with PHIS2 in excellent time, which reflects the accuracy of the proposed design. It also shows that their performance with respect to the number of wrong menus choices and icons was also excellent, which reflects the efficiency and effectiveness of the new UI design. User Satisfaction improvement. The users were asked to complete a User Satisfaction Questionnaire to determine users' impressions and opinions of the system, evaluating the user response to the interaction with PHIS2. The results of the questionnaire revealed that all the users gave ratings of ‘very easy’ and ‘easy’ for the tasks regarding learning the program, using the program, performing the particular task, finding the features, understanding the instruction, and recovering from error. In addition, 100% of the experienced users preferred using PHIS2 to using the PHIS, and 100% of the participants asserted that PHIS2 helps them improve the quality of their work.
6 Conclusion and Future Work The purpose of this research was the application of HCI principles, based on a sound goal-driven approach, to derive a new quality-in-use model from the medical field users’ view of the quality of HIS interfaces which will help to minimize human error and user frustration, and to make the tasks of doctors − with little or no training − more pleasant, as well as efficient and effective. Such a model can serve not only to
438
R. Al-Nanih, H. Al-Nuaim, and O. Ormandjieva
evaluate the quality-in-use of an existing HIS, but also as a guide for the user interface design of new medical software. The PHIS evaluation based on our HIS Quality-in-Use model resulted in major improvements to the user interface design in its improved version, PHIS2. Results of the formal evaluation of PHIS2 reveal that the system is a success in terms of helping users perform tasks in excellent or acceptable time, efficient and effective in terms of limiting the number of wrong menu choices made and wrong icons selected, and more pleasant to use, even with no training whatsoever. More case studies will be evaluated in our future work to determine the applicability of our quality-in-use model to different types of HIS.
References 1. Basili, V., Caldiera, G., Rombach, H.D.: The Goal Question Metric Approach (1) Institute for Advanced Computer Studies, Department of Computer Science, University Of Maryland, College Park, Maryland (2) FB Informatik, Universität Kaiserslautern, http://wwwagse.informatik.uni-kl.de/pubs/repository/ basili94b/encyclo.gqm.pdf 2. The Canadian International Development Agency (CIDA) and managed by the Canadian Society for International Health, The South Caucasus Health Information Project (2004), http://www.csih.org/what/schip/March 3. Kushniruk, A.: Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Computers in Biology and Medicine 32, 141–149 (2002) 4. Fabbrini, F., Fusani, M.: Evaluation of Quality Characteristics in Health Care Information Systems, ERCIM News No.28 (January 1997), http://www.ercim.org/publication/Ercim_News/enw28/ fabbrini.html 5. Burnham. Dr. W.: The Human Factor, http://www.humanfactorsmd.com/hfandmedicine.html (2005) 6. Sawyer, D.: An Introduction to Human Factors in Medical Devices, Office of Health and Industry Programs, U.S Department of Health and Human Services, Center for Devices and Radiological Health (1996), http://www.fda.gov/cdrh/humfac/doitpdf.pdf 7. Nielsen, J.: Medical Usability: How to Kill Patients through Bad Design (2005), http://www.useit.com/alertbox/20050411.html 8. Marcus, A.: Metaphor Design in User Interfaces: How to Manage Expectation, Surprise, Comprehension, and Delight Effectively. California (1997), http://www.sigchi. org/sigchi/chi97/proceedings/tutorial/am.htm 9. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human-Computer Interaction, 3rd edn. British Library Cataloguing, England (2004) 10. Booth, P.: An Introduction To Human-Computer Interaction, 4th edn. Lawrence Erlbaum Associates, Mahwah (1989) 11. Norman, D.: The Design of Everyday Things. Doubleday/Currency, New York (1988) 12. McCracken, D., Wolfe, R.: User-Centered Website Development: A Human- Computer Interaction Approach. Pearson Education, USA (2004) 13. Norman, D.: Introduction to HCI, B.Sc. Applied Computing (2004), http://hamilton.bell.ac.uk/btech/hci/hciintro.pdf 14. ISO/IEC 9126-4:2001, Software engineering — Product quality — Part 4: Quality –in-Use 15. Dumas, J.L., Redish, J.C.: A Practical Guide to Usability Testing, 3rd edn. Intellect, Wiltshire (1999)
An Information Visualization Approach to Hospital Shifts Scheduling Carmelo Ardito, Paolo Buono, Maria F. Costabile, Rosa Lanzilotti, and Adalberto L. Simeone Dipartimento di Informatica, University of Bari Via E. Orabona 4, Bari 70125, Italy {ardito,buono,costabile,lanzilotti,simeone}@di.uniba.it
Abstract. Scheduling staff shift work in a hospital ward is a well-known problem in the operation research field but, as such, it is very often studied from the algorithmic point of view and seldom from the human-computer interaction perspective. In most cases, the automatic solutions that operations research may provide do not satisfy the involved people. After discussing the inconveniences of an automatic approach with physicians, we have designed a staff scheduling system that combines an expert system with an information visualization (IV) system; in this way the schedule generated by the expert system is presented through the IV system to the schedule manager, who can modify the results if last minute changes are necessary, by directly manipulating the visualized data and obtaining immediate feedback about the changes made. Keywords: Information Visualization, Shift Scheduling.
can alternate appropriately. For these reasons, it is useful to employ a computer system able to store the history of work shifts for the last few years. However, fully automatic solutions cannot effectively cope with last minute changes: if someone gets sick, the overall schedule needs to be redefined. A mixed approach, combining algorithms that automatically compute a schedule with Information Visualization (IV) techniques capable of involving users to drive decision making process appears valuable. Visualizations can thus become the mean through which users and computers can cooperate to obtain efficient results with a semi-automatic process that leaves final decisions to users. This paper presents the development of a system composed by two modules: 1) an expert system, 2) an IV system. The first module automatically generates schedules, which are then visualized on the computer screen, so that users can interact and modify them. The user interface also allows users to specify some employees' constraints and preferences, which the expert system tries to satisfy. According to User-Centered Design methodologies [2], the system has been developed by taking into account the needs of final users, which have been involved in our design. We have collaborated with physicians of the Paediatric Hospital “Giovanni XXIII” in Bari, Italy. After an initial requirement gathering phase, some paper prototypes were devised and evaluated with the medical staff at the hospital. This led to several refinements of the prototypes that contributed to enhancing the usability of the system. The final system is the result of all the feedback received during the prior stages of the design phase. The paper has the following organization. Next section reports related work in this field. Section 3 describes how the system requirements have been gathered. Section 4 describes the developed scheduling system. Finally, Section 5 we report our conclusions.
2 Related Work Different works exist in literature that have explored the issue of hospital personnel scheduling. As shown in Burke et al. [3], most analyses focus on comparing different algorithmic solutions with the results obtained from other approaches and less on the day-to-day reality of hospitals. Another literature survey is provided in [4]. Other works study different aspects of the schedule generation process such as satisfying work rules, taking into account personal preferences, and performance in general. These systems are often developed for testing special purpose algorithms and not with the intent of being actually deployed in hospitals. In this work, the focus is on those studies who detailed experiences of complete systems being used in real world situations. Among these, Scheduler solves the generation problem algorithmically [5]: the application permits both total or partial schedule generation and manages unlimited work contracts. The schedules obtained with Scheduler can be visualized either as Gannt-diagrams, text or histograms, whereas our system offers different views, depending on the time granularity. Scheduler has been in use at several sites that could make use of its features: at Kolmårdens Djurpark, a Swedish zoo, users reported positively on its adoption, after an initial testing period. Very appreciated was the fulfilment of all work and law obligations, a requirement that is open to be much more error prone if worked on by humans, that
An Information Visualization Approach to Hospital Shifts Scheduling
441
may overlook or forget certain rules. In H&M Rowells’ call center, a mail order subsidiary of Hennes & Mauritz (a Swedish based clothes manufacturer and retailer), before the introduction of Scheduler, staff was used to get the breaks they wanted (with people they knew), while the optimised breaks generated by the application have maximised staff availability for high demand periods. Ghosh et al in their study [6] propose a computer aided system that supports managers in their work by computing optimal staff allocation based on a number of parameters. They report that the adoption of the system brings several benefits such as ensuring optimal nurse availability and simulate different staffing scenarios. The multitude of controls present in the interface disorients people not accustomed to the particular domain and would probably require some training before being able to use it with proficiency. From a human-computer interaction perspective, there is certainly room for improvement. The application presented by Beliën et al [7] enables managers to allocate the resources needed for the master surgery unit, through a graphical user interface. Both the schedule and the available resources are shown in the same screen. The schedule area is further subdivided by days per week and according to the various operating rooms. Schedules have to be built manually, by assigning each surgeon to the desired time slot. The application helps the schedule manager by instantly visualizing the impact of each choice: in this way conflicts can be more easily identified and solutions found. This system has been tested at the Belgian university hospital Gasthuisberg and, according to the authors, it was deemed to be very promising for the purpose of facilitating and improving the schedule generation process. The interface itself, again, could require some training to be used to its fullest. Among commercial systems, we examined Kappix’s DRoster [8] and PlanningPME [9] have been examined: while they both do not allow automatic generation of schedules, they offer several features that allow users to specify work shifts through an interface similar to calendar applications. In DRoster, an automated rule engine spots violations or inapplicable schedules. An interesting feature is the possibility of specifying “places” and authorize staff from this point of view. DRoster allows schedules to be viewed both in a daily and in a hourly resolution. PlanningPME visualizes schedules in several different resolutions: morning or afternoon shift and daily, weekly or monthly.
3 The Requirement Gathering Process With the aim of designing the scheduling system according to real users’ needs, we involved some physicians at the Pediatric Hospital “Giovanni XXIII” in Bari. They were happy to collaborate with us because they are open to new technologies. We have successfully collaborated with them also in the development of a mobile system for supporting physicians treating epilepsy [10] and a system supporting remote collaboration among physicians [11]. A field study with contextual interviews has been performed during the requirements analysis to better understand the medical domain, how the physicians operate and the main features of the application to be implemented. User observation methods and principles of participatory design have proven to be effective in user interface
442
C. Ardito et al.
design. The initial investigation was carried out by HCI researchers who performed contextual analysis at the hospital. They observed how physicians define the schedule and discussed the current process with them through interviews and focus groups. The familiar workplace helped the interviewed to recall certain work anecdotes that could have otherwise been neglected should the interview had taken place in another context. We discovered how the scheduling process was handled: despite technological advancement, schedules were still built manually via several paper forms, during a 30-day period. The different agreements that have to be made in order to guarantee the fulfilment of preferences and special requirements are the results of personal negotiations between the interested parties and the schedule manager. All the material was collected for later examination and for a preliminary description of the interaction possibilities that would have had to be managed. This material was used to formulate interaction scenarios that could depict the most frequent tasks users should be able to perform. The requirement gathering phase highlighted that the system to develop is very complex from the functional viewpoint. In fact the resulting schedules have to satisfy several constraints, some of which related to regulations, others to the history of previous schedules and others to individual physicians’ needs or preferences. For this reason, it was decided to rely on an expert system for the management of the schedule generation process, but leaving users the possibility of making changes, should special occasions arise that would make certain staff members unexpectedly unavailable; for instance, it could obviously happen that a physician cannot get to work for some days because s/he must attend a professional meeting or because s/he might be ill; it is thus necessary to insert this information and have the system recompute schedules. Therefore, it was necessary to design an intuitive interface that provided a) an overview representation of the overall situation and b) the possibility of interacting directly with visualized data.
4 The Scheduling System The scheduling main components are: the expert system module and the visualization module. The former is responsible of analyzing the facts provided in input by the schedule manager (in this paper we assume to be a male physician) and computing a solution that tries to satisfy the constraints. The latter visualizes the computed schedules and allows the operator to modify them. These modules are described in detail in the following sections. 4.1 The Expert System Module The expert system is developed in Java and JClips [12] and uses XML for data interoperability. It exploits one of the most commonly used techniques to develop expert systems, namely rule-based programming. With this paradigm, a set of rules (which represent heuristics) indicate how the system should behave in a variety of situations. Each rule is composed by an “if” and “then” portion. The former contains a series of patterns (or facts) that, if satisfied, will cause the rule to be applicable. The latter represents a set of actions that will be executed when the rule is applicable.
An Information Visualization Approach to Hospital Shifts Scheduling
443
To begin the scheduling process, the system must know some data that are inserted by the user such as: start and end date of the period to be scheduled, physician’s days off, etc. Other data that is needed is those which is computed internally by the system and is related to the amount of duty physicians and working and festive availabilities assigned to physicians in the Operative Unit. The insertion of facts in the knowledge base allows the expert system to activate the rules that will produce the final state, and hence the full schedules for the considered period of time. The expert system models these constraints that emerged during the requirement analysis phase, e.g.: • • • •
A physician can work only one shift per day. A physician assigned the night shift must not be assigned further shifts on the next day and the day after that. A physician who worked on a particular bank holiday (e.g.: New Year’s Eve) will not be expected to work on the same day of the following year. Some physicians may be exonerated from working night shifts
Among the different strategies that can be used to compute the schedules within CLIPS, our module uses the Simplicity and Random ones, that in the performed experiments produce better results in the distribution of physicians in the resulting schedule. Simplicity means that a rule is chosen depending on the number of facts necessary for it to be satisfied: less is better. Random, as the name implies, means that rules with the same salience value will be chosen randomly. The execution of a rule may change the list of applicable rules by adding or removing facts. Therefore, after a rule is executed, the environment selects the next applicable rule, until no more remain. 4.2 The Information Visualization Module The information visualization module is the result of a series of refinements which contributed to its overall improvement. This was obtained through various iterations of paper prototypes. Developing paper prototypes before any implementation phase is fundamental in the software life-cycle. Through paper prototypes it is possible for final users to experience the interface before it is actually implemented. Should problems arise, they can be fixed and the interface modified in a quick and cheap way, according to the user’s feedback and criticism. For our system, we built several paper prototypes before getting to the final interface. Each prototype was discussed with the physicians at the hospital in order to gain their feedback about our proposals. One of the main improvements was the introduction of a data entry interface (see Fig. 1.) that facilitated physicians in entering personal information and adjusting their preferences and requirements for the schedule generation. For example, in order to specify working preferences, the schedule manager simply has to click on the corresponding colored buttons. The feedback of this action is immediately observable visually in the interface. From the interviews conducted after examining paper prototypes, we were able to gather some interesting insights: physicians appreciated the feature which let them have an overview of schedules (i.e., yearly) rather than shorter time periods (i.e., monthly or weekly).
444
C. Ardito et al.
4.3 A Walkthrough in the System In this section we describe a walkthrough of the information visualization module in order to describe how the schedule manager inserts data for starting the schedule process (data-entry interface), and how he interacts with the scheduling interface. Data-entry interface As discussed earlier, the scheduling process starts by requiring some data from the schedule manager. The system shows the interface in Fig. 1. The user indicates which period he may want to generate schedules for. By using the two colored bars (referred to as “Preferenze Generali”, Italian for “General Preferences”) in the upper part of the interface, the operator may insert the preferences of each physician. General preferences concern which days of the week (and in which shifts) a physician would like to work or those to avoid working. The operator may assign different preference levels on each item by clicking the respective colored button. For example, for each day the operator can select the green button for expressing the best preference, orange for neutral, or violet to express that the physician dislikes to work in that day. Similarly, he can indicate the preferred shift among morning, evening and night. In Fig. 1 the physician, whose name is highlighted in the leftmost column, would like to work on Wednesday, Friday and Saturday (green colored), avoiding Tuesday, Thursday and Sunday (violet colored); about the shifts, he would like to work in the afternoon. The bottom of the screen (referred to as “Preferenze Dettagliate”, Italian for “Detailed Preferences”) allows the operator to assign to specific days preferences about the
Fig. 1. The data entry interface
An Information Visualization Approach to Hospital Shifts Scheduling
445
desired shift or time off. For example, the operator can specify that a physician would like to have a week off in this month. After having inserted the desired information about general and/or detailed preferences for the physicians to be considered, the schedule generation process may begin. Scheduling interface The monthly scheduling, computed by the expert system module, is visualized in Fig. 2. It allows the schedule manager to compare each physician’s schedule to those of colleagues of the same Operative Unit. The example shown in Fig. 2. refers to the October 2008 schedule. Each row refers to one of seven physicians belonging to the Operative Unit, while the columns represent days. By using colors and labels we are able to codify different activities the physician has to carry out during each day. For example, on October the 30th, the light blue icon with an “A” inside indicates that the first physician, whose name is Arcangelo Clemente, must work in the surgery unit of the hospital, while the fifth physician, Anna Mele, must work during the evening shift, as shown by the blue icon. In the monthly visualization, physicians are not interested in seeing the schedules belonging to physicians outside their unit in this interface; instead, they prefer to reserve this feature for the yearly overview, where the comparison between different Operative Units was felt to be important. In the yearly visualization, as it can be seen in Fig. 3, there is a row for each month and they are aligned depending on the first day of the month. In the shown example, August (abbreviated AGO, for Agosto, in Italian) starts on Tuesday (abbreviated MA, for Martedì, in Italian) while September (SET) starts on Friday (GIO) and so on. Each row is further divided in as many columns as there are days in that particular month. Each day is then divided in three further rows which represent the three different shifts there are in a given day: morning 8 a.m. – 2 p.m., evening 2 p.m. – 8 p.m., night
Fig. 2. Monthly overview
446
C. Ardito et al.
8 p.m. – 8 a.m.. Thanks to this visualization, holidays can be immediately spotted. In fact, physicians strongly stressed the need to highlight holidays and holiday eves. From our interviews it came to light the need of introducing the so called “totalizers”, that is, counters which represent the sum of the total amount of shifts performed by each operative unit. These are displayed as an optional sliding panel on the right, which displays the totals both as a number and as a proportional bar. If the schedule manager is not satisfied with the computed output, or needs to adjust some last minute details of the schedule, he may customize the results. For example a problem has occurred with the first physician: he cannot work on Wednesday. The operator chooses him and selects “Modify” from the menu. The data entry interface will be displayed showing the current schedule for the chosen physician. The operator can now insert the new preference (i.e.: avoid working on Wednesday) and the overall schedule will be recomputed to satisfy the updated situation.
Fig. 3. Yearly schedules overview
5 Conclusions In this paper we presented a system which supports staff shift scheduling. It is a challenging activity, because several aspects must be considered and it is even harder to satisfy all the constraints. Algorithms that automatically generate such schedules do not satisfy people in most cases, especially when last minute modifications are necessary, as it is often the case in hospitals. It is useful to employ a computer system able to generate schedules. The approach adopted in this paper combines an expert system
An Information Visualization Approach to Hospital Shifts Scheduling
447
that generates the schedules on the basis of rules specific for the hospital with Information Visualization techniques in order to give users the possibility to drive the decision making process and possibly perform the necessary adjustments. Since the early stages of system design, we have involved physicians of a paediatric hospital. Thanks to the feedback received during the various steps of the development, including the revisions of paper prototypes, the system resulted being usable by the schedule managers. Among the system features, the interactive overview was much appreciated because it represented an innovative idea, since users were accustomed to working with paper schedules. The rapidity with which users are able to deal with several unexpected situations gives the scheduling system an edge over old-fashioned manual approaches. These advantages come from the adopted visualization techniques, which enable users to quickly understand the status of a particular schedule, just by looking at the visualization. Acknowledgments. This work was partially supported by University of Bari (“ex60%” funding) and by EU and Regione Puglia under grant “DIPIS”. We like to thank Ass. Prof. Stefano Ferilli for his valuable collaboration and the students of the Computer Science curriculum who participated to the development of the system.
References 1. Aiken, L.A., Clarke, S.P., Sloane, D.M., Sochalski, J.A., Silber, J.H.: Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. J. of the American Medical Association 288(16), 1987–1993 (2002) 2. ISO 13407, Human-centered design process for interactive systems. International Organization for Standardization (1998) 3. Burke, E.K., De Causmaecker, P., Van Den Berghe, G., Van Landeghem, H.: The State of the Art of Nurse Rostering. J. of Scheduling 7(6), 441–499 (2004) 4. Ernst, A.T., Jiang, H., Krishnamoorthy, M., Sier, D.: Staff scheduling and rostering: A review of applications, methods and models. European J. of Operational Research 153(1), 3–27 (2004) 5. Eveborn, P., Rönnqvist, M.: Scheduler – A System for Staff Planning. Annals of Operations Research 128(1), 21–45 (2004) 6. Ghosh, B., Cruz, G.: Nurse requirement planning: a computer-based model. J. of Nursing Management 13(4), 363–370 (2005) 7. Beliën, J., Demeulemeester, E., Cardoen, B.: Visualizing the Demand for Various Resources as a Function of the Master Surgery Schedule: A Case Study. J. of Medical Systems 30(5), 343–350 (2006) 8. DRoster, http://www.kappix.com/ 9. Planning PME, http://www.planningpme.com/ 10. Ardito, C., Buono, P., Costabile, M., Lanzilotti, R.: Two Different Interfaces to Visualize Patient Histories on a PDA. In: MobileHCI 2006, Espoo, Finland, pp. 37–40. ACM Press, New York (2006) 11. Costabile, M.F., Fogli, D., Lanzilotti, R., Mussio, P., Piccinno, A.: Supporting Work Practice through End-User Development Environments. J. of Organizational and End User Computing 18(4), 43–65 (2006) 12. JClips, http://sourceforge.net/projects/jclips/
Abstract. This paper uncovers issues in the design of camera-based technologies to support retail shopping in a physical store, specifically clothes shopping. An emerging class of technology is targeting the enhancement of retail shopping, including the trying on of clothing. Designing such systems requires careful considerations of physical and electronic design, as well as concerns about user privacy. We explore the entire design cycle using a technology concept called the Responsive Mirror through its conception, prototyping and evaluation. The Responsive Mirror is an implicitly controlled video technology for clothes fitting rooms that allows a shopper to directly compare a currently worn garment with images from the previously worn garment. The orientation of images from past trials is matched to the shopper’s pose as he moves. To explore the tension between privacy and publicity, the system also allows comparison to clothes that other people in the shoppers’ social network are wearing. A user study elicited a number of design tradeoffs regarding privacy, adoption, benefits to shoppers and merchants and user behaviors in fitting rooms. Keywords: Ubiquitous computing, pervasive, ambient intelligence, retail technologies, privacy, online social networks, fashion.
technologies are providing new capabilities to supplement the physical shopping experience. In addition to displaying conventional product information found on the web, these technologies also augment a human’s assessment of the more subjective, perceptual properties by capturing the shopper’s experience for use as a memory aid and as an artifact for sharing with friends and family. This paper explores the design considerations for creating interactive technologies for clothes fitting rooms using the design and evaluation of a prototype technology called the Responsive Mirror (Figure 1) [35, 36, 37], which is an implicitly controlled video technology for clothes fitting rooms that allows a shopper to directly compare a currently worn garment with images from the previously worn garment. The orientation of images from past trials is matched to the shopper’s pose as he moves. To explore the tension between privacy and publicity, the system also allows comparison to clothes that other people in the shoppers’ social network are wearing.
Fig. 1. The Responsive Mirror concept
2 Related Work Clothing stores often provide an area for trying on clothes. Generally, there is a private place in which to change clothing, which we refer to as a changing area, and an area with a mirror to view the fit, which we refer to as a fitting area. In this paper, we consider these areas separately, but they are sometimes combined in one place. The fitting-area mirror is a specific point of customer interaction with products where a shopper selects items to purchase. Marketers have recognized this “point of decision” as an opportunity to engage the customer with supplemental information.
450
B. Begole et al.
2.1 Clothes-Fitting Technologies Technologies in Fitting Rooms: A Prada boutique in New York contains a sophisticated dressing room designed and constructed by Ideo and IconNicholson [16]. A scanner identifies each garment as the shopper takes it in, providing additional information about the garment’s price and alternate colors and sizes – the kind of information shoppers can find when browsing products online. The fitting room also contains a motion-triggered video camera that records the shopper and plays back the video after a pause. One of the central components of the system is the Magic Mirror which allows a person trying on clothes to send video of himself to friends who can send back comments and vote (thumbs up/down) [21]. The Magic Mirror can also project a static image of an alternate garment onto the mirror allowing the person to see roughly how it might look on him. The Magic Mirror has also been deployed in Bloomingdales. Similar systems have been created and deployed for market evaluation in a number of other retailers [34]. The Gardeur Shop in Essen Germany created an RFID-based system that provides product information including price, available sizes and colors, material and care instructions. Warnaco created a system to show fitting information for women’s braziers. Poggi reports other retailers employing non-sensor technologies in fitting rooms, such as Metropark, Nordstrom, Kira Plastinina and Macy’s. Virtual Fittings: Some online services, such as Intellifit [14] and MyShape [19], take measurements of a customer’s body and suggest clothes that will fit and provide a flattering appearance for that body shape. These services have been deployed in stores such as Levi’s and Charming Shoppes in Ohio, Florida and Texas [34]. Other web services such as MyVirtualModel [20], Knicker Picker [17], and 3Dshopping [1] provide a set of predefined body types that allow a shopper to view the fit of clothes on their selected body type. It is also possible to use augmented reality techniques to map an image of a garment onto a captured image of the customer that will change shape in real time as the customer moves in a virtual mirror [8]. In essence, these virtual fitting technologies are comparable to seeing clothing on a personalized mannequin. However, just as one needs to try on clothes seen on a mannequin, a customer must try the garment on his own body to get the full experience of feel, drape and look. Although virtual fitting technologies continue to advance, Protopsaltou et al. report that reliance on such technology often results in unexpected fit or color, and unpleasant feel of the fabric [25]. Such technologies also do not cover the full range of ways in which a person can wear a garment. A buttoned down shirt, for example, can be worn buttoned or unbuttoned, tight or baggy, tucked or untucked, tied at the bottom, and sleeves rolled to different lengths. Exploring this variety of options ultimately requires physical control of the clothes. 2.2 Reactive Displays The ability to change information in reaction to the presence and motion of people has been demonstrated in several prior systems. Haritaoglu and Flickner describe a prototype that uses computer vision to count the people looking at a display and to infer demographic information [13]. There are some commercial advertising technologies
Designed to Fit: Challenges of Interaction Design
451
(Reactrix [26], Freeset Human Locator [10], P.O.P. ShelfAds [24]) that detect presence and motion of people and change the content of the display in reaction. These reactive displays do not display images that mirror the image or motion of the person looking at the display as the Responsive Mirror does. 2.3 Privacy Concerns from Cameras in a Fitting Room A common concern that sensor technologies face is how they affect a user’s sense of personal privacy. Will users accept a camera or other sensors into a traditionally semiprivate space such as a fitting area? What concerns would they have and what measures should the system design incorporate to mitigate such concerns? In the press articles, privacy concerns are sometimes raised but rarely explored in depth. People are subjected to video capture on a daily basis, sometimes happily (home movies, video conferencing, camera-based game controllers) and sometimes without much choice (store surveillance, toll collection, traffic cameras). Such applications have achieved at least some level of tolerance, yet debate continues about the overall effects of sensing technologies on the long-term health of our society. Grudin [12] observes that collecting and distributing such data electronically heightens concerns of being susceptible to undesirable uses. A basic framework of feedback and control of image capture was outlined by Bellotti and Sellen [5], which systems such as the Responsive Mirror commonly employ. Nevertheless, when a camera or other contextsensing technology is introduced into an unexpected situation, such as a fitting room, we need to examine the potential impact on privacy. Palen and Dourish [22], synthesizing Irwin Altman’s research of face-to-face interaction [2], describe the boundaries and tradeoffs between publicity and privacy that people navigate under technology-mediated interactions. Systems such as the Responsive Mirror introduce new capabilities that affect people’s expectations under the three boundaries of their framework, described next. Also relevant is the work of Petronio [23] who examines a multitude of publicity/privacy boundaries and the varying permeability of the boundaries. We describe below the potential encroachments across these boundaries that camera technologies create. Disclosure: People naturally have an expectation that a fitting room is a private place where they can change clothes without being seen. Although the Responsive Mirror cameras do not capture images of people changing clothes in the changing area itself, only in the fitting area outside, an individual may still have some concern of undesired disclosure of potentially embarrassing interactions in the mirror. In addition to the awkward poses that a shopper may take to test the fit of clothing, a shopper naturally expects that she can experiment with clothes styles in which she might not like to be seen for any number of reasons: fit, style, status, expressiveness, etc. She may be troubled if the images were seen by other people. On the other hand, a shopper interacting with Responsive Mirror may not desire that he fittings be completely undisclosed. She may want to show clothes to friends, or publish them to a social fashion network. Identity: The Responsive Mirror presents images of the shopper as well as others while making decisions about clothing that will become part of his presentation of self. The ability to see images of others provides a means for a person to test the fit of
452
B. Begole et al.
his presentation of self among the presentations of a community. He can examine styles that others are wearing and choose where he would like to fit. On the other hand, he may not desire to be seen as actively positioning himself along such dimensions, perhaps preferring to present a nonchalant attitude about fashions. Thus, a record of his examinations of the images of other people could be sensitive. Temporal: Shoppers today expect that an experimental fitting is transient, not lasting for others to see at another time. In addition to the obvious concern that a shopper might have about recoding images in an embarrassing outfit, images in flattering outfits could become a concern in the future as style and tastes change. While a flattering garment may be an excellent choice at the moment, it could come to represent something contrary to the shopper’s style in the future. This concern can be mitigated by providing the ability to delete regrettable images, but the damage cannot be wholly undone as the images may have been stored in an Internet archive. Furthermore, simple deletion is possibly more drastic than people would desire. The temporal boundary is one that has a varying permeability over time and people might prefer simply that it become more difficult, but not impossible, to see older images. Social Aspects of Fashion Decisions: Another under-explored consideration for such technologies is the social aspects of shopping and fashion. Only the Magic Mirror begins to provide support for social interactions by employing mechanisms for synchronous image sharing and voting by remote friends or family. These capabilities are only the tip of the iceberg regarding social dynamics of shopping and clothing. What kind of social impressions are shoppers concerned about? How do such systems support exploration of the “language” of clothing? When buying clothes, shoppers often assess how others would perceive a particular item, that is, the “fashion statement” she is making. Indeed, people often ask companions or store assistants how they look to get this kind of feedback. Although evaluating one’s self image is an important part of clothes-buying decisions, shoppers in a store are provided very little information about what other people are wearing, other than what is presented in advertisements. Outside of the store, people find inspiration from others and from fashion media, some of which today contain photographs of amateur people. Sociologists have long explored the multiple roles that fashion plays in society. In a synthesis of more than 200 sources, Davis [7] examines the social construction of meaning in fashion and how an individual’s choices communicate social status, gender identity, sexuality, and conformity among other characteristics. Clothing is undoubtedly a form of communication, as declared famously by Lurie [16] and examined in depth by Barnard [4], but Davis believes it is a code of “low semanticity” because the meanings differ with time, trends and among different individuals. Although technologies cannot interpret fashion, they can mediate human discourse on the topic. We see this happening today in online social fashion networks, such as ShareYourLook [28] and IQONS [11, 15]. These sites allow members to upload photographs of themselves wearing various outfits and they can categorize, tag and comment on the images. The services merely provide scaffolding; the members construct the semantics through tagging and commentary. Providing access to a shopper’s social fashion network during a shopping experience would provide access to the state of fashion as constructed by the shopper and her online community.
Designed to Fit: Challenges of Interaction Design
453
3 Formative Evaluation of Responsive Mirror Our prototype was designed to test assumptions about the usefulness of the new capabilities and to draw out lessons for future design. First, will shoppers benefit from being able to directly compare their clothing alternatives in a physical shopping experience? Second, will shoppers find value from images of other people wearing similar and different clothes? We also wanted to learn which, if either, feature would shoppers prefer, what uses they would make of them and what other capabilities they would desire. We considered conducting field observations in an actual store’s fitting room but determined that before doing so we needed to gain some understanding of the privacy implications of the camera. Therefore, in this initial study, we observed people’s behavior using the prototype in our lab. We recruited 12 male participants who wear medium to large men’s shirt size in the age range of 28-52 years old. We were only able to use male participants because the prototype’s body segmentation and clothes detection algorithms are initially targeted for men’s shirts because there is less variance among men’s shirt varieties than among women’s. We recognize that men are only a sub-population of the eventual user population but testing on this demographic allowed faster iteration of the technology-design cycle. A limitation of this study is that it used participants of only one gender (male). Thus, it is not clear to what extent the same phenomena would be observed with participants of the other gender (female), particularly regarding questions of privacy and social effects. The experiment consisted of three conditions. 1. Mirror: Conventional mirror alone. 2. Previous Outfit: Conventional mirror with a display on the left showing an image of the person from the most recent previous fitting. The orientation of the person in the image matches the orientation of the person in realtime. 3. Other People: Conventional mirror with a display on the right showing four images of other people. For each condition, participants put on six different shirts consisting of three collared polo-style shirts and three crew neck shirts of a mix of colors. The price of the shirt was removed. There were three sets of six shirts and there was a comparable mix of colors and patterns across the sets. The order of conditions and shirt sets was counter-balanced using a latin-square three-by-three design. 3.1 Task and Procedure Participants were told to imagine themselves in a scenario where they need to decide which, if any, of the six shirts they will buy. To eliminate price as a factor in their decision, participants were told to imagine that they can afford as many shirts as they like and would consider the prices to be reasonable in all cases. For each shirt, the participant stood approximately 250 cm (8.2 ft) in front of a fulllength mirror (165cm (65 inches) high by 73cm (29.5 inches) wide). Participants were asked to view the shirt from multiple angles in a natural way. After trying all six shirts per condition, the participants were asked to fill out a questionnaire measuring how appealing they found the shirt and how likely they are to buy the shirt on a 5 point
454
B. Begole et al.
scale. They were also asked questions about the condition they had just experienced. After all three conditions were complete, the participants filled out a questionnaire with questions spanning the conditions and potential privacy issues. 3.2 Results To examine whether the display conditions affected participants’ perception of the appeal or desire to purchase any of the shirts, we conducted two repeated-measures ANOVAs with Display Condition as the within-subject variable, and the appeal and purchase variables as dependent variables in turn. Neither the ratings of the desire to purchase nor appeal of the shirts were significantly different among the conditions (respectively F[2, 110] = .42, p = .66 and F[2, 110] = .32, p = .73). We asked participants to rank their preference for the three conditions plus a configuration using all three components (which they did not experience). To test whether participants ranked these four configurations differently, we performed a Friedman test on the ranked values within subjects. There was a significant difference in the rankings (χ2 = 9.10, p = .03), where the preference (from strongest to weakest) were: Previous Outfit plus Other People (M = 1.92) > Previous Outfit (M = 2.00) > Other People (M = 2.83) > Plain Mirror (M = 3.25). And finally, we asked participants how much the Previous Outfit and Other People conditions enhanced their shopping experience. Participants rated the Previous Outfit (M = 3.00, SD = .85, where 5 = Extremely Helpful and 1 = Not Helpful At All) as having enhanced their experience more than the Other People (M = 2.50, SD = .52) condition, which is in line with the ranked measures. A paired-samples t-test showed the ratings to be significantly different between the conditions (t[11] = -2.57, p = .03). 3.3 Results from Privacy Related Questions Regarding the issues of disclosure, we asked participants how much it would bother them if someone from same or opposite gender and specific social group (family, friends, co-workers, and stranger) viewed images of them that were captured by the system. They specified their response on a five point scale (5=bothers me a great deal, 1=doesn’t bother me at all). The responses were not significantly different according to the gender of who might see the image, so we calculated the mean for each social group across gender. The means were Family = 1.08 (SD =.19), Friends = 1.50 (SD = .71), Stranger = 2.08 (SD = 1.06) and Co-worker = 2.25 (SD = 1.14). We performed a repeated-measures ANOVA and found there was a significant difference among the social groups (F[3,33] = 5.76, p = .003). Post-hoc contrasts revealed that Family is not significantly different from Friends. and that CoWorkers is not significantly different from Strangers. Family and Friends are significantly different from CoWorkers and Strangers (p = .01). The frequency distributions for all groups have a mode of 1 (doesn’t bother me at all) except Coworkers which is bimodal with modes at 1 (doesn’t bother me at all) and 3 (bothers me somewhat). We asked participants how much it would bother them on the same 5 point scale if other people saw images of them wearing a shirt that looks good versus a shirt that looks bad on them. Participants rated their level of concern significantly higher for bad shirts (M=3.0) versus good shirts (M=1.42) (p = .001).
Designed to Fit: Challenges of Interaction Design
455
Regarding the aspects of personal and group identity, we asked participants how often (5=Always, Often, Sometimes, Seldom, Never=1) they think of someone else who might like the clothes they are trying on. The mean response was 2.67 (SD=0.98). We asked participants how often they think about how similar the clothes are to what other people they know and don’t know are wearing. The mean for people they know was 2.92 (SD=0.9) and for people they don’t know was 2.33 (SD=0.98) with no significant difference between them. Participants responded with a mean of 3.6 (SD=1.07) to the question of how often do they consciously consider how others will perceive them in the clothes they are trying. Regarding the issues of temporality, we asked participants if their tastes changed in the future, would they want to remove images of themselves wearing a contrary style. On a five point scale (5=Definitely, 1=Definitely Not), participants responded with a mean of 3.08 (SD=1.16) (closest to Possibly). We asked participants at what period of time would they consider removing images of themselves that they had allowed others to see. The highest number of responses was for 3 months (5 participants) and the distribution of the remaining responses was spread across times within 1 year (5 participants) and never (2 participants). These responses can guide the points in time at which to remind users of the existence of past images.
4 Lessons for Design Our experience with the Responsive Mirror uncovered some unexpected issues in the design of user experience in this domain. Because of the error rates of sensing technologies, designers need to anticipate variance in the user experience. Our prototype used a straightforward color-histogram comparison from a relatively small data set and yet we found that sometimes the system retrieved different matches for different participants wearing the same clothes. Although this inconsistency was not noticed by the individual participants, designers need to be aware of potential variance in system behavior and avoid designs that expect a deterministic result. Designers of pervasive technology must be aware of the ways that information systems such as these are affected by our notions of personal privacy. We already see that the increasing volumes of social content creation, sharing and retrieving are modifying societal norms. Designers of retail systems should shape the system to provide benefits that balance, or potentially outweigh, the risks to privacy in terms of disclosure, identity and temporal boundaries. Our experience indicates that some users are receptive to a system that bundles the capture of potentially sensitive images with the benefit of seeing images of self and others. Perhaps most important to merchants, our results showed no difference in participants’ perceptions of their likelihood to buy a shirt after using our prototype. The quality of the product matters more than the technologies used to evaluate it. Nonetheless, participants did perceive the system to have been “helpful” to their buying decision, providing value that merchants should be able to capitalize on. Pervasive technologies such as the Responsive Mirror change the clothes-buying experience by connecting information between our physical lives and online digital identities. Currently these realms are artificially separated in a false dichotomy of
456
B. Begole et al.
“virtual” versus “real” life. Context-sensing technologies enable the design of applications such as Responsive Mirror that not only augment the immediate physical experience but also integrate information across the multiple aspects of one’s social life. As we see increasing spread of such technologies, designers need to be informed of the unique considerations that must be made in these environments, particularly with respect to privacy and social goals.
References 1. 3DShopping.com, http://www.3dshopping.com 2. Altman, I.: The Environment and Social Behavior: Privacy, Personal Space, Territory and Crowding. Brooks/Cole Pub. Co., Inc., Monterey (1975) 3. Anguelov, D., Lee, K., Gokturk, S.B., Sumengen, B.: Contextual identity recognition in personal photo albums. Computer Vision and Pattern Recognition (CVPR) (2007) 4. Barnard, M.: Fashion as Communication, 2nd edn. Routledge (2002) 5. Bellotti, V., Sellen, A.: Design for privacy in ubiquitous computing environments. In: Proceedings of the European Conference on Computer-Supported Cooperative Work, ECSCW, pp. 77–92. Kluwer Academic Publishers, Norwell (1993) 6. Brown, J.: Prada Gets Personal, BusinessWeek. The McGraw-Hill Companies, March 18 (2002), http://www.businessweek.com/magazine/content/02_11/ b3774612.htm 7. Davis, F.: Fashion, Culture and Identity. University Of Chicago Press (1994) 8. Ehara, J., Saito, H.: Texture overlay onto deformable surface for virtual clothing. In: Proceedings of the 2005 international Conference on Augmented Tele-Existence ICAT 2005, vol. 157, pp. 172–179. ACM Press, New York (2005) 9. Ehrlich, S.F.: Strategies for encouraging successful adoption of office communication systems. ACM Trans. Inf. Syst. 5(4), 340–357 (1987), http://doi.acm.org/10.1145/42196.42198 10. Freeset Human Locator, http://www.freeset.ca/locator/ 11. Geller, M.: Friendly fashion — Cyber-style!, Reuters (April 28, 2007), http://blogs.reuters.com/2007/08/28/ friendly-fashion-cyber-style/ (Last accessed January 29, 2009) 12. Grudin, J.: Desituating Action: Digital representation of Context. Human-Computer Interaction 16(2-4), 269–286 (2001) 13. Haritaoglu, I., Flickner, M.: Attentive billboards. In: Proceedings of the Conference on Image Analysis and Processing 2001, September 26-28, 2001, pp. 162–167. IEEE Press, Los Alamitos (2001) 14. Hilsmann, A., Eisert, P.: Deformable object tracking using optical flow constraints. In: 4th European Conf. on Visual Media Production, 2007, November 27-28, 2007. IETCVMP, pp. 1–8 (2007) 15. Intellifit, http://www.intellifit.com/Intellifit/Home.aspx 16. IQONS, http://www.iqons.com/ 17. KnickerPicker.com, http://www.knickerpicker.com/ 18. Lurie, A.: The Language of Clothes, Random House (1981) 19. Like.com, http://www.like.com/ (Last accessed September 8, 2007) 20. Lou, H., Luo, W., Strong, D.: Perceived critical mass effect on groupware acceptance. Eur. J. Inf. Syst. 9, 91–103 (2000)
Designed to Fit: Challenges of Interaction Design
457
21. MyShape, http://www.myshape.com/ 22. My Virtual Model, http://www.mvm.com 23. Nanda, S.: Virtual Mirrors. Reuters, January 29 (2007), http://www.reuters.com/news/video/videoStory?videoId=5219 24. Palen, L., Dourish, P.: Unpacking "privacy" for a networked world. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI 2003, pp. 129–136. ACM Press, New York (2003), http://doi.acm.org/10.1145/642611.642635 25. Petronio, S.: Boundaries of Privacy: Dialectics of Disclosure. State University of New York Press, Albany (2002) 26. P.O.P. ShelfAds, http://www.popbroadcasting.com/main/intilashelf. html 27. Protopsaltou, D., Luible, C., Arevalo, M., Magnenat-Thalmann, N.: A body and garment creation method for an Internet based virtual fitting room. In: Computer Graphics International 2002 Conference Proceedings, pp. 105–122. Springer, Heidelberg (2002) 28. Reactrix, http://www.reactrix.com/advertisers_details.php?id=9 29. RockYou, http://www.rockyou.com/ (Last accessed September 8, 2007) 30. ShareYourLook, http://www.shareyourlook.com/ (Last accessed September 3, 2007) 31. Song, Y., Leung, T.: Context-aided human recognition – clustering. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 382–395. Springer, Heidelberg (2006) 32. Weiser, M.: The computer for the 21st century. Scientific American 265, 94–104 33. Staedter, T.: Smart’ Fitting Room Suggests What to Wear. Discovery News, January 23 (2008) 34. Poggi, J.: Dressing Rooms Of The Future, Forbes.com, July 22 (2008) 35. Chu, M., Begole, B.: Human-Centric Interfaces for Ambient Intelligence. In: Aghajan, H., Delgado, R.L.-C., Augusto, J.C. (eds.) Natural and implicit information seeking cues (to appear) 36. Zhang, W., Matsumoto, T., Liu, J., Chu, M., Begole, B.: An intelligent fitting room using multi-camera perception. In: Proceedings of the 13th international Conference on intelligent User interfaces (IUI 2008), pp. 60–69. ACM, New York (2008) 37. Zhang, W., Begole, B., Chu, M., Liu, J., Yee, N.: Real-Time Clothes Comparison Based on Multi-View Vision. In: Proceedings of ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC 2008), September 7-11 (2008)
Usability for Poll Workers: A Voting System Usability Test Protocol Dana Chisnell1, Karen Bachmann2, Sharon Laskowski1,3, and Svetlana Lowry3 1
UsabilityWorks, 453A Chestnut St., San Francisco, CA 94133 [email protected] 2 Seascape Consulting, Inc. 13911 W Hillsborough Ave, Tampa, FL 33635 [email protected] 3 National Institute of Standards and Technology 100 Bureau Drive, Gaithersburg, MD 20899 sharon.laskowski, [email protected]
Abstract. In this paper, we discuss our efforts to develop a repeatable test protocol for assessing usability for poll workers—temporary election officials who ensure secure and private voting in voting places. The research described in this paper is part of a larger effort to develop a standard for voting systems. This is the first time that formal and substantial usability requirements as part of a standard for voting systems have been established in the United States. The standard includes requirements for poll worker usability and associated test methods to assess whether a system meets these requirements. The test method described in this paper sets up a protocol and pass/fail criteria for assessing the usability of voting system documentation for poll workers. Keywords: usability testing, pass/fail criteria, elections.
1 Background While much research on elections has concentrated on voting and the voter experience, usability for poll workers is equally important. Poll workers spend extremely long hours for very little pay on Election Day setting up voting stations, making sure the equipment is operating properly, assisting voters, and shutting down the voting stations when the polls close. Along the way, they often encounter problems with voting systems that delay or even prevent voters from voting or cause votes to go uncounted. 1
This document describes research in support of test methods and materials for the Election Assistance Commission's next iteration of the Voluntary Voting System Guidelines (VVSG). It does not represent a consensus view or recommendation from National Institute of Standards and Technology (NIST), nor does it represent any policy positions of NIST. Certain commercial entities, equipment, or material may be identified in the document to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that these entities, materials, or equipment are necessarily the best available for the purpose.
Usability for Poll Workers: A Voting System Usability Test Protocol
459
The purpose of the research described in this paper is to explore how to perform usability testing to ensure that voting systems achieve an acceptable level of usability for poll workers, meaning that they can complete typical poll worker tasks by following instructions in voting system documentation. This work will form the basis of a repeatable test protocol to be employed for uniform testing of voting systems. Other research is being conducted to learn about usability for voters. In the United States, under the auspices of the U. S. Election Assistance Commission (EAC), a standard for voting systems has been developed called the Voluntary Voting System Guidelines (VVSG)2. The National Institute of Standards and Technology (NIST) is currently developing test methods for determining if a voting system complies with the requirements in the VVSG. A significant set of usability requirements is part of the VVSG and, in particular, there are requirements for usability for poll workers. Test laboratories have been designated to test voting systems to determine if they meet the standard and, if so, the EAC certifies the systems. Elections are run by the States. The States are encouraged to purchase systems that have been certified. For more information about the EAC, see http://www.eac.gov. For more information about NIST’s work related to voting, see http://vote.nist.gov. The VVSG can be found at http://vote.nist.gov/vvsg-report.htm. It is undergoing a public review process; its current status at http://www.eac.gov. This is the first time that formal and substantial usability requirements as part of a standard for voting systems have been created in the United States. It is critical that the test methods for determining conformance to the standard include usability testing in addition to checking for adherence to design requirements through review. It is important to confirm that poll worker interaction with the system is effective. (There is other research looking at voter interaction.) This can only be judged by observing and measuring the voting system interface performance with actual users performing typical election tasks. Further, because of the certification process, the test methods employed by the test laboratories must each have a pass/fail criterion: a voting system is certified if it meets all the VVSG requirements by passing all the associated test methods. The goal of the research described in this paper is to establish a test protocol to evaluate usability for poll workers. The challenges are that the protocol must y y y y y
apply to the testing of any voting system include clear pass/fail criteria be repeatable in any test laboratory be feasible in time and cost ensure that a successful outcome demonstrates an acceptable level of usability.
2 Poll Worker Usability Needs The usability of both the documentation and the voting system itself affect poll workers. The VVSG reflects this, saying that procedures should be “reasonably easy” to “learn, understand, and perform,” and that voting system documentation should include “clear, complete, and detailed instructions” for poll workers. 2
In this paper, “VVSG” refers to the VVSG Recommendations to the EAC, August 31, 2007.
460
D. Chisnell et al.
Clearly written instructions are essential to usability. Most poll workers get training provided by the election jurisdiction. That training, as well as any troubleshooting needed on Election Day, ultimately relies on information that comes from documentation supplied by voting system manufacturers. Even with training, poll workers have been reported to misinterpret messages from voting systems, thereby mistakenly invalidating ballots; have problems managing accessible voting systems, leaving people with disabilities unable to vote without assistance; or issue the wrong type of ballot— among other issues. The voting system itself, including system messages and troubleshooting documentation, must be usable by poll workers. Also, voting systems and their documentation must support the realistic situation of the polling place on Election Day. In our review of existing voting system documentation supplied by manufacturers, we found that much of it fell short of best practices for technical communication and information design. [1] We focused on evaluating these two aspects of usability of voting systems for poll workers: system documentation and poll workers using the system along with the documentation at the voting place.
3 Designing the Usability Test Protocol To meet the VVSG requirements and to determine whether voting systems and associated documentation are usable for poll workers, we had to design a test method that goes beyond typical usability testing: How do you design a usability test that someone else, whom you don’t know and can’t talk to, will have to conduct? How do you create a usability test that determines whether something meets requirements in the VVSG rather than generating data to use to improve the design of the system? How do you design a usability test for dozens of different systems that all do the same things in different ways? How do you design a usability test of the documentation in conjunction with the operation of those systems? How do you define success or failure for a requirement? To answer these questions, we constructed a usability test protocol that includes pass/fail criteria. We then designed and conducted an exploratory study in which we acted as the test administrators of the protocol. This helped us see how the protocol might work in practice, better understand the context in which the test would be used, and refine our methodology. 3.1 Creating the Initial Protocol We first created a test design. An essential element of the test design was determining how many participants to use. For the exploratory study, we decided on eight participants, who would work in teams of two in each session for four test sessions. While it is common to use individual participants in usability tests, poll workers work in teams on Election Day to do the tasks we wanted to study in our research. It was our hypothesis that four teams would provide enough evidence for a pass/fail determination. We assumed we were looking for usability problems that any usability expert would identify as severe—that is, based on the observed evidence during the test, poll
Usability for Poll Workers: A Voting System Usability Test Protocol
461
workers could not complete the task. We also assumed that this is one of many tests for the many requirements (including other usability requirements) in the VVSG so the cost of the test must be reasonable. We then created a proposed formal testing protocol. A test administrator at a voting system test laboratory will follow the protocol to conduct usability tests with poll workers on voting systems and their documentation. We developed clear criteria for evaluating system operation and its documentation based on the VVSG and best practice in technical communication and information design. If participant pairs were unable to complete tasks using instructions in the voting system documentation, we considered this a failure of the voting system. The hypothesis is that these criteria will ensure consensus among experts observing the same performance. The testing protocol included instructions and a script for the test administrator, and task scenarios for the participants to follow. In addition, we created a pass/fail checklist for the administrator to document the evaluation. These are described below. Script for the test administrator. The script for the test administrator includes specific, detailed instructions for the administrator to follow, so every test session is administered the same way. For example, it includes instructions to the test administrator about where to be in the room in relation to the test participants to get the maximum benefit from observing the participants without being obtrusive. The script includes specific wording for the test administrator to say at each step during the test. The script also describes what the participants should be doing and how to interact with the participants. Task scenarios. There are three tasks for participants to do in this usability test: y Set up the voting equipment to open the polls. y Conduct voting. y Close the polls to end voting. The test protocol includes task scenarios describing a task goal that participants must respond to by performing the tasks while following instructions in the voting system documentation provided. Here the protocol directs the test administrator what to say and do to introduce each task scenario, and shows what the participants should be doing in response to each step. The protocol also describes how to provide assistance to the participants if they cannot continue with a task. Pass/fail checklist. The pass/fail checklist is a one-page form for the test administrator to use during each test session to document the usability evaluation of the voting system operations and documentation. Because the test administrator at the test laboratory will have experience as a usability test moderator, will be familiar with the VVSG and voting systems, and will have knowledge of technical communication and information design, we felt it should take little training or instruction to use the checklist effectively. The instructions could be minimal. Figure 1 below shows an example of the checklist from the exploratory study. In this case, the main criteria may have been met (the X marks in the boxes for
462
D. Chisnell et al.
Fig. 1. The pass/fail checklist in use during a testing session
Task 1 and Task 2), but the test administrator, using usability and domain expertise, could determine that the documentation and system fail to support this team of poll worker test participants because a key element was missing. For example, here the system messages did not clearly convey that the voter may leave some contests unvoted and still have this ballot count. The form includes: • Evidence—Behaviors to look for. For example, test administrators observe for whether “Participants can use the documentation to find the information they need to complete tasks,” or “… read, understand, and react to system messages.” • Criteria—Operationalized principles on which to evaluate usability. For example, “Are the instructions easy to act on;” “Are poll workers able to respond to messages appropriately?” • End states—Descriptions of what the voting system should be doing, should not be doing, or should be showing that indicate that participants have completed each task successfully. For example, the end state of a successful Task 3 is that the machine cannot take any more votes and that system messages indicate the polls are closed. • Final determination—Based on observations of each pair of participants, and taking into account the criteria and end states for each task, the test administrator decides whether the voting system documentation has an acceptable level of usability for poll workers. Acceptable usability means that poll workers could complete the task using instructions in the documentation. This assessment figures into the larger evaluation taken across sessions. • Comments—An area where the test administrator can include reasons for the final determination, including specific usability problems that participants encountered during the test.
Usability for Poll Workers: A Voting System Usability Test Protocol
463
Attending to the combination of evidence, end states, and criteria for all the test sessions, a test administrator makes a determination about whether the usability of the documentation is acceptable and therefore passes. If a pair of participants is unable to complete tasks, this constitutes a failure. If a majority of participant pairs is unable to complete any one of the tasks, then the documentation, and hence the system, fails the test.
4 Testing the Proposed Test Protocol in an Exploratory Study Having drafted a protocol for test laboratories to use, we tested it in an exploratory study to further refine the script, task scenarios, and checklist. We conducted the exploratory study to answer these questions about the protocol: y y y y y
Do the scripts for the test administrator to use during the session work well? Do they produce appropriate behavior from participants? What are the appropriate performance criteria for determining the acceptable usability of the voting system instructions? What should the success/failure criteria be? What data gathering tools or forms should be in the final protocol? Can the voting system test laboratories conduct the sessions in a reasonable amount of time?
First, we conducted several dry runs of the protocol with different voting systems, refining it each time. Next, we conducted day-long sessions at a research facility, asking pairs of participants who were poll workers to work with two different voting systems, one system at a time. Each of four pairs of poll workers worked on a separate day. Each pair of poll workers worked with one system and its poll worker documentation in the morning and a different system and its documentation in the afternoon. We alternated the order of voting systems each day. All of the participants used the same two voting systems. The poll worker participants used the manufacturer-supplied documentation for each system to open the polls, conduct voting, and close the polls. The tasks did not include dealing with security of the systems (except as the systems are delivered by the vendor) because security procedures are different in each jurisdiction and the VVSG has other requirements specifically to evaluate security. If participants encountered problems for which they asked questions, we gave them hints. If they still could not proceed, the administrator gave them direct assistance. We gave participants hints and assistance because we wanted each pair to attempt each task so we could exercise the entire protocol and script. This helped us refine the script and the wording of the task scenarios, and clarify the pass/fail criteria. Assistance with completing tasks is not part of the proposed protocol for voting system test laboratories to use in certification evaluations. Based on interviews with election directors and our own experiences using voting systems while setting up polling, voting, and closing the polls, we estimated that it could take between one and three hours to complete all of the tasks on each voting system. We used two different voting systems in this study. One was an optical scan system in which voters fill in bubbles on a paper ballot and submit that to a ballot
464
D. Chisnell et al.
scanner. The other was a direct record electronic (DRE) voting system with a touch screen interface. Both are currently being used in elections in the United States. Participants were told that they must follow the instructions in the voting system documentation to do the tasks they were given. The documentation was printed on regular paper measuring 8 ½ by 11 inches. It was printed from files delivered electronically by the system manufacturers to NIST in color, double-sided. The pages were bound with large clips. The documentation for one system was 116 pages. The documentation for the other system was a group of short manuals that taken together was 123 pages.
5 Findings from the Exploratory Pilot Study We theorized that using pairs of poll workers as participants would both reflect a realistic situation for using the voting system and generate evidence on which to base pass/fail determinations. We also theorized that developing specific criteria and end states would ensure consensus among experts observing the same performance. In exploring these two issues, we also wanted to learn how many pairs of poll worker participants it should take to determine whether the voting system documentation passed or failed certification. While the draft version of the protocol that we went into the test with worked reasonably well, we iterated improvements to the test design and protocol as we learned from the sessions. We are reasonably confident that the final test kit would work well for test administrators and that ratings would be similar from administrator to administrator as they use the pass/fail checklist. However, the test must be repeated by others who were not involved in designing the test to verify that it is suitable for use by the test laboratories. 5.1 Tasks Took about 90 Minutes The timing of each session with a voting system ranged from 75 to 135 minutes. Most pairs were done with a system in about 90 minutes. There is one time-dependent activity. The first task, opening the polls, should be stopped after one hour. The usual amount of time scheduled for setting up a polling place is typically between 45 minutes and 90 minutes, with one hour being the most common amount of time allotted. The other tasks are not time-dependent in the same way as the first task. They both happen based on a schedule for Election Day that need not figure into this test. 5.2 Five Sessions Should Be Enough to Determine Pass/Fail In addition to the test administrator’s dry run of the protocol with the voting system and its documentation, we recommended five sessions with poll worker participants to: y lessen any effects for any particular pair of participants having issues with working together y minimize any effects for individual poll workers having a bad day y be confident in an expert analysis based on observation of poll worker performance whether the documentation supports poll workers.
Usability for Poll Workers: A Voting System Usability Test Protocol
465
In the exploratory study, we conducted four sessions each on two voting systems. Our experiences in the exploratory study suggest that as few as three sessions would generate enough evidence to pass or fail the voting system. However, conducting at least two additional sessions—for a total of five—should help the test administrator make a sound analysis that should agree with results if the testing was repeated. For example, if a majority of participants across sessions have the same type of problems with responding to messages (or the lack thereof) about undervoted ballots, the consensus across the test sessions would show that the voting system fails the test. 5.3 Using Pairs of Poll Workers Worked Well We hypothesized that because a polling place relies on a group of strangers working together for a day that using pairs of poll workers as participants would work well for this protocol. We expected that one participant would read the documentation while the other tried to perform the steps being read. This was what happened; participants regularly traded the documentation back and forth between them without prompting from the administrator. The arrangement made it easy to observe whether the instructions were supporting the tasks that the poll workers were trying to do. For example, one poll worker might begin reading the documentation in response to a task but not be able to find the information needed, and then they might switch roles. 5.4 Reaching Specific End States Constitutes Successful Completion In typical usability tests, the design team determines successful completion criteria as well as error conditions. While the test we designed does define successful completion criteria—participants must follow instructions in manuals to reach specific end states—it is not designed to gather error data to improve the design of the documentation or the user interface of the voting system. It is specifically designed to determine if poll worker requirements in the VVSG are met. Task completion here is clear: for Task 1, the system must have the polls open and be ready to receive the first ballot; for Task 2, the system must have ballots cast; for Task 3, the system must be unable to accept any more votes. 5.5 Documentation Failure Conditions are Based on Behavior and Performance Evidence We found there are two fundamental ways that the voting system can fail: y Performing the tasks, participants exceed the total allotted time of two hours for the entire test session. y Participants ask for assistance. Many issues may contribute to participants exceeding the time allotted for the test or having to ask for assistance. For example, for Task 1 opening the polls, for most DREs there are two parts: a voting station and a card activator (or a PIN (Personal Identification Number) generator). If either is not set up properly, the system fails to meet the VVSG requirement for that team of test participants for that task. Analysis
466
D. Chisnell et al.
by the usability expert might show, for example, that the instructions have been left out completely, are in the wrong place in the documentation, or are not easily found. For a laboratory test for specific VVSG requirements, we must assume that assistance is not available. If poll workers must get help from sources other than the documentation, it has failed to support poll worker tasks. If a majority of participants are unable to complete any one of the tasks in usability tests conducted by a voting system test laboratory, then the system documentation fails the test. However, results from our exploratory study suggest that additional analysis may be required for the decision to pass or fail the voting system.
6 Conclusions and Future Research Typically in a usability test, the purpose is to find and remedy design problems. However, testing for voting system certification occurs when a voting system is ready to be deployed. Systems are certified if they meet all the requirements of the standard. Conducting the exploratory study allowed us to refine a certification test protocol: we clarified the wording of the tasks for participants, adjusted the format of the protocol, and tightened the level of detail on the pass/fail checklist. The result is a usability test protocol that, based on our exploratory study, is suitable for use by voting system test laboratories to use to determine whether to certify voting systems for use in elections. However, there is some additional research needed to ensure that the test protocol is ready to be added to the set of VVSG test methods. It is critical to have other usability practitioners perform the tests to verify repeatability and confirm that all aspects of the test protocol are clearly defined. The pass/fail checklist was easy for us to use and we had high inter-rater reliability, but we designed it. We have some questions about the process of making the pass/fail determination. For example, does a failure of one task in one session imply failure of the system or should we require a majority of the sessions to show this same task failure to determine whether a voting system fails the test, and hence certification? Finally, while much research on elections has been focused on voting, usability for poll workers is equally important. Poll workers’ answers to questions and knowledge about voting systems (or lack thereof) have determined whether voters can vote. Our protocol for evaluating voting system documentation based on poll worker performance reveals issues that expert review alone does not. We encourage further research focused on poll worker interaction with voting systems. Acknowledgements. Dana Chisnell and Karen Bachmann were funded under NIST contract SB1341-05-Z-0023-67310. The entire team wishes to thank our many reviewers for their valuable feedback on this paper and the many deliverables generated for the research project.
References 1. Chisnell, D.E., Becker, S.C., Laskowski, S.J., Lowry, S.Z.: Style Guide for Voting System Documentation. Improving U.S Voting Systems, http://www.vote.nist.gov/NISTIR-7519.pdf (accessed November 21, 2008)
Usability for Poll Workers: A Voting System Usability Test Protocol
467
2. Dumas, J.S., Redish, J.C.: A practical guide to usability testing. Ablex, Norwood (1993); Rev. Sub. edn. Intellect Ltd. (January 1, 1999) 3. Laskowski, S.J., Redish, J.C.: Making ballot language understandable to voters. In: USENIX/ACCURATE Electronic Voting Technology (EVT) Workshop, Vancouver, BC, August 1 (2006), http://www.usenix.org/events/evt06/tech/tech.html (accessed on December 7, 2007) 4. PlainLanguage.gov. Federal plain language guidelines, http://www.plainlanguage.gov/howto/guidelines/ reader-friendly.cfm (accessed October 15, 2007) 5. Redish, J.C.: Guidelines for writing clear instructions and messages for voters and poll workers. NIST (National Institute of Standards and Technology) (2006), http://vote.nist.gov/032906PlainLanguageRpt.pdf (accessed September 17, 2007) 6. Rubin, J., Chisnell, D.E.: Handbook of Usability Testing. In: How to Plan, Design, and Conduct Effective Tests, 2nd edn. Wiley, Chichester (2008) 7. Skelton, T.M.: Testing the Usability of Usability Testing. Technical Communication, Third Quarter, 343–359 (1992)
CAD and Communicability: A System That Improves the Human-Computer Interaction Francisco V. Cipolla Ficarra1,2 and Rocío A. Rodríguez1 HCI Lab. – F&F Multimedia Communic@tions Corp. ALAIPO: Asociación Latina de Interacción Persona-Ordenador 2 AINCI: Asociación Internacional de la Comunicación Interactiva Via Pascoli, S. 15 – CP 7, 24121 Bergamo, Italy [email protected] 1
Abstract. We present an analysis of communicability methodology in CAD interactive systems, called DOQ (DObby Quality). This methodology has been under development between 2001 and 2008, obtaining excellent results in both educational and productive contexts. In studies where there is a bi-directional interrelation between usability and communicability of technical interfaces for the design of tissues in cotton, linen, etc., they ease the learning process of the designers who use a textile interactive system for the first time. We also present an educational prototype that can adapt easily to the real production of fabric: DobbyCAD. Keywords: Dobby, CAD, Computer Graphics, Interfaces, Communicability, Usability, Education.
CAD and Communicability: A System That Improves the Human-Computer Interaction
469
The excellent results obtained in heuristic assessment have allowed us to establish two modalities of presenting the same information on the screen. These two modalities are essential for the potential users since it allows them to simulate real fabrics in little time. This important option does not exist in the other assessed systems. Besides, the results of research with real users of CAD systems has made it possible to make a simulator where it is feasible to have on a same screen up to 12 models of fabrics on the same screen that occupies a A4 printer page. Consequently, another advantage of the methodology we present here is saving impression costs on the colour of textile simulations. The beta versions of the product have been experimented successfully in the industrial context and the current final version has been positively assessed by textile design studies, in university teaching centres, etc. The communicability assessment techniques have made it possible to develop an effective emulation system which adapts easily to different kinds of users. The compatibility with operative systems and hardware is 100%. The used methodology allows one to obtain an excellent quality product whose cost is tantamount to 10% of the price of the current commercial systems aimed at industrial textile design. In the current work a brief statement of the state of the art of the textile industrial situation is made, related to the users and responsible staff of textile design. Additionally, there is a summing-up of the methodology used in the realization of the current system. Later on the main requirements of software engineering are included such as the personalization of the different modules of a CAD/CAM system, the different interfaces of the interactive system. Finally, the importance of the current interactive system is stressed, not only in the production of tissues, but also to insert textures in virtual characters and to increase realism.
2 Design, Textil CAD and Users One of the main problems that is observed with the future industrial tissue designers is that in the technical teaching institutes aimed at textiles, the young users are forced to work with real CAD/CAM industrial systems. Obviously, the teachers from these institutes also teach in universities. However, this teaching staff does not make a differentiation by age, training or previous experience of the potential users of these systems. The main problem lies in the interface of the interactive systems. The commercial textile CAD systems for dobby tissues carry out the same functions more or less. However, through the communicability and usability studies that have been made of them it has been seen that the conception of the system, the mindset of the potential users , is practically equal to nil. That is to say, there is a total lack of communicability of the design, which may entail serious problems in usability. One consequence is the high rate of staff change in these design offices. We can see that educative training is aimed more at the industrial factor than the didactic factor (which is common in many textile provinces/regions of Northern Italy). Additionally, every manufacturer and industrialist wants a special personalization of the CAD software, with which the user requires a long learning period of the software in order to make his textile designs. In our case, the CAD system has been created for the different kinds of users based mainly on the educative aspect and simultaneously in the industrial sector. DobbyCAD is easy to use, but through it you can reach real
470
F.V. Cipolla Ficarra and R.A. Rodríguez
manufacturing of tissues. Currently, computer aided design systems have reached a stage of maturity and consolidation in the set of performances and functionalities they offer to potential users, whether they are experts or not. The industrial sector has a wide range of CAD commercial systems available in accordance with its productive activities and services. A user can assess at the moment of designing whether what he is virtually creating adjusts to and fulfills the intended functionalities. In the digital model it is possible to test, validate and verify a product. Here it is necessary to carry out a small anchoring of the notional product in 2D and/or 3D design [5]. When we talk about a product we refer to a house, a piece of an engine or a tissue, to mention some examples. The word "product" means that in some textile industries from Bergamo (for instance) they prefer that name instead of the CAD sector. Obviously, we are in the face of an ambiguous and opaque situation in internal organization those industries. Aside from this confusion situation, having a virtual prototype eliminates the need for the construction of a physical prototype. Simultaneously, the current CAD systems allow the interface to adapt to the needs of personalization of products and potential users [6]. This is a direct consequence of the flexibility of the software [7], [8]. To make changes, design variables in the virtual models have a lower cost than in the physical models. Additionlly, it allows users to instantly contribute improvements to a product that is being designed with a team who work on-line. The systems analysts, programmers and graphic informatics engineers are constantly moving towards a complete integration and interrelation with other programs that intervene in the development of a product. It allows for the prevention and minimisation of manufacturing mistakes and on top of that, production costs. Most of the CAD systems present a modular structure that can be widened in order to work in the different stages of the life cycle of a new development. This modular structure exists in the medium range CAD systems, such as the high rank systems. In the case of textile products, the different modules may refer to the type of tissues, that is to say, dobby. jacquard, etc. The presented educational prototype respects the notions of modularity and flexibility. 2.1 Software Development and Design: Personalization The users play a determining role in the incorporation of the new performances. In the same way that the context of the industrial products is characterized by a high personalization, in CAD systems this tendency is also detected, by incorporating new commands that are the results of requests from the different users. In the case of the textile, the hardware usually sets very important differences in the interface, especially if the Macintosh brand is compared with the other personal computers (or classically called ‘compatible with IBM PC’). In the Macintosh environment CAD systems are aimed at the realization of models, that is to say, one works with image desktop publishing programs (DTP), such as Photoshop or others related to vectorial images, such as is the case of the Corel Draw, for instance. That is to say, it is an environment that usually does not make issues. However, within the portability of the software it is necessary that the CAD system in an industrial environment should be able to function in both hardware platforms. In our educational prototype the compatibility of the software is guaranteed 100%. These kind of users are experts in desktop publishing and design. Consequently, the systems usually follow the style rules in the design of
CAD and Communicability: A System That Improves the Human-Computer Interaction
471
Macintosh interfaces [9]. In the industrial context the CAD systems usually generate their interfaces in accordance with the used programming language. The preponderance of the style of the Windows operating system in some components of the interface is observed, such as: menu bar, dialog boxes with display rectangles, labels for interface elements. etc. The requirements on the part of the users of the first group – Macintosh, are different to the second group– Windows. The first are related to the performances of the bi-dimensional drawing of human figures, the colouring of dresses, and exceptionally the inclusion of real textures of clothes in their creations. Each one of them may need simulation of the movements in these clothes in relation to the different raw materials used [10], that is to say, cotton, wool, linen, silk, etc., in virtual characters. An important factor of CAD is the possibility of extending the data to the production, that is to say, CAM. Having two or three systems in parallel is synonymous with a loss of financial resources and time, that is to say, a system for design (CAD), another for the management (administrative data, accountancy, staff, etc.) and the third for the CAM (production). Currently, these requirements are not necessary in the production sector who work in a Windows interface environment, because they are aimed at the real creation of tissues, that is to say, there is a symbiosis between CAD/CAM. The design is more related to technical issues, since in a later stage there lies the real production of these creations that originated on computer screens, which is mistakenly called in many Lombardy industries the ‘product department’. Obviously, it can be said that the degree of personalisation surpasses exponentially that of the first group. The origin of this phenomenon lies in the lack of graphical knowledge or computer design on the part of the people who run it, whether it is in the design department as in the data processing, and/or in the information technology department. Consequently, the costs of the second CAD systems widely surpass the first group, although the principles of applied graphical informatics are the same.
3 DobbyCAD: The Main Advantages and Characteristics DobbyCAD is a program for the design of textile tissues, which can be inserted in any kind of virtual characters, regardless of the modeling software or the animation that has been used. The executable program takes less than one Mb. Addtionally, the application is modular and does not require any hardware in particular. That is to say, from the point of view of the software and the hardware it is 100% compatible with the current computers. –PC or Mac– and operative systems (Windows XP, Vista, Linux, Mac OSX, etc.). Consequently, it does not need additional costs for its incorporation into the design departments or offices. (figure 1). The user, starting from the basic notions of textile design [11], will be able to create freely in a few hours those models he/she is most interested in so as to increase the realism of the scenes where there are human characters who use shirts, T-shirts, blouses, trousers, jeans, skirts, dresses, handkerchiefs, caps, etc. avoiding that metalized material in the clothes, which have so often been seen in computer animated films. Besides, the texture created with DobbyCAD, whether it is in a bitmap and vectorial format (jpg, tif, bmp, etc) [12], can be modified later on with any self-edition program: Corel Draw, Illustrator, Photopaint, Photoshop, etc. (figure 2).
472
F.V. Cipolla Ficarra and R.A. Rodríguez
Fig. 1. DobbyCAD –Easy design and simulation of real textile tissues
Fig. 2. DobbyCAD –Weave Design
In the same way as the CAM (computer.-aided-manufacturing) module makes it possible to transform the data of the graphical information, the management software for production, that is to say, the loom. In other words, it is feasible to create any kind of ligament and fold and weaving that the designer always has available in a database for its latter visualization and modification. DobbyCAD has demonstrated very well its ability in such environments, as it can be appreciated through the exquisite and demanding customers of the textile world, to which daily and through the years it is producing kilometers of tissues of the highest
CAD and Communicability: A System That Improves the Human-Computer Interaction
473
quality for the later elaboration of prestigious clothing made in Italy (Bergamo) – currently, factories and/or academic institutions in Argentina, Brazil, Britain, Canada, China, Costa Rica, France, Germany, India, Mexico, Portugal, Spain, Turkey, U.S.A., Venezuela, etc. The origin of the design of the real tissues lies in the notion of weaving and warping. The plot or weaving is the horizontal line of the clothes, and the warping is the vertical one. In the representation a square box system is used in which the horizontal and vertical threads crisscross (Figure 1), that is to say, the weaving and the warping. Starting from these thread combinations the tissue is born (see figure 3). With regard to the threads there is an interesting variety to be considered in the real world, such as chenilles, flamés, friezes, etc. Obviously, these threads can be generated through the use of commercial DTP programs or computer animation, such as Illustrator, Maya, etc. It is necessary to point out that the quality obtained through the tridimensional threads these days surpasses by far digital photography.
Fig. 3. DobbyCAD –Access to database and automatic simulation of textile tissues
These threads can be stored later on in a database. Another way to obtain them is through the use of a special scanner, working directly on threads or real tissues [13]. One can also resort to digital photography. Additionally, there are those CAD textile system programmers who maintain that photography of the threads microscope is enough in order to modify them and give them colour later on. Obviously, the option of making them manually with some desktop publishing program is a valid alternative, when other resources of commercial graphical computers are not available, such as can be 3D Studio Max, Rhinoceros 3D, Softimage, etc. As for colour, its selection is very easy with DobbyCAD and can be made through the keyboard or the scanner. In the first case, it is through the incorporation of the Pantone code, or giving a personalized code to each one of the colours that are used in the collection that is being created (figure 4). The second alternative is to pick up a thread or a skein, a cardboard or paper and get its colour. In both options, the human factor is always important because a mistake at that moment may entail significant loss of time and, indirectly, money.
474
F.V. Cipolla Ficarra and R.A. Rodríguez
Fig. 4. DobbyCAD –An easy selection of the colors for the yarns
The issue of the threads and the colours is very interesting because sometimes the prevailing ignorance in some working environments entails changing a whole CAD/CAM system, because according to the unaware bosses in the design department the CAD can not simulate/emulate colourless tissues, when in reality it is just simple bitmap files where it is necessary to simulate and emulate the ‘z’ coordinate, adding some other lighting effect. With programs such as 3D Studio Max, Cinema 4D, LightWave, etc. or even the classical ‘Paint’ of the Windows operating system, these effects can be achieved in a splendid way. The costs from the human, labour and economical points of view may be very high in the face of heads of the creative or design sectors who lack a minimal information in infography. Of course in these cases these software textile selling industries will invite you to change completely the CAD/CAM system. The management of the threads and the colours from DobbyCAD is very simple. Additionally, the changes made with the 2D or 3D commercial software on the threads can easily be seen on the computer screen or in the printer. In the triadic relationship between printer, screen and the real tissue textures, a whole calibration process of the colours has to be carried out, which entails several hours of tests, so that it adjusts as close as possible to the 100% of realism, between what is seen and what is printed. DobbyCAD admits the possibility of printing several tissue models on a same page: 1, 2, 4, 6, 8, 9, 12, etc. In the figure 6, six real simulations of tissues with different colours in the selection of the threads are depicted, and have been printed in a single folio. The philosophy followed in this software is ecological, also in the consumption of paper. Each one of the advantages presented in the DobbyCAD is the result of a long analysis process and perfecting of a series of heuristic techniques which have made up a methodology or method. The method used for the making of the current system has basically consisted of the heuristic analysis of the main CAD/CAM systems in the European market and work with a set of strategies (figure 5). To this end quality attributes have been used as well as the usability and communicability metrics of MECEM (Metrics for the Communication Evaluation in Multimedia) [14]. Aside from the heuristic assessment table (readers who are interested will find a wide bibliography in [14], [15], [16]).
CAD and Communicability: A System That Improves the Human-Computer Interaction
475
Fig. 5. DobbyCAD –Communicability, Design and Software Strategies
The results obtained with the application of that methodology have made it possible to develop the first beta versions of the system, which has been successfully adopted by real users, without the need to make tests in usability laboratories, for instance. Furthermore, the costs in the stage design of the system have been minimal thanks to the presence of a heuristic assessor who specializes in communicability and usability.
Fig. 6. DobbyCAD –Possibility of printing several tissue models on a same page
4 The Textures Increase the Realism of the Scene If we watch the evolution of computer made productions on the screen, we find that the first characters that mimicked human beings were tin or plastic puppets: the short film Tin Toy or the full-length film Toy Story [17]. However, already in the Woody
476
F.V. Cipolla Ficarra and R.A. Rodríguez
character from Toy Story the first shirt with its big checks appears. This type of shirt depicts the very history of textile industry, because the classical fashion made with stripes, squares or just the cloth without any design has endured until today. Now there is a tendency to emulate 3D with these two lines: warping and weaving. Now, we see too that there is a new tendency to solve certain problems which are inherent to the handling of textile textures at the moment of the animation. Nevertheless, in the computer graphics studies [18], the textures are still being obtained from databases with tissues that have been previously digitalized through a scanner or with a photography of them. This is the great Achilles heel of the bookshops in programs such as Quidam, Zbrush, etc. Through the DobbyCAD with a few hours and some basic notions of textile design the user can make real tissues for the virtual characters. An example of this is in the figure 7 of the named ‘Antaxel’. One of the problems that persists and which has been solved with DobbyCAD is to have real tissues available, which can be interactively modified through the simulator in the different shades of colours and even go down to the minimum texture detail.
Fig. 7. Antaxel –Virtual Character
Fig. 8. Textil Design and Visual Illusions
It shouldn't be forgotten either that through the tissues there is a series of images that from the point of view of perception can generate a series of illusions, making a synthetic character be accepted or not, as we can see in the texture of figure 8 (the combination of weavings and warpings are white, although we may perceive them as black).
5 Conclusions There is an excellent convergence of graphical engineering and person-computer interaction in the current work through the possibility of combining commercial programs of graphical computers to generate 2D and 3D animated characters and the novelty contributed from DobbyCAD (whose cost is 90% lower to those currently existing in the international market). This graphical software has evolved qualitatively, serving since 2002 the real production of wear clothes and world-renowned stylists. Through it, it is feasible to improve the quality of realism in animated graphical computers and very especially, in the virtual characters, thanks to its splendid interactive simulator.
CAD and Communicability: A System That Improves the Human-Computer Interaction
477
Acknowledgments. A special thanks to Emma Nicol (University of Strathclyde), Maria Ficarra (Alaipo & Ainci – Italy and Spain) and Carlos for their helps.
References 1. Kaldor, J., James, D., Marschner, S.: Simulating Knitted Cloth at the Yarn Level. ACM Transactions on Graphics 27, article 65 (2008) 2. Goldenthal, R., et al.: Efficient Simulation of Inextensible Cloth. ACM Transactions on Graphics 26, article 49 (2007) 3. Shneiderman, B.: Designing the User Interface –Srategies for Efective Human-Computer Interaction. Addison Wesley, Massachusetts (1998) 4. Paoluzzi, A.: Geometric Programming for Computer Aided Design. John Wiley, West Sussex (2003) 5. Mitchell, W., McCullough, M.: Digital Design Media. Van Nostrand Reinhold, New York (1995) 6. Ishii, H.: The Tangible User Interface and Its Evolution. Communications of the ACM 51, 32–36 (2008) 7. Ebert, C.: Open Source Software in Industry. IEEE Software 25, 52–53 (2008) 8. Ghezzi, C.: Fundamentals of Software Engineering. Prentice Hall, New Jersey (1991) 9. Apple: Macintosh Human Interface Guidelines. Addison-Wesley, Massachusetts (1992) 10. Kautz, J., Boulos, S., Durand, F.: Interactive Editing and Modelling of Bidirectional Texture Functions. ACM Transactions on Graphics 26, article 53 (2007) 11. Grana, C.: Tecnologia e Merceologia Tessile. San Marco, Bergamo (2003) 12. González-Jiménez, J.: Visión por Computador. Paraninfo, Madrid (2000) 13. White, R., Crane, K., Forsyth, D.: Capturing and Animating Occluded Cloth. ACM Transactions on Graphics 26, article 34 (2007) 14. Cipolla-Ficarra, F.: Communication Evaluation in Multimedia –Metrics and Methodology. LEA, Mahwah, vol. 3, pp. 567–571 (2001) 15. Cipolla-Ficarra, F.: Communicability design and evaluation in cultural and ecological multimedia systems. In: Proc. Workshop Communicability MS 2008, pp. 1–8. ACM Press, New York (2008) 16. Cipolla-Ficarra, F.: Table of Heuristic Evaluation for Communication of the Multimedia Systems. In: Proceedings of the HCI International, pp. 940–944. LEA, Crete (2003) 17. Smith, T.: Industrial Light & Magic: The Art of Special Effects. Del Rey Books, New York (1986) 18. Newman, W., Sproull, R.: Principles of Interactive Computer Graphics. McGraw-Hill, New York (1979)
A Novel Visualization Tool for Evaluating Medication Side-Effects in Multi-drug Regimens Jon Duke1,2, Anthony Faiola1, and Hadi Kharrazi1 1
Indiana University Purdue University Indianapolis, School of Informatics 535 W. Michigan St., Indianapolis, Indiana 46202 2 Regenstrief Institute, Medical Informatics, HITS 2000 550 W. 10th Street Indianapolis, IN 46209 [email protected], [email protected], [email protected]
Abstract. The evaluation and management of medication side-effects is a common and complex task for physicians. Information visualization has the potential to increase the efficiency and reduce the cognitive load involved in this process. We describe the design and development of Rxplore, a novel tool for assessing medication side-effects. Rxplore supports simultaneous lookup of multiple medications and an intuitive visual representation of query results. In a pilot study of Rxplore’s usability and utility, physicians rated the system highly for efficiency, intuitiveness, and clinical value. Keywords: information visualization, medical informatics, adverse reactions, medications, side-effects.
A Novel Visualization Tool for Evaluating Medication Side-Effects
479
with each medication and note how frequently these effects are reported in clinical trials. Yet even the most efficient systems require physicians to look up medications one-by-one. Furthermore, physicians must rely on memory or note-taking to organize the information they uncover while interacting with these systems. In this paper, we describe the development of a new visualization tool designed to address these issues and to expedite the review of side-effect data. We also present the results of a pilot study on the usability and clinical utility of this system.
2 Theory and Background: Information Visualization and Biomedical Informatics Information visualization is a mapping technique that utilizes visual graphics to represent non-spatial abstract datasets in physical space [6]. Visualization enables users to make discoveries and decisions regarding complex data, elucidating subtle patterns and making information-intensive tasks more manageable. In the following section, we explore the theoretical underpinnings of information visualization as a means to augment human cognition. Then we review some current applications of visualization in the healthcare setting. 2.1 Information Visualization and Augmented Cognition A prime challenge of modeling a meaningful visualization is identifying an appropriate visual mapping of the information that best supports correlations and relationships between the unstructured data; thereby enhancing new understanding. How people perceive, understand, and apply information visualization is essential to grasp the relationship between human-computer interaction and human cognition. As Heer et al. state, information visualization seeks to “augment human cognition by leveraging human visual capabilities to make sense of large collections of abstract information” [7]. The amplification of cognition through information visualization modeling, joined with data analysis, provides users interaction with a support for more sense-making of large and complex datasets. For example, amplifying cognition through information visualization both increases cognitive resources by expanding working memory and reduces search by representing large datasets in a small space. As Beale et al. state [8], cognitive amplification through the use of visualization methods “helps to shift the work load from the cognitive to the perceptual system; expands the working memory; and allows a high level of interaction… [thus aiding in the users’] confirmation and discovery of knowledge”. Furthermore, information visualization supports relationships of perceptual inference that would otherwise be quite difficult to reproduce. One of the greatest benefits of information visualization is that it (unlike static media) provides a means to manipulate and enable the exploration of a space of parameter values [8,9]. This is observed in the theory of external cognition as applied to design issues and an analysis of how graphical representations are used during cognitive activities such as problem-solving [10]. Specifically, external cognition refers to the “interaction between internal and external representations when performing cognitive tasks such as learning” [11]. The design dimensions of information visualizations are intended to help researchers and practitioners, through
480
J. Duke, A. Faiola, and H. Kharrazi
combinations of visualizations, to experience a type of “computational offloading, the extent to which different external representations vary the amount of cognitive effort required to carry out different activities” [11]. This concurs with Norman’s early work [12,13] in the psychology of humancomputer interaction, in which he argued for a fundamental paradigm shift in understanding the way interactive products are designed. For example, he suggested that well-designed artifacts should reduce the need for users to remember large amounts of information. In the section titled “The Conspiracy against Memory” [12], Norman highlights our inability to freshly retain many items, i.e., the way these items work and the way they relate to one another. He asserts that the human mind is limited in its ability to think deeply about any given topic, primarily because of the restricted capacity of working memory. For this reason, visual aids, such as information visualization, are necessary to support both cognition and an array of learning processes, as well as to reduce human error. In sum, researchers and system designers need to find ways to arrange complex systems that are visible to reduce information in memory and reflect human logic. Consequently, we can consider the visualization interface functions as the contact point for interaction, where information enters through the human sensory system, is organized, and finally recognized to ensure an error-free display of the information. When interacting with a visualization system, users may process the information into different forms, e.g., perception and transformation of understanding. On one hand, the computational modeling of data visualization is critical for an accurate display of information. On the other, interaction design and information design are at the heart of information visualization. Most importantly, empirical findings have shown that properly designed visualizations have the potential to give a clear picture at a glance. This is because the laws of semantic configuration must be learned, but images provide automatic comparisons and obvious relationships [14]. 2.2 Information Visualization in Healthcare Information retrieval is an important part of the daily routine of physicians and other healthcare providers. Yet the body of knowledge in medicine is increasing enormously, forcing clinicians to rely on various external sources of medical information [15]. These resources can range from textbooks, to external websites, or even to fully integrated clinical decision support (CDS) systems. When discussing these tools, it is important to differentiate between those designed purely for “data availability” and those which have extended this concept to include the notion of “information extraction” [16]. Resources that offer data availability alone (such as a textbook or patient chart) will provide the physician with accurate information but leave him with the burden of collecting, maintaining in working memory, and interpreting these data. Conversely, resources that move beyond availability and provide assistance with information extraction (such as context-sensitive clinical reminders or real-time cardiac telemetry) can reduce the physician’s cognitive load. Information visualization has been shown to be one such successful method of supporting extraction and reducing cognitive load in the medical setting [17,18]. There are currently a number of visualization applications in clinical use. One such application that is often overlooked is the electrocardiogram (EKG), which is
A Novel Visualization Tool for Evaluating Medication Side-Effects
481
essentially a visual representation of the heart’s direction and force throughout the cardiac cycle. More traditional forms of data visualization would be graphical monitoring of blood pressure, pulse, and other vital signs in the intensive care unit [19]. Graphical displays of laboratory results have also become a standard feature of electronic medical record systems in both the hospital and outpatient settings. More recently, work has been done to visually represent patient clinical histories, such as diagnoses, medications, surgeries, and so forth [20,21]. These clinical visualizations provide an integrated view of the temporal characteristics of a patient’s history, giving the physician an integrated high-level view of the patient’s symptoms and treatments over time. Despite these areas of progress, there are relatively few reports in the literature of visualization techniques being applied to the evaluation of medications and their sideeffects. One such application is DOPAMINE, a tool for visualizing drug therapeutic classes, but this work was primarily focused on ontology development rather than clinical practice [22]. Another application is called “Mister VMC,” a tool which implements an anatomic representation technique to portray side-effects, contraindications, and interactions in a real-time clinical context. A study showed improved accuracy and speed of obtaining drug information using this tool when compared with a standard text-based system [23]. However the major drawback of this system is its complex iconography, requiring prior training in a proprietary visual language to use the application effectively. 2.3 Adverse Drug Reactions and Clinical Decision Support While the availability of graphical decision support systems for drug information is quite limited, text-based decision support applications have been increasingly adopted by physicians. Two prime examples are Epocrates and UpToDate. These applications have been shown to be effective in reducing prescribing errors and assisting clinical decision-making [24,25]. Both solutions are popular, with 100,000 Epocrates users and over 300,000 UpToDate users visiting their websites every month [26,27]. Whether accessing the PDA or Desktop version, a user interacts with these applications by searching for a medication of interest and having the option to choose from several pertinent categories of prescribing information. Side-effect data are generally displayed in a slightly condensed (UpToDate) or significantly condensed (Epocrates) form. Users typically write down or just remember the information they need before closing the application and returning to the task at hand. There is no current method of looking up multiple medications simultaneously, or of searching across a set of medications for a particular side-effect. While Epocrates and UpToDate are both excellent information resources and have improved the efficiency of retrieving drug side-effect data, they have not significantly reduced the overall cognitive complexity of the process. In other words, their focus remains on data availability rather than information extraction. In the following section, we describe the creation of a novel drug information resource which aims to support both these elements in equal measure through effective data visualization methods.
482
J. Duke, A. Faiola, and H. Kharrazi
3 Methods 3.1 Development of the Rxplore Visualization Tool The development goals were to create a tool which would allow users to rapidly retrieve side-effect data on multiple medications and to deliver these results in an easily interpretable manner with minimal cognitive load. The development process comprised two stages: 1) creation of a database containing quantitative data on medication side-effects 2) construction of a web interface to query this database and display the results of multiple queries in graphical form. Creation of the Quantitative Adverse Reaction Knowledgebase (QuARK). Adverse reaction data were extracted from 250 FDA-approved medication labels by both manual and natural language processing extraction methods. These data were based on multiple sources (e.g., clinical trials, post-marking reports) and could be either quantitative or qualitative. In order to account for these differences, we constructed algorithms to simplify the relationship of each medication and side-effect to a singular value called the Rxplore Score (Table 1). Table 1. Example algorithms used in generating Rxplore Scores Reporting Format Drug and Placebo Frequency Range Qualitative Terms
Example Nausea seen in 27% on Drug X and 12% placebo Itching occurred in between 3% and 9% of patients Rash occurred “rarely”
Algorithm Drug % - Placebo %
Rxplore Score 27 – 12 = 15
LB+(UB–LB) / 3
3+(9-6)/3 = 5
Expert consensus + Matching terms in quantitative datasets
0.3
This scoring system was not designed to be an exact measurement of frequency, but rather to provide a conceptual gauge reflecting the association of each drug with a particular side-effect. The original data were retained in the knowledge-base to allow recalculation of the scores in case of an algorithm modification. The final Rxplore database contains information on over 16,000 medication and side-effect pairs. Interface and Visualization Design. A web-based interface to the Rxplore data was then developed using ASP.NET, Flash, and SQL Server. The requisite features were: 1) the ability to create a list of multiple medications 2) a means to search for a specific side-effect across this entire list and 3) an intuitive visual display of the search results. For the visualization, we selected a stacked-bar format in which each row represents a single side-effect, and each bar within the row reflects the Score of that effect for a given medication (Figure 1). Bars are differentiated by color to allow the user to see which medications are associated with a particular effect. The size of each bar is proportional to the calculated Rxplore Score, so that when seen as a collective, the graph conveys instantaneously which medications are most likely to cause a given effect.
A Novel Visualization Tool for Evaluating Medication Side-Effects
483
Fig. 1. A summary graph from Rxplore, depicting the most common side-effects reported with a regimen consisting of six different medications. Larger bars suggest a higher associated frequency between the drug and a side-effect.
Fig. 2. A summary graph of medication side-effects with detailed frequency data displayed. Moving the mouse over a bar on the graph retrieves the specifics for that medication and sideeffect combination.
An “overview plus detail” model was used to provide textual side-effect data on demand. For example, if the user wants to retrieve the details on the relationship between Zoloft and Insomnia, he can hover over the graph in the Insomnia row on the Zoloft bar and the desired information will be displayed (Figure 2). Hence, the visualization technique allows for rapid interpretation of the high-level data as well as expedient retrieval of lower level details.
484
J. Duke, A. Faiola, and H. Kharrazi
In addition to displaying the most common side-effects (Figure 2) for a medication regimen, the user can also retrieve the most common effects of a particular type (e.g. cardiac, gastrointestinal) or search for an individual side-effect of interest. Medication regimens and custom side-effects lists can be stored for future review and adjustment. 3.2 Evaluation Study Research Questions. In conducting a preliminary evaluation of Rxplore, we sought to answer two basic questions. First, how do health care providers rate the system’s usability? Specifically we were interested in its learnability, efficiency, and ease of use. Second, how do these providers rate the system’s clinical utility? Our metrics for utility were perceived clinical value and relevance to doctors’ daily work. Participants. We invited 63 physicians and nurses at the Indiana University Medical Group to participate in this evaluation study, of whom 10 enrolled. Treatment. The subjects were first shown an introductory video to explain the concept and features of the Rxplore tool. Then, they were presented with two patient scenarios, each containing a clinical description, current medication use, and recent history and physical exam findings. For each scenario, there were 5 questions regarding the patient’s symptoms and potential causal relationship of the patient’s medications (e.g. “Which of these medications is most likely to be causing his Nausea?”). Subjects worked through the questions independently using the Rxplore tool. Upon completion of the tasks, they were given an online questionnaire regarding the system’s usability and clinical utility. Follow-up interviews were conducted to gain further insight into the subject’s experiences with Rxplore.
4 Results 4.1 Demographics Participants consisted of 90% physicians while 10% were with nursing backgrounds. The subjects described themselves as 56% avid users of computers, 33% average users and 11% with poor computer skills. We explored their current drug information needs and found that 67% used an electronic resource to look up medication sideeffects on a daily basis, 22% did so a few times per week, and 11% accessed such a resource only a few times a month. 4.2 Utility Rxplore’s effectiveness and clinical utility were measured by task-oriented selfreported questions. For example, the users were asked to rate their experience when performing tasks such as adding a medication, deleting a medication, finding a specific side-effect or selecting a side-effect category. Rxplore was rated by average 94.5% effective in accomplishing the aforementioned tasks.
A Novel Visualization Tool for Evaluating Medication Side-Effects
485
4.3 Usability A number of questions focused on the usability of the system. Its Intuitiveness rated 4.7 out of 5 while Ease to Use rated 4.6 out of 5. Participants overall found the system appealing (4.6 out of 5) and also aesthetically pleasing (4.9 out of 5). Subjects rated Rxplore’s Speed and Time-Saving potential as 4.4 out of 5. Furthermore, when asked if it was faster than their current source of drug information, subjects rated Rxplore at 96% for retrieving all common side-effects and 100% for retrieving the frequency of a specific side-effect. Participants were also asked specific questions about the system’s visual concept and the meaning of its elements. For example, regarding the clarity of the bars in predicting the risks of experiencing a side-effect, users rated the system 100%. The same result was concluded regarding the relationship between multiple bars on the same side-effect and several medications causing the effects. One area of weakness was in recognizing Rxplore as a conceptual measure rather than as a precise statistical metric. This concept received only a 60% rating for clarity. 4.4 Usefulness Regarding the potential clinical value, subjects rated the system 4.5 out of 5 in terms of relevancy to their work and 4.9 out of 5 in terms of usability in clinical practice (Figure 8). Overall satisfaction was high, with 90% of subjects rating their experience with Rxplore as Excellent or Good; none had a Fair or Poor experience. Likelihood of using Rxplore again was high (4.6 out of 5) as was the willingness to recommend Rxplore to colleagues (4.6 out of 5).
5 Discussion The benefit of visualizing complex information has been established in multiple health care settings. In this paper, we have described a new use of information visualization for the purposes of evaluating medication side-effects. This visualization tool, known as Rxplore, has two main advantages over current methods of researching adverse drug reactions. First, it allows for retrieval of information on multiple medications simultaneously. Second, by quantifying the relationship between each medication and its associated effects, Rxplore can provide an immediately interpretable visual representation of this complex set of data. Although small in size, our pilot study’s cohort of physicians demonstrated considerable enthusiasm for the tool’s usability and potential value in the clinical setting. By using standard visualization methods, including stacked bar graphs and rollover text boxes for detailed data, Rxplore’s learning curve was kept to a minimum. No dedicated training was required to use the system, and participants rated it highly in terms of both ease of use and intuitiveness. While quantitative measures of cognitive load were not recorded in this study, subjects reported the system was efficient, timesaving, and faster than their current drug information resource. This increased efficiency may relate to decreased demands on working memory and the computational off-loading achieved by providing information on multiple medications in a single image. Typically, users would need to remember their findings for each individual
486
J. Duke, A. Faiola, and H. Kharrazi
drug as the data are looked up sequentially. With Rxplore’s simultaneous graphical retrieval technique, cognitive load previously spent on memory tasks can be shifted to the interpretation of the data. This shift in effort results in improved information extraction without additional work on the part of the physician. Despite its potential benefits, Rxplore does have several limitations as well. The most significant issue being the reliability of the underlying algorithms used to calculate the Rxplore Scores. Though validation studies are underway, these Scores are necessarily coarse reflections of the complex (and sometimes conflicting) data available regarding medication side-effect frequencies. In absence of a gold standard, physicians must rely on the body of available knowledge and their own clinical experience to judge potential adverse reactions in any given patient. Thus, the Rxplore Scores have been designed not as a precise statistical model of side-effect likelihood, but rather as a guide to encourage physicians to explore the underlying data in a more efficient manner. The challenge in any information visualization is to ensure that the viewer understands the full meaning of the displayed information. With Rxplore, avoiding hasty assumptions and encouraging exploration of the underlying data is absolutely essential. Based on the study results, several physicians did not fully understand the use of Rxplore Score as a conceptual rather than statistical metric. Thus, we must present the methodology to users more clearly to ensure clinical judgments are based on the full spectrum of available data. The major limitation of our pilot study was its small size and absence of a control resource for comparison. Upcoming work will include a comparative study between Rxplore and traditional drug information resources. We will also expand the number of participants as well as the acquisition of quantitative data for measuring usability, efficiency, and cognitive load. Finally, integration of Rxplore into an existing electronic medical record system is now underway and will provide a rich test bed for further investigation into the system’s use.
References 1. Lazarou, J., Pomeranz, B.H., Corey, P.N.: Incidence of Adverse Drug Reactions in Hospitalized Patients: A Meta-analysis of Prospective Studies. JAMA 279, 1200–1205 (1998) 2. Ernst, F.R., Grizzle, A.J.: Drug-related morbidity and mortality: updating the cost-ofillness model. J. Am. Pharm. Assoc. 41, 192–199 (2001) 3. DailyMed: About DailyMed, http://dailymed.nlm.nih.gov/dailymed/about.cfm 4. Veehof, L.J.G., et al.: Adverse drug reactions and polypharmacy in the elderly in general practice. European Journal of Clinical Pharmacology 55, 533–536 (1999) 5. Kaufman, D.W., et al.: Recent Patterns of Medication Use in the Ambulatory Adult Population of the United States: The Slone Survey. JAMA 287, 337–344 (2002) 6. Voigt, R.: An Extended Scatterplot Matrix and Case Studies in Information Visualization. Published as Diplomarbeit (2002), http://www.vrvis.at/via//resources/DA-RVoigt/DA.pdf 7. Heer, J., Card, S., Landay, J.P.: A Toolkit for Interactive Information Visualization. In: Proceeding of the ACM Conference on Human Factors in Computing Systems, pp. 421–430 (2005)
A Novel Visualization Tool for Evaluating Medication Side-Effects
487
8. Thomas, J.J., Cook, K.A.: Illuminating the Path. 200 (2005) 9. Card, S.K., Mackinlay, J.D., Shneiderman, B.: Readings in Information Visualization 686 (1999) 10. Scaife, M., et al.: External Cognition: How Do Graphical Representations Work. International Journal of Human-Computer Studies 45, 185–213 (1996) 11. Rogers, Y.: New theoretical approaches for human-computer interaction. Annual Review of Information Science and Technology 38, 87–143 (2004) 12. Norman, D.A.: The Design of Everyday Things 257 (1990) 13. Norman, D.A., Draper, S.W.: User Centered System Design 526 (1986) 14. Gregg, L.W., Bower, G.H., University, C.: Cognition in Learning and Memory, pp. 51–88 (1972) 15. Holzinger, A.: HCI and Usability for Medicine and Health Care 458 (2007) 16. Workman, M., Lesser, M.F., Kim, J.: An Exploratory Study of Cognitive Load in Diagnosing Patient Conditions. Int. J. Qual. Health Care 19, 127–133 (2007) 17. Shachak, A., et al.: Primary Care Physicians’ Use of an Electronic Medical Record System: A Cognitive Task Analysis. J. Gen. Intern. Med. (2009) 18. Stoicu-Tivadar, L., Stoicu-Tivadar, V.: Human-Computer Interaction Reflected in the Design of User Interfaces for General Practitioners. International Journal of Medical Informatics 75, 335–342 (2006) 19. Wenkebach, U., Pollwein, B., Finsterer, U.: Visualization of Large Datasets in Intensive Care. In: Proc. Annu. Symp. Comput. Appl. Med. Care, pp. 18–22 (1992) 20. Shahar, Y., Cheng, C.: Intelligent visualization and exploration of time-oriented clinical data. Top Health Inf Manage 20, 15–31 (1999) 21. Plaisant, C., Mushlin, R., Snyder, A., Li, J., Heller, D., Shneiderman, B.: LifeLines: Using Visualization to Enhance Navigation and Analysis of Patient Records. In: Proc. AMIA Symp., pp. 76–80 (1998) 22. Wroe, C.J., Solomon, W.D., Rector, A.L., Rogers, J.: Dopamine: A Tool for Visualizing Clinical Properties of Generic Drugs. In: Lavrac, B.K.N., Miksch, S. (eds.) International Workshop on Intelligent Data Analysis in Medicine and Pharmacology. The Fifth Workshop on Intelligent Data Analysis in Medicine and Pharmacology (2000) 23. Lamy, J., Venot, A., Bar-Hen, A., Ouvrard, P., Duclos, C.: Design of a Graphical and Interactive Interface for Facilitating Access to Drug Contraindications, Cautions for Use, Interactions and Adverse Effects. BMC Medical Informatics and Decision Making 8, 21 (2008) 24. Berner, E.S., et al.: Improving Ambulatory Prescribing Safety with a Handheld Decision Support System: A Randomized Controlled Trial. J. Am. Med. Inform. Assoc. 13, 171– 179 (2006) 25. Wilcox, R.A., Whitham, E.M.: Reduction of Medical Error at the Point-of-Care Using Electronic Clinical Information Delivery. Internal Medicine Journal 33, 537–540 (2003) 26. Quantcast UpToDate Profile, http://www.quantcast.com/uptodate.com 27. Quantcast Epocrates Profile, http://www.quantcast.com/epocrates.com
Design of a Web Intervention to Change Youth Smoking Habits Kim Nee Goh1, Yoke Yie Chen2, Emy Elyanee Mustapha3, Subarna Sivapalan4, and Sharina Nordin5 1,2,3
Department of Computer and Information Sciences 4,5 Department of Management and Humanities Universiti Teknologi PETRONAS Bandar Seri Iskandar, 31750 Tronoh, Perak Darul Ridzuan, Malaysia {gohkimnee,chenyokeyie,emyelyanee,subarna_s, shahrina_mnordin}@petronas.com.my
Abstract. Web interventions are gaining popularity in trying to change a person’s behavior. However, poorly designed intervention websites will affect the learning process of a person, what more to remember the content that they have learnt. The objective of this paper is to discuss the design of a web intervention using Gagne’s Condition of Learning Theory and cognitive dissonance. The usage of learning theory enables the designers to develop the intervention website based on users’ learning capabilities, thus placing users as priority. This ensures that users are able to maximize learning and recall of its content when faced with the decision to smoke or not. We are planning to recruit target users who are smokers, ranging from the 18 to 22 age range. Smokers will be categorized according to a baseline survey. Each category of users will go through the web intervention of different content. It is hoped that by combining the abovementioned theories, smokers are able to reduce their intake. Keywords: Smoking, Gagne’s Condition of Learning Theory, Cognitive Dissonance, Web Intervention, User Interface, HCI.
Design of a Web Intervention to Change Youth Smoking Habits
489
Bernhardt [1], the Internet is a new tool which can revolutionize health education. The web is flexible; hence delivery of health messages can be instantaneous and tailored to the specific needs, characteristics, and preferences of the user [1]. Because health education is not always a desirable topic to be discussed by the younger generation, health educators seem to embark on other/more advanced technological methods to gain students’ interest in learning about a health topic, specifically topics which could generate controversy, for instance alcohol consumption, sex education and even smoking. According to statistics, Malaysia had 3.6 million smokers in 2000 and this number is expected to increase to 4.6 million by 2025 [15]. The Malaysian government is trying to stem this increase by forbidding the sales of cigarettes to individuals less than 18 years of age. In addition, non-smoking campaign such as ‘TAK NAK’ (Say No to Smoking) has also been implemented nationwide. However, the effectiveness of non-smoking campaigns was never at a satisfactory level for the huge amount of money invested in promoting the campaign. [15]. The study reported in this paper therefore seeks to propose the use of a web-based educational approach as an alternative approach towards educating smokers on the adversities of smoking. Research indicates that information technology is a novel platform to reach out to the public. It has also been deemed as a cost-effective way to educate the public on issues concerning health. These traits are seen as instrumental in developing websites that are individualized and tailored to the needs to each patient, depending on their history, addiction patterns, age, race etc. Computers have made it possible to easily incorporate personalized feedback or tailored messages into a print or Internetgenerated intervention. [9]. For health educators, the practical implication of this research is that mass media channels are appropriate for creating awareness, but interpersonal interactions are essential for persuading individuals to adopt healthpromoting behaviors [3]. As this project is still in the preliminary stage of development, this paper will discuss the interface design and the experimental design of the proposed web intervention.
2 Related Works Information about smoking can be obtained from almost anywhere; Internet, mass media, brochures, cigarette packages; however most of the messages are being ignored as the medium used to disseminate this information is considered “old fashion”. Furthermore, the information is seen as repetitive and boring; hence people tend to be ignorant over time. According to [2] as cited in [3], mass media is an effective way to reach and inform large number of audience, but it was found that interpersonal channels are more successful in influencing attitudes and motivating behavior changes in a person. Rather, patients/users prefer information that is personalized to their individual needs and situations [11]. Previous research indicates that tailored information is more likely to be read, remembered, and experienced as personally relevant, which in turn has a greater impact in motivating patients to make a behavior change [4]. If tailored messages increase the attention span of users, which thus educate them, information seekers will then benefit from using tailored health education online [1]. Hence, a web-based tailored application on smoking may be used as an innovative
490
K.N. Goh et al.
intervention or prevention tool for teenagers who have different smoking patterns and personal characteristics (e.g. age, hometown, race, gender etc.). Many works have been done regarding intervention websites for smoking [19], alcohol [12], skin cancer prevention [1] and others. Different design methodologies have been adapted into these researches, for example, developing content which follows certain theory [20] or even creating websites that dynamically change the message content and design tailored for individual recipients to increase the user’s attention span [1]. Although the message content has been tailored, many individuals still do not pay enough attention to the health message they received [1]. Hence the focal point of this work will not only tailor the message to each individual, but also ensure that the screen is designed in a way that enables the individual to remember the information easily. This design is based on the learning theory described in section 2.1. Section 2.2 will then discusses how we can use cognitive dissonance, a behavioral theory, to develop our message. 2.1 Gagne’s Condition of Learning Theory A learning theory has been adapted to develop an intervention website that persuades changes in a person’s behavior. Gagne’s conditions of learning theory stresses on the different kinds of learning levels in individuals’ [7], namely (1) verbal information, (2) intellectual skills, (3) cognitive strategies, (4) motor skills and (5) attitudes. Hence, to cater to a wide group of learning styles, the 9 instructional events that Gagne outlines plays an important role for designing the intervention website [7]: 1) 2) 3) 4) 5) 6) 7) 8) 9)
Gaining attention Informing learners of the objective(s) Stimulating recall of prior learning Presenting the stimulus Providing learning guidance Eliciting performance Providing feedback Assessing performance Enhancing retention and transfer
An important component in Gagne’s theory emphasizes on selecting the right media for designing the instructional events. This is related to the concept of transfer of learning, which was introduced by Edward Thorndike and Robert S. Woodworth in 1901, where designers ought to design website which uses media elements that enable the transfer of knowledge easily from past experiences of the users [14]. For example, media elements that relate well to the target audience need to be used, where they have experienced those elements before and seeing them again in the intervention website will enable them to easily learn and respond to those media, when necessary. For example, cell phones are gadgets that every young adult owns nowadays, thus, if the web intervention contains an animated cell phone media, the user would easily transfer the knowledge from the real world to the virtual environment. For users of this intervention website to maximize their gain and share knowledge easily, low road transfer would be a better option as compared to a high road transfer when designing the website [14].
Design of a Web Intervention to Change Youth Smoking Habits
491
2.2 Cognitive Dissonance Cognitive dissonance theory was first introduced by Leon Festinger in 1957. This theory suggests that people are motivated to reduce their cognition dissonance by changing their attitudes [5]. A study done by Rachel D. Graham shows that people experience cognitive dissonance after making a difficult decision concerning morality [6]. A number of works done using cognitive dissonance theories [8, 13, 16], indicate that this theory has always been applied to making decision based on two options. Between these two options, there are self-debated consequences attached to oneself. Regardless, when someone makes a decision that is against their own judgment, cognitive dissonance will increase. A number of studies were done focusing on cognitive dissonance among smokers [13, 10, 17]. Cognitive dissonance theory is a theory which is synonym in explaining the complexity of an individual who smokes. In 1991, McMaster and Lee wrote regarding cognitive dissonance in tobacco smokers among Australian smokers. Based on their study, they conclude that smokers may experience cognitive dissonance because they use tobacco despite its well-publicized ill-effects. Therefore, they wrote that interventions targeting rationalizations for smoking will be useful in smoking cessation [13]. Meanwhile in 1994, Halpern wrote regarding the effect of smoking characteristics on cognitive dissonance in current and former smokers. He concluded that on overall, current smokers exhibited more cognitive dissonance than former smokers with regards to smoking-related belief [10].
3 Design 3.1 Interface Design As mentioned by Gagne, Briggs and Wager [8], the abovementioned instructional events should be satisfied, which serves as a basis for designing instructions and selecting appropriate media. An example of how the nine events are applied in the intervention website is shown in Table 1. Table 1. Example of events for learning theory Events 1. Gaining attention 2. Informing learners of the objective(s) 3. Stimulating recall of prior learning 4. Presenting the stimulus 5. Providing learning guidance 6. Eliciting performance 7. Providing feedback 8. Assessing performance 9. Enhancing retention and transfer
Example Ask users what are their most important values. Tell the users why it is important to reduce/eliminate smoking habits. Review what they have learnt previously. Give the users the facts of smoking. Give the importance / facts of each value they selected. Ask them why they smoke. Provide feedback based on their answers. Provide solutions to problems they face. Recall the importance of their values.
492
K.N. Goh et al.
Referring to Table 1, in the event of gaining attention, we can ask the users to rate their most important core values in life from choices like family, friends, finance, health, self-confidence and others. From these ratings, the web intervention is able to capture their important core values and reemphasize them throughout the web intervention and tailor the messages accordingly to remind them of the values they have chosen. This can also enhance their retention and transfer according to Gagne’s learning theory. In presenting stimulus, we could ask the user a series of questions that will reaffirm their knowledge of smoking, for example some questions to identify between fact and fiction. Some example of questions are “People gain weight when they quit smoking.”, “Quitting smoking will be hard.”, “I have too much stress in my life to quit right now” and others. 3.2 Experimental Design Fig. 1 shows the design process of this web intervention. Pre-screening questions will be given to purposefully sampled smokers to narrow down the data scope to be collected. Some examples of pre-screening questions are demographic questions, family history of smoking, what influenced them to start smoking and how many sticks of cigarettes they smoke in a day. The collected results will then be screened to categorize these students into three main categories: (1) heavy, (2) moderate and (3) low smokers based on certain criterias. For each category, the respondents will be given different sets of experiments, which will contain different approach and content. Experiment A, B and C will contain similar interface screens, but the content of message will be different as the tailoring approached will be used. Similar interfaces will ease in web development, but new interfaces development may be necessary to enhance delivery depending on content. The target respondents are undergraduate students in a local private university in Malaysia aged between 18-22 years. Students enrolled in the same program will be selected for easy tracking of these students where a follow-up study will be conducted after a period of 6 months. A total number of 30 students were chosen based on their smoking history rather than random sampling for better filtering. Students will be given a span of two weeks to complete the web session. Each session should last for about 30 minutes. By going through the series of web intervention sessions, it is able to educate the students about smoking habit and provide them with information about smoking. The web sessions will mostly provide information that is tailored to each student so
Fig. 1. Design Process
Design of a Web Intervention to Change Youth Smoking Habits
493
that information can be individualized. By this method, students will feel that the information is more relevant and suited to their needs, based on their response given during the web experiment [9]. After the implementation of this study, it therefore seeks to experiment how interface design can affect or influence change in a person’s behavior (from breaking a bad habit) without the use of external tools such as nicotine replacement therapy. This experiment may not be able to entirely eliminate the smoker’s smoking behavior but it reduces their amount of intake. For example, if they smoke one packet (24 sticks) a day, they can reduce their intake to 10 sticks a day.
4 Conclusions The number of smokers has not decreased for the past decades. Smoking relateddiseases kill one in ten adults globally, or cause four million deaths. If current trends continue, smoking will kill one in six people by 2030 [18]. It is believed that if individuals do not get into the habit of smoking in their youth, they would probably never smoke as adults [15]. There are many reasons why smoking starts so early among the youngsters. One of the main reasons is the promotion by tobacco companies that projects a positive and appealing image of smoking. Growing information about the enjoyment and satisfaction derived from smoking projected by the mass media has all contributed to obscure the fact that smoking endangers health. It is therefore believed that by combining both learning theory and cognitive dissonance in designing the user interface of a web-based tailored application on smoking, the young smokers would be able to reduce their amount of cigarette intake. The intervention study is currently underway (as of January 2009) and the effectiveness of the website in changing the students’ smoking behavior will be reported in a separate publication.
References 1. Bernhardt, J.M.: Tailoring Messages and Design in a Web-Based Skin Cancer Prevention Intervention. The International Electronic Journal of Health Education 4, 290–297 (2001) 2. Backer, T.T., Rogers, E.M., Sopory, P.: Designing health communication campaigns: What works? Health Communication on the Internet: An Effective Channel for Health Behavior Change? Journal of Health Communication 3, 71–79 (1992) 3. Cassell, M.M., Jackson, C., Cheuvront, B.: Health Communication on the Internet: An Effective Channel for Health Behavior Change? Journal of Health Communication 3, 71–79 (1998) 4. Davis, S., Abidi, S.S.R.: Adaptive Patient Education Framework Featuring Personalized Cardiovascular Risk Management Interventions. In: Wade, V.P., Ashman, H., Smyth, B. (eds.) Adaptive Hypermedia and Adaptive Web-Based Systems. Springer, Heidelberg (2006) 5. Festinger, L.: A Theory of Cognitive Dissonance. Stanford University Press, Stanford (1957) 6. Graham, D.R.: Theory of Cognitive Dissonance as it Pertains to Morality. Journal of Scientific Psychology, 20–23 (2007) 7. Gagne, R.M.: The Conditions of Learning Theory of Instruction. CBS College Publishing, New York (1985)
494
K.N. Goh et al.
8. Gagne, R., Briggs, L., Wager, W.: Principles of Instructional Design, 4th edn. HBJ College Publishers, Fort Worth (1992) 9. Hageman, P.A., Walker, S.N., Pullen, C.H.: Tailored Versus Standard Internet-delivered Interventions to Promote Physical Activity in Older Women. Journal of Geriatric Physical Therapy. Journal of Geriatric Physical Therapy 28(1:05), 28–32 (2007) 10. Halpern, T.M.: Effect of Smoking Characteristics on Cognitive Dissonance in Current and Former Smokers. Addictive Behaviours 19(2), 209–217 (1994) 11. Hoffman, T., Russell, T., McKenna, K.: Producing computer-generated tailored written information for stroke patients and their careers: system development and preliminary evaluation. International Journal of Medical Informatics 73, 752–758 (2004) 12. Linke, S., Brown, A., Wallace, P.: Down your drink: A web-based intervention for people with excessive alcohol consumption. Alcohol and Alcoholism 39(1), 29–32 (2004) 13. McMaster, C., Lee, C.: Cognitive Dissonance in Tobacco Smokers. Addictive Behaviours 16, 349–353 (1991) 14. Perkins, D., Solomon, G.: Transfer of Learning. In: Postlethwaite, T.N., Husén, T. (eds.) International Encyclopedia of Education, 2nd edn. Pergamon Press, Oxford (1992) 15. PROSTAR (Healthy Programme Without AIDS for Youth), http://www.prostar.gov.my/JR_Rokok_BI.htm 16. Radhakrishna, G., Saxena, A.: Application of Cognitive Dissonance Theory to Reduce Dropouts in Distance Education System. In: ICDE International Conference, pp. 1–5 (2005) 17. Simmons, V.N., Webb, M.S., Brandon, T.H.: College-student smoking: An initial Test of an Experiential Dissonance-enhancing Intervention. Addictive Behaviours 29, 1129–1136 (2004) 18. World Health Organization Smoking Statistics, http://www.wpro.who.int/media_centre/fact_sheets/ fs_20020528.htm 19. Lenert, L., Muñoz, R.F., Stoddard, J., Delucchi, K., Bansod, A., Skoczen, S., Pérez-Stable, E.J.: Design and Pilot Evaluation of an Internet Smoking Cessation Program. Journal of the American Medical Informatics Association 10(1), 16–20 (2003) 20. Lucero, A., Zuloaga, R., Mota, S., Muñoz, F.: Persuasive Technologies in Education: Improving Motivation to Read and Write for Children. In: Persuasive Technology, pp. 142–153. Springer, Heidelberg (2006) 21. Aggarwal, A.K., Bento, R.: Web-based education. In: Aggarwal, A.K. (ed.) Web-based learning and teaching technologies: Opportunities and Challenges. IGI Publishing, USA (2000)
Smart Makeup Mirror: Computer-Augmented Mirror to Aid Makeup Application Eriko Iwabuchi, Maki Nakagawa, and Itiro Siio Ochanomizu University 2-1-1 Otsuka, Bunkyo-ku, Tokyo 112-8610, Japan {iwabuchi.eriko,nakagawa.maki}@is.ocha.ac.jp, [email protected]
Abstract. In this paper, we present a system that aids people in wearing makeup easily and make the process enjoyable. We call the proposed system the “Smart Makeup Mirror”, which is an electronic dressing table that facilitates the process of makeup application. In this system, we place a high-resolution camera above a computer display and have added some functions such as “Automatic zoom to a specific part of the face”, “Display the face from various angles”, “Simulation of lighting conditions”, and “Internet voting on better makeup results” to facilitate the makeup application process. People who use this device for applying makeup will obtain highly satisfactory results, while enjoying the process.
2 Smart Makeup Mirror Figure 1 shows the over view of the proposed system. As part of this system, we place a high-resolution camera (Point Grey Research, Grasshopper, with IEEE-1394b connection, 1624 x 1224 pixels, 30 fps) above the computer display. The camera is trained on the woman, and a mirror image of the woman is shown on the screen; thus, the device functions and assists the woman in applying makeup effectively while looking at her image. There are some researches that use a standard-definition TV camera and display images as a mirror would [2][3]. In the proposed system, we use a high-resolution camera in order to enable practicable makeup application. In order to realize the functions described later in the paper, we processed the images captured by the camera. We set up a low-resolution camera (Logicool, Qcam Pro for Notebooks, in a 320 x 240 pixel mode, 30 fps) for such image processing. We used OpenCV in the image data processing and FlyCapture for the controlling the high-resolution camera. These cameras are connected to a PC (Gateway GT5092j) with Windows Vista OS. Moreover, we set up an infra red range sensor and a proximity sensor for performing the following functions. These devices are controlled with a microcomputer, and communicate with the PC program through serial connection. We developed the following functions to support the makeup application process.
Fig. 1. Overview of Smart Makeup Mirror
3 Makeup Support Function Using this system, users apply makeup while looking at the computer display where an image of the user’s face is captured by the high-resolution camera is displayed. This configuration is slightly different from a realistic mirror because the user’s gaze
Smart Makeup Mirror: Computer-Augmented Mirror to Aid Makeup Application
497
is not aligned with the image, because the camera takes pictures from the top of the display. The response of the image may be delayed because of computation. Further, the resolution is lower than that of a realistic mirror, and the image may be out of focus where a user moves considerably from the focusing position of the camera. These differences may obstruct the makeup application process. The purpose of this research is to implement an artificial mirror with functions that cannot be achieved by using a realistic mirror in order to compensate for the abovementioned differences. In order to estimate the influence of the abovementioned differences on the makeup application process, we applied makeup using an early-stage prototype of our electronic mirror. The following observations were made. y Satisfactory makeup application using the electronic mirror is generally possible. y However, a function that enables gaze alignment is required. When a user tries to see an enlarged view of her face and comes considerably close to the camera, the face image gets blurred as the camera does not have an auto-focus mechanism. From the above observations, we conclude that at least two additional functions are required to make the electronic mirror as effective as a realistic mirror in the makeup application process. One is to capture and display the user’s face from the front, and the other is to enlarge this image for ease of use. The details of these functions are explained in the following sections. 3.1 Automatic Zoom and Pan During the makeup application process, we usually bring our face close to the mirror in order to ensure that our makeup is satisfactory and to check the makeup near certain face parts such as the eyes and the mouth. This is a critical part of the makeup application process, is frequently needed, and is often the most troublesome as it requires a strenuous and awkward posture. For this purpose, we developed an automatic zoom function in the proposed system for zooming at a particular at part of the face. When a user brings the makeup tool, with an attached color marker (a small green sticky label), near her eyes, the camera recognizes the marker, zooms in, and switches to an enlarged image of the eye. In order to detect the color marker, it is not practical to use the high-resolution camera image because the processing time will increase significantly and the response will be unacceptable: moreover, the accuracy of position detection achieved by the high-resolution camera is not required. Therefore, we use an additional low-resolution camera for recognizing markers. This enables users to apply makeup without having to physically approach the display. 3.2 Intuitive Magnification Rate Control A professional makeup artist recommends having at least one magnifying glass at hand as it aids in carrying out a thorough checkup of the applied makeup [4]. We developed a “magnifying glass function” in order to provide finishing touches to the makeup application process, thereby improving the quality of the applied makeup. The proposed system uses an infrared range sensor to measure the distance between the camera and the face. Further, the magnification rate of the display image can be
498
E. Iwabuchi, M. Nakagawa, and I. Siio
increased/decreased by simply and slightly approaching/leaving the display. A similar technique was recently used for a document browser [5]. 3.3 Left and Right Reversing Mirror A professional makeup artist will always recommend checking a person’s appearance from the viewpoint of another person standing in front, with the help of a left and right reversing mirror (the reversal mirror) [4]. The impression of the hairstyle: the shape of the eyebrow, eyes, and lips: and the positions of moles are different, when the face is seen in a reversal mirror. Therefore, we added this functionality to the proposed system, enabling the system to as display the image captured by the camera as a left and right reversing mirror would. 3.4 Profile Check In everyday life, the face is observed not only from the front but also from the side, from the bottom, and in the diagonal direction. Thus, while applying makeup, we must inspect the face from various angles. In the proposed system, the camera captures the image of our face and displays it for several seconds. By looking at that picture, we can inspect the makeup on the face from various angles. As a result, it is possible for us to check our profile and our view from the back, which is not possible to observe in a conventional mirror. When a user captures her face while she is looking at the camera, she can check the gaze-aligned view of her face that a conventional mirror always provides. 3.5 Lighting Mode It is usual that to apply makeup in a place that is as bright as possible. However, the makeup of a user and hence her appearance may appear different under different lighting sources such as sunlight, fluorescent lamps, and incandescent lamps. This system simulates the lighting conditions depending on the surroundings. (Fig.2) The lighting conditions are makeup mode, office mode, cloudy mode, clear weather mode, red sunset mode, and candle mode. The selected lighting mode is displayed in the title bar of the interface and can be confirmed easily. 3.6 Makeup Log When we change cosmetics, tools, or the method of makeup, it is difficult to find differences in the face between how our face looked with the old makeup and how it looks with the new makeup. Moreover, it is also difficult to see our own face objectively in a conventional mirror, because we usually hold a good expression in front of the mirror. The smart makeup mirror allows us to save the image being displayed to a computer file with a particular file name and date. By using this function, we capture an image of our face after applying makeup. This will help us to accurately compare the color, brightness, or texture that suits our face by capturing an image every time in the same environment.
Smart Makeup Mirror: Computer-Augmented Mirror to Aid Makeup Application
499
Fig. 2. Differences in the face appearance in different lighting modes
3.7 Noncontact Operation While applying makeup, our fingers become dirty and it is undesirable to touch anything with the dirty hand. All the operations of the abovementioned functions can be operated without touching any part of the system. Automatic zoom to a specific part of the face and intuitive magnification rate control have already been achieved by noncontact by using a color marker (makeup tool) and the range sensor (position of the body). We attached four proximity sensors to the right of the display. On bringing our hand close to the proximity sensor, we can operate the left and right reversing mirror, profile check, lighting mode, and makeup log functions.
500
E. Iwabuchi, M. Nakagawa, and I. Siio
4 Evaluation Test 4.1 Test by the Author One of the authors (a 23-year-old woman) applied makeup by using this system. Figure 3 shows the steps from “no makeup” to “full makeup and hair arrangement”. The author’s opinions are listed below: y It is easy to line the lower eyelid but not the upper eyelid. y For some makeup activities, the timing of zoom switching and position of color markers need to be adjusted. After considering these findings, we have improved the timing of zoom switching, and the position of the marker.
Fig. 3. Steps from “no makeup” to :full makeup and hair arrangement.”
4.2 Test by Ordinary Women We have carried out an evaluation test (Fig. 4). Two women (age: 23 and 25 years) participated in the test. They are students in our laboratory and familiar with how to operate a computer. They applied eye shadow, eye liner, eye blowing out, mascara, lipstick, and blush on. The results are given below: y The two women found the proposed system easier to use than they expected. y They did not like the fact that they could see spots and dullness more clearly in the electronic mirror than in a conventional mirror. y They liked the automatic zooming function. y They occasionally lost sight of the spot where they were applying make up when the system switched to an enlarged image. y They found the left and right reversing mirror function useful for checking of their appearance. y They also found the ability to change the lighting mode useful.
Smart Makeup Mirror: Computer-Augmented Mirror to Aid Makeup Application
501
y They found it easy to apply makeup to the lower eyelid, but difficult to do so on the upper eyelid. y They found it difficult to use the eyelash curler because of the motion delay caused by the computation. Difficulty of applying makeup to the upper eyelid was pointed out by both the author and the two subjects. We will describe the improvement related to this issue in the following section.
Fig. 4. Evaluation tests by two women
5 Adjusting Position of Display In order to decrease the difficulty in applying makeup to the upper eyelid, we carefully observed users when they applied makeup to the eyelid. When they applied makeup to the upper eyelid using a mirror, they wanted to see their face from below. For this purpose, they raised their head and lowered their line of vision. (Fig. 5 left) Conversely, when they applied makeup to the lower eyelid, they lowered their head and glanced upward (Fig. 5 right). In the prototype system, the image is displayed at the upper part of the display: therefore, users felt that it was easier to apply makeup to the lower eyelid than to the upper eyelid.
Fig. 5. Application of makeup to upper eyelid (left) and lower eyelid (right)
502
E. Iwabuchi, M. Nakagawa, and I. Siio
This problem could be solved by changing the display position of the image according to the makeup activity to the lower and upper eyelids. Thus we have implemented a function that displays a magnified image of the eyes at the upper part of the display when the user applies makeup to the upper eyelid, and displays the magnified image at the lower part of the display when the user applies makeup to the lower eyelid. The user’s activity of applying makeup to the upper or lower eyelids is detected by means of computer vision through the low-resolution camera. Thus, users easily apply makeup not only to the lower eyelid but also to the upper eyelid.
6 Internet Voting By using the makeup log function, a user can capture an image of her face after applying makeup. Therefore, she can accurately compare the color, brightness, or texture that suits her face by capturing an image every time in the same environment. However, it is not sufficient to just find the makeup method suited to the user. Even if a user thinks that the one of the makeup results is suitable for her, other people may not think so. Then, we have developed a web page function that the user’s friends can use to judge whether the makeup suits the user (Fig. 6).
Fig. 6. Web page where users’ friends can vote
Smart Makeup Mirror: Computer-Augmented Mirror to Aid Makeup Application
503
The picture of a face after applying makeup is open to the public on a web page that is password protected. The page shows pictures of a user’s face in a favorable order. User’s friends who are given the password can change the order of pictures with the arrow buttons on the page. They can arrange better pictures above worse pictures. By using this web page, a user can get useful opinions from her friends in an anonymous manner. This page would provide objective opinions on her makeup.
7 Conclusions and Future Work We have proposed and implemented an electronic mirror that facilitates the process of applying makeup and makes it enjoyable. During the feasibility test with a female subject (age: 23 years), we have received favorable comments for the system. We carried out the evaluation tests with two women (age: 23 and 25 years). Further, we improved the system on the basis of the result of the tests. Additionally, we developed a web page function that allows friends to vote on a makeup result that suits the user the most. We are planning to verify the utility of this system by using it for the long term.
References 1. Takagi, S., Namikawa, C., Yoshimoto, F.: Advice System for Progress in Makeup Skill. The Journal of theSociety for Art and Science 2(4), 156–164 2. Kawauchi, H., Inoue, A., Ichimura, S.: Electronic make-up mirror that supports point make-up in real time. In: The 69th national convention of IPSJ, pp. 4-201–4-202 (March 2007) 3. Furukwa, T., Tsukada, A.: Magic Makeup Mirror – Makeup Simulation based on Real-time Face Recognition. Imagelab 13(10), 34–38 (2002) 4. Watarai, H.: The true basis of make-up. Studio Tac Creative Co., LTD, Tokyo (2006) 5. Harrison, C., Dey, A.K.: Lean and zoom: proximity-aware user interface and content magnification. In: Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing System, CHI 2008, Florence, Italy, April 05-10, 2008, pp. 507–510. ACM, New York (2008)
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work: The Case of IT Service Delivery Eser Kandogan, Eben M. Haber, John H. Bailey, and Paul P. Maglio IBM Almaden Research Center, 650 Harry Rd., San Jose, CA 95120 {eser,ehaber,baileyj,pmaglio}@us.ibm.com
Abstract. IT service delivery is challenging to study. It is characterized by interacting systems of technology, people, and organizations. The work is sometimes reactive, sometimes carefully planned, often risky, and always complex and collaborative. In this paper we describe how we’ve learned about IT work, using a variety of methods including naturalistic observations, contextual interviews, surveys, and diary studies. We provide examples of our study results, showing what we’ve learned with the different methods. We argue that to effectively study such systems, a variety of methods may be needed to complement insights and validate findings. We found that naturalistic observations were extremely time and labor intensive, yet offered us the time and space to observe unplanned events and long-lasting tasks, bringing out the full complexity and risks involved in real work. Contextual interviews and diary studies provided fewer details, yet gave a broader context to individual’s work. Surveys provided an even broader picture, going beyond individual differences, yet they were limited by details and issues of sampling.
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work
505
administration work, and they ignore the everyday constraints and issues that make the work dynamic and reactive to the unplanned events. We believe short lab studies offer only limited insight into use cases and fail to sufficiently explore the design space. We found ethnography a good fit for studying IT administration work. Naturalistic observations and contextual interviews conducted in the field allow inquiry of phenomenon within its real-life context, allowing time and space to see unplanned, transient and long-spanning events and tasks, all of which we found to be characteristic of system administration work. Field studies allow the observer to be situated in the work environment for extended periods, and they help facilitate in-depth study of the development, adoption, and use of new tools, practices, and organizations. We complemented field studies with diary studies and surveys to validate our findings across broader time scales and populations, though we achieved less detail with these methods. In this paper, we present examples from our findings to demonstrate the benefits and challenges in our study methods.
2 Studies of IT Services We conducted a series of studies in IT service delivery organizations employing methods including naturalistic observations, contextual interviews, a survey, and a diary study. In 16 site visits to large corporate, university, and government data centers across the United States, we observed and interviewed more than 30 IT workers. We observed the work practices people involved in management of security, databases, web sites, storage, operating systems and data center operations [1]. Our methods were as follows: y Contextual Interviews: 30-90 minute interviews with subjects at their desks, inquiring about their work and encouraging them to show us examples of important tools, techniques, and artifacts they use on a daily basis. y Naturalistic Observations: following, observing, and if possible recording all the work-related activities of an administrator for anywhere from a few hours to a week. This usually involved two researchers, one to operate audio/video recorders, and the other to take notes and occasionally ask questions (though trying to avoid interrupting the flow of work). We asked participants to speak aloud if possible, and to put phones on speaker. At the end of each day, we asked clarifying questions about the observations from that day. Additionally, we collected physical and electronic materials and took pictures their work environment. y Surveys. To validate our findings across a wider population, we administered one survey on tool use to 100 administrators. We have also worked with the SAGE organization which performs an annual survey on 1000-10,000 sysadmins, looking through their data for further correlations, and also adding a few new questions. y Diary Study. One of our observation subjects was willing to share his 10-month diary listing, for each day, all the work-related activities he had performed that day. Below we present samples from our findings to illustrate the results from each method. The first case shows how naturalistic observations were able to illuminate the extremely complex world of security administrators at a university computing center. In the second case, we employed contextual interviews to examine an organization
506
E. Kandogan et al.
delivering storage management: Centralized Storage Services (CSS1). Here, contextual interviews helped us gain insight into the workings of the larger organization through descriptions of work processes that could take as long as a month with multiple people across different organizations. In the third case we describe our findings from our survey of trust issues in the design of system administration tools [6], which helped validate and refine some of our observational findings. We also describe some of our experience working with the data from the SAGE survey. Finally, we discuss a diary study that provided long-term validation for our observational findings with respect to collaboration between administrators. Let’s describe each in more detail. 2.1 Case 1: A Game of Cat and Mouse… We conducted two separate week-long observations of the five-member data center security team at a large public university in the United States. We saw how their dayto-day work included continual monitoring for new security threats, policy formation, and helping remove security vulnerabilities. We also got to watch a notable episode, as they responded to a world-wide security incident, Stakkato. Stakkato had persistently attacked military, educational, and government sites across the United States and Europe. We observed the senior security administrator, Joe, handle this incident through both local work and also collaboration with other educational and government institutions. Collaboration was necessary because the attacks came through a complicated maze of computers in as many as 7 countries, making tracking very difficult. Joe was working with Aaron, a junior security administrator, and others to respond to the attacks. This was very much a game of cat-and-mouse: the attacker would use vulnerabilities to gain access to machines, while the security admins knew about vulnerabilities in the attacker’s tools that they would use to try and trace the attack back to its source. Often a compromised machine would be left vulnerable to allow more time to trace the attacker. While the center had not been attacked for couple of weeks, the attacks remained a primary concern for Joe, and appeared to have become a personal issue for him. Thus Joe seemed sad and frustrated when he received the news that a compromised machine at another institution was turned off – he wanted the machine to stay on so he could continue to use it to trace the attacks, yet the institution that owned the machine didn’t want to leave it vulnerable any longer. Tracing attackers was no easy task. It was technically challenging as security admins needed to exploit the particular techniques the hackers were using. On a social level, it was even more challenging as they needed to coordinate work with several sites and develop an understanding to share information that could be sensitive. Security staff at a number of institutions held weekly calls and exchanged information, tools, and strategies through email, phone, etc. Joe played an important role coordinating the effort and communicating their findings: Joe: Usually they [the attackers] come in from Europe to a machine in the US, and they make either one or two more hops before they start launching any attacks. We had that narrowed down to where they are coming in, but that site wasn’t able to help 1
To protect the identity of the people, organizations, and companies, we used fictitious names throughout the paper.
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work
507
us. So, we knew that machine, we knew the second machine they are hopping to, and there is a third machine that was here in town that they are using as well. I need to call them today, and I need to track down some of these other sites, the admins there. The particularly challenging issue was that the attackers kept coming back relentlessly using new malware. Upon reviewing session logs Joe noticed a number of new exploits and asked Aaron’s help to find more out about them. Joe and others at the center used an elaborate directory structure to collaborate on incidents. They had directories for each incident to keep scans and exploit code organized: Joe: They have two new things in there, that I didn’t notice. I told them about identity key there (pointing to ingresd.x.x.edu) and I am not sure what it is from. They are trying to figure out what it is from as well. […] But that mod_rootme, we need to find out what that is. […] The other thing they are using is that ussl thing. Those are two new tools, I haven't seen them used before. Aaron’s research on several security sites revealed that mod_rootme was a highthreat vulnerability of Apache Web Server. It allowed a remote user to get a root shell without being noticed as communication was disguised as normal web traffic. He also found a copy of the exploit source code and noticed a couple of identities there: printf("[*] named 8.2.x ( < 8.2.3-REL) remote root exploit by lucysoft, Ixix"); printf("[*] fixed by [email protected] and [email protected]\n\e");
A web search on these identities, i.e. lucysoft, [email protected] and [email protected], led him to the BIND exploit. Aaron explained that while it was very difficult to understand source code, references to identities often led him to further clues on the exploits: Aaron: […] and that is what I generally look for. Like most of them have assembly code or something else. Rather than spending time on what exactly the exploit is doing, first I try to look at the comments, explanations, and signature of the authors. Because generally all the hackers they tend to write their own signature handle. Aaron also noticed another source code (a.c) in the sessions which he discussed with Joe, later in his office: Joe: Okay. Where was that a.c? Was that in there?? Aaron: Uhh. Yeah, it is in canopy. Joe: So, this is a BIND one, huh? We should be able to build, put that together. Aaron: main.c and main.h Joe said they would often try these exploits in quarantine environments on virtual machines with limited connectivity and learn how they worked. When Aaron got back to this office and began working on other tasks, he got an email alert from the intrusion detection system, Bro. Aaron and his colleagues used several automated systems that scanned network traffic and computer activity for suspicious events. This alert was about new activity on a formerly-compromised host: Watchdog found the following alerts in tcpread. These seem to be coming from the known compromised hosts. Please take time to investigate. Non-XX IP (Once Compromised IP’s) x.x.31.10 #nyx.engine.xx.edu (6/8 from victor, used as login to XX) XX IP’s Connections > Jul 27 15:10:14 x.x.31.10 0.1kb > x.x.63.22/http 711kb 0.0b %77125
508
E. Kandogan et al.
The alert stated that there had been recent network activity, an HTTP transfer of a file of size 711kb with the log ID 77125, to a local machine. This was a machine reported as compromised by Victor, a fellow security administrator at another site, on 6/8. Aaron had customized the intrusion detection system to register certain hosts as compromised and added further contextual information such as who reported it and when and how it was compromised in the past. This alert was of utmost importance as the host in question was directly related to the Stakkato incident. When Aaron examined the HTTP logs he found out that a particular file had been downloaded. Aaron first identified the host name, and then searched for the owner of the machine on an online database. Aaron immediately did a search and found his home page. Examining web page he discovered that the user worked on high performance computing, making the file in question a legitimate download. Both Tom and Aaron were relieved as this potentially serious alert turned out to be a false alarm. Responding to a security attack required a range of activities, from minute-byminute monitoring of activities to long-term research, planning, and collaboration across multiple sites. Security administrators have a fascinating culture, always trying to learn about vulnerabilities of the enemy, while preventing the enemy from realizing what is known about them. Naturalistic observation was very helpful in enabling us to gain a detailed understanding of the full richness of this environment. 2.2 Case 2: Everybody Thinks That They Are Number One We conducted a three-day study with several contextual interviews in the Centralized Storage Services (CSS) group at a large IT Service company. CSS offers enterprisescale storage solutions centered at one location, with managed storage resources distributed over 10 sites across the world. With a novel offering that delivered ondemand storage capacity allocation, 24 by 7 monitoring, and management services, CSS grew its customer base from only 2 customers with 2 high-end “SHARK” storage systems to 27 customers with about 50 SHARKs, just within two years. We interviewed Henry, who worked in the Storage Operations Center (SOC) as the storage team “focal” (i.e., project lead) since CSS’s infancy. Henry’s group handled anything having to do with storage, from technical work such as updating microcode software to coordinating work such as negotiating change windows with customers. Henry explained that customers got more or less equal treatment from CSS, though certain tasks took a higher priority: Henry: Everybody thinks that they are number one, you know. So, our model is that nobody is given priority over anybody else. Yes, we set a priority over how it has to be done cause basically if there is an outage, lets say, filesystem has reached 99.99%, and it is about to crap out and cause the server to break down, we may have to push doing that allocation before that one. Nonetheless, with rapid customer growth they hit technical limits that required significant human intervention, as illustrated by one incident the previous week: Henry: The SHARK has a limitation that allows you so many port logins. We hit that limit. So, support came back and said that the only fix for that was to recycle [reboot] the host bays. So, now you gotta get approvals from the customer. That is a problem. We had like 50-60 servers, which has a different application for each one. You gotta get
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work
509
approval from all these people. So, we finally got approval. Problem was fixed but it took like a week and a half to two weeks away, just to get approvals. Customer is happy now. They see the storage, but we hit one of those limits that is in a book like that thick. Rapid growth not only brought technical limits but also organizational issues to deal with. A simple setting, maximum number of port logins, when exceeded required approvals from each customer. While customers were isolated from each other in terms of service quality, we saw that collective use of shared storage resources led to new issues that would not have risen otherwise. In CSS, they aimed for no outages and downtimes. This meant that operations needed to be performed concurrently, i.e. without taking the whole cluster down. One of the critical issues in concurrent operations was microcode updates. To address this issue Henry and others developed a 4-week process, depicting tasks and interaction with customer as a flowchart. The first two weeks were primarily for aligning requirements and schedules with the customer. Handshaking occurred from both a technical perspective and social perspective: Henry: It is a four week process from the point we approach the customer. We establish the [change] window, we tell them here are the servers we see you have, storage on your SHARK, and levels of SDDs you running on these servers. A lot of times what you see on the SHARK is not what you get. So, you have to make sure there is a handshake between the zoning and the SHARK. So, between weeks 2 and 3 we do all that handshaking making sure that SAs are aware. Is it compliant, if not we tell you, hey are you willing to bring down your server? The remaining two weeks were primarily for performing technical work. Week 3 was for upgrades to meet minimum requirements. Week 4 was final check with the customer detailing schedules, contact information and technical details to give a last chance to withdraw. Once SOC performed the change they closed the ticket with a final report to the customer. Synchronization and coordination of work among parties always presented a challenge. Henry mentioned that when storage work was completed in SOC, it might still take time to bring up the servers to verify the change. And he found it always frustrating when customers could not see their storage and he had already moved to another issue. Remembering changes from weeks before was a burden, especially as administrators were not devoted a specific customer. To help with this they developed a database to keep track: Henry: Do you really remember what you did four weeks ago? We have a database that can house that information so you can pull it up easily. Because if I am working on customer A today, three weeks from now, customer C says, well, you worked on my stuff a month ago. Customer A is on my mind right now. I am not thinking about customer C! Henry described how most of these challenges at CSS were manageable through careful planning and processes developed in the SOC. Henry and others developed spreadsheets and flowcharts to guide work and coordinate with customers. Lags within the organization presented a special problem. Differences in storage design, allocation, and server board times were particularly difficult to deal with. Solving these issues required invoking organizational memory to recall problems, participants, and solutions to revisit and resolve them with the right people and tools.
510
E. Kandogan et al.
2.3 Survey Studies In our field studies we observed numerous cases where the tools failed to support system administrators, reporting incorrect data, working unreliably, or simply being poorly aligned with the administrator’s work practices. Given the risk factors in the IT administration we expected that trust played a major role in the way administrators used and interacted with information, people, tools, and systems. To see if these findings were valid across a broader population, we performed a survey of about 100 system administrators recruited through online news groups, and local and national system administrator organizations. In the survey we asked specific questions about administrators’ comparative qualitative judgments of the CLIs and GUIs for system management they use regularly. One set of questions concerned the perceived speed, ease of use, reliability, robustness, accuracy, trustworthiness, and likeability of the CLIs vs. GUIs they used for system administration. Additional questions about both CLI and GUI tools were based on McAllister’s survey, using a 7-point Likert scale, which measures monitoring behavior and cognition-based trust for interpersonal relationships in organizations. The details of the survey results are included in [6]. In sum, we found statistically significant preferences for command-line tools over graphical tools in the areas of trustworthiness, reliability, robustness, accuracy, and speed. In response to the McAllister questions, we found that cognition-based trust ultimately plays a major role as opposed to seemingly affective factors. We received some comments, such as, “I know what I am doing. Please NO MORE GUI! If people need a GUI they are not qualified to be doing whatever they are trying to do.” In search of further statistically grounded data, we also worked with SAGE, the System Administrators Guild, which annually administers a survey intended to collect demographic and salary information. This survey is very large, with between 680 and 9600 respondents depending on the year. The survey includes questions about supervisory roles, and also administrators per site, from which estimates of administrator team size can be inferred. Further, we were able to add some explicit questions about collaboration into the 2007 survey. The results suggest that administrators work in teams regardless of company size, and that the majority of them spend a third or more of their time working with others. We did need to carefully formulate the questions to make them easy to answer, since subjects often failed to answer complex or openended questions. 2.4 Diary Study George was a web-application administrator in a large corporate service delivery center. We observed him in two separate week-long field studies conduct several tasks such as installing web servers, configuring authentication servers, trouble shooting problems, etc. For his own purposes, he kept a record of tasks performed daily, such as troubleshooting (e.g. “Continued with webseal problems on acp2”), and meetings (e.g. “RFS customer call - 16 new servers coming in”), etc. On a typical day, he had about five to ten records. He did not, however, attribute the amount of time spent on each task. George was willing to share his diary with us. We analyzed it by categorizing tasks in each record, and by noting what tools and people were mentioned. Our purpose was to get a break down of tasks over an extended periods and get a sense of the
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work
511
extent of collaboration in conducting tasks. Our analysis confirmed that collaboration was indeed practiced extensively with about 23% of the records referring to meetings with other system administrators. Of the remaining tasks such as planning (21%) and trouble-shooting (11%) , the diary mentioned other admins nearly half the time. Across all tasks he indicated collaborating with others about 45% of the time. This was a high number even though we suspected that collaboration could have been downplayed in self-reports. We were able to evaluate self-reporting by comparing a 3 hour troubleshooting session we video-recorded with the diary for the same day. In coding the session, we observed him working with one of ten other people for 90% of the time. His report, however, discounted the extensive collaboration simply saying, “Communication problem with PD server - we had port opened in wrong direction were able to use extra port 7236 - works fine now.” The only indication of teamwork was mentioning “We” as opposed to “I”.
3 Analysis: Multiple Methods for a Complex Topic System administration work is clearly a challenging area to study. The work is sometimes reactive, sometimes carefully planned, often risky, and always complex and collaborative. We have found that a variety of methods is necessary to generate a complete and accurate picture of administration work: ethnographic approaches such as naturalistic observation and contextual interviews, as well as surveys, and diary studies. Consider the case of the security administrators reacting to alerts about potentially suspicious activity. It is hard to imagine any way of capturing the intricate and coordinated response and evaluation of the alerts, short of watching them unfold. Consider also the complexity of the technological arms race and information warfare going on between the security administrators and their adversaries. The admins are continually researching vulnerabilities in their own systems. They must also search for the often subtle signs of intrusion, and try to find vulnerabilities in the intruders’ tools. Attacks, when detected, are often permitted to continue, in order to trace them back to their source. Any data left behind by the attackers must be analyzed, reverse engineered, or even decrypted to find clues about the attack’s origin. Furthermore, attacks often come from compromised machines at other institutions, requiring collaboration across the broader community. Observation seems to be the only way to get a clear picture of this very complex environment. Contextual interviews are an important complement to naturalistic observations, they provide the context, history, motivation, and descriptions of exceptional or rare events that might be difficult to tease out from observations alone. Our interviews with Henry about his storage administration work gave us insights into the history of his group, and the clearly defined workflow that formalized the interaction among the participants. For example, Henry explained the elaborate flowchart for the four-week long interactions with the customer when updating microcode software. A significant part of the interactions focused on synchronization and coordination. We learned that rapid growth in the customer base led to technical issues, which in turn led to a weeks-long process to get approvals from all the participants involved and perform changes. Dynamic growth also required them to perform flexible resource allocation. This reflected in the way CSS got organized over long periods of time.
512
E. Kandogan et al.
An ethnographic approach was extremely valuable in studying IT service delivery because it allowed us to capture work broadly and flexibly in time and space with all the interdependent variables of technology, people, and organizations as they emerged during the course of study. Thus, we argue that to understand fundamental issues and formulate hypothesis of work in the context of larger systems of people and technology an ethnographic approach is invaluable. The most common alternative to ethnography, in-lab user studies, begin by eliminating or controlling variables, thus they are more suited to achieve shorter term goals such as measuring usability or task performance. Another alternative is modeling usage in a computational way in terms of actions and affordances. This approach does help to get statistical data on performance, yet it may be neglecting important aspects of experience [3, 4]. Ethnography goes beyond actions of individuals and considers cognition also from a social perspective. Thus, ethnographic studies are better suited for verifying hypothesis. Expecting similar short term goals such as design guidelines from ethnographic approaches is not the best criterion for the relevance, utility, and quality of an ethnographic account of work, and such an expectation indeed underplays potentially radical gains that can be achieved by laying out a broader design space [2]. Ethnography does have its limitations or challenges, too. First, ethnographic methods are expensive. It takes considerable time and commitment to conduct long-term observations. It also takes time to establish the trust of the members of the study group, who must put up with being observed, followed, and questioned. We found it considerably easier to obtain permission to observe and record employees within our own company; obtaining the legal agreements required to do such work at other companies proved to be extremely challenging. Once the data was collected, it took even more time to analyze: an hour of video might require a day to a week to truly understand. We even developed our own video analysis tools to speed up the process of video analysis and annotation. Ethnographic studies are not ideal for developing low-level design guidelines that necessitate quantification of user needs and performance. Absence of quantification, and lack of control for variables are often used to question the validity and reliability of results though both issues are addressable when ethnography is used to derive hypotheses. Validity is addressed by collecting data from diverse situations and sources and considering all to expand hypotheses instead of discarding any data that did not fit a hypothesis. Reliability is addressed by triangulation where data is collected from multiple sources, across situations, and time. Last but not least, ethnographic accounts are descriptive rather than prescriptive thus making multiple interpretations a possibility, a quality that can in fact be considered as an advantage. We faced several challenges as we conducted our ethnographic field studies. First, the work of the system administrators was very complex, and outside of our own areas of expertise. It was nearly impossible for us to track and understand commands, errors, and discussions surrounding them as the work was unfolding. Likewise, a lot of the work was dynamic and concurrent, many issues were being addressed at the same time, through simultaneous conversations among staff and interactions with multiple systems. To address these issues we utilized video analysis to help us better understand the work. However, it still required extensive effort afterwards to make sense of all the technical issues and unfold multiple concurrent interactions into coherent parts, and establish cause and effect. All this required meticulous attention to every detail, on the computer screen and utterances of the participants, as well as
Studying Reactive, Risky, Complex, Long-Spanning, and Collaborative Work
513
technical documentation studied after the fact. Tracking issues from multiple perspectives of the participants also helped to the extent possible. Unlike traditional anthropologists, we were not able to spend months or years with our subjects. This meant that our observations and interviews drew from a limited sample. Data sources such as George’s diary were extremely valuable in showing that collaboration practice was not an isolated case but occurred on longer time spans across different types of tasks. The extremely labor-intensive nature of naturalistic observation also left us with a limited population sample. Surveys were one means to validate findings across a broader population, especially when we could frame very specific questions to ask.
4 Conclusion The purpose of this article is to discuss the strengths and limitations of various study methods as applied to the dynamic, risky, complex, and long-spanning work environment of IT service delivery centers. We argue that observational methods offer both time and space to observe short-term reactive work and long-lasting phenomena and tasks with its full complexity with all considerations of the interplay of technology, people, and organization systems through detailed illustrations of two cases from our field work. We also describe how contextual interviews can provide further background, details, and historical information. We tried to address some of the shortcomings of ethnographic approaches through surveys and diary studies that offered validity beyond time and individual differences. Finally, we discussed challenges we faced, and made recommendations on possible ways to address them.
References 1. Barrett, R., Kandogan, E., Maglio, P.P., Haber, E., Takayama, L., Prabaker, M.: Field Studies of Computer System Administrators: Analysis of System Management Tools and Practices. In: Proc. of the CSCW 2004, pp. 388–395. ACM, New York (2004) 2. Dourish, P.: Responsibilities and implications: further thoughts on ethnography and design. In: Proceedings of the 2007 Conference on Designing For User Experiences (2007) 3. Norman, D.: Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books (2004) 4. Picard, R.: Affective Computing. MIT Press, Cambridge (1997) 5. Spohrer, J., Riecken, D.: Introduction. Communications of the ACM 49(7), 30–32 (2006) 6. Takayama, L., Kandogan, E.: Trust as an underlying factor of system administrator interface choice. In: CHI 2006 Extended Abstracts on Human Factors in Computing Systems, pp. 1391–1396. ACM, New York (2006)
Human Computer Interaction in Virtual Standardized Patient Systems Patrick G. Kenny, Thomas D. Parsons, and Albert A. Rizzo Institute for Creative Technologies University of Southern California 13274 Fiji Way Marina Del Rey, CA 90292, USA {kenny,tparsons,rizzo}@ict.usc.edu
Abstract. Interactive computer generated characters can be applied to the medical field as virtual patients for clinical training. The user interface for the virtual characters takes on the same appearance and behavior as a human. To assess if these virtual patients can be used to train skills such as interviewing and diagnosis they need to respond as a patient would. The primary goal of this study was to investigate if clinicians could elicit proper responses from questions relevant for an interview from a virtual patient. A secondary goal was to evaluate psychological variables such as openness and immersion on the question/response composites and the believability of the character as a patient. Keywords: Virtual Patients, Artificial Intelligence, Clinical Psychology.
Human Computer Interaction in Virtual Standardized Patient Systems
515
One of the main research questions of the virtual human work is in developing these characters so they appear and act like real humans without falling into the Uncanny Valley [3]. The Uncanny Valley was defined by Masahiro Mori in the 1970’s to describe robots and characters that look like humans, but don’t act like humans, or what we expect them to act like. As the realism of the appearance of the character approaches human appearance, for example the face, eyes or skin, while the actions and the behavior don’t, or are off just a little bit, for example blank stares, or lips that don’t move with the same muscle fidelity as humans, this causes an uncomfortable feeling amongst people interacting with them and destroys their believability. Our virtual human system, although they are realistic looking and acting, have not had any problems with falling into the uncanny valley. The focus of the virtual patient is applying these virtual humans to the medical domain to create virtual standardized patients (VP) that can be used to teach interview, diagnosis, and social-interaction skills. The primary goal of this research is to assess the technology and system in eliciting correct question/response pairs from novice clinicians in a clinical interview. A secondary goal is to investigate the impact of psychological variables such as the subjects’ state, current mood, and personality traits, openness to new experiences and immersion upon the resulting question/response composites and the overall believability of the characters. Medical students currently perform interview training with human actors acting as standardized patients. The actors portray some clinical problem in what is called an Objective Structured Clinical Examination (OSCE) [4]. These tests typically take from 20-30 minutes, a faculty member watches the student perform. The evaluation consists of self assessment rating along with faculty assessment. Although schools commonly make use of standardized patients to teach interview skills, the diversity of the scenarios that standardized patients can characterize is limited by the availability of human actors and their skills at portraying the condition. Additionally the actors most likely vary their performance from subject to subject and location to location. This is an even greater problem when the actor needs to be an adolescent, elder or portray a difficult condition. Our process is similar to an OSCE, but the actor is replaced with a virtual patient and an observer is replaced by video recording. Using virtual patients will allow standard performance assessments for all subjects. The virtual patient system was used in a series of subject testing experiments with novice clinicians and medical students. The role of the clinician was to ask appropriate and relevant questions to elicit correct responses from the virtual patient in a structured, yet free flowing, interview for history taking and diagnosis of the character. The results of the subject testing will be discussed as well as the human interaction issues with the system. Enabling rich and engaging interaction with virtual characters in a medical setting will ultimately allow powerful experiential learning engagements for new and experienced students on a continual basis for practice with a variety of patient cases they may get little or no training with. 1.1 Virtual Human Interaction Human to human interaction is very complex, enough so that many people in many fields devote a lot of time trying to understand it, and people find it so interesting that
516
P.G. Kenny, T.D. Parsons, and A.A. Rizzo
most movies produced are about human behavior and relationships. The interactions can vary based on numerous factors in both the person and the social setting. One on one interaction is different than multi-party interactions. Trying to re-create and model these interactions in virtual human systems is a great challenge. The assumptions and expectations that people have while interacting or engaged with other humans is brought over when people interact with the virtual humans [5], and these need to be replicated in the virtual character. For example users of the virtual human system expect to talk to the characters as they would with real humans, sometimes forgetting, or not knowing, the complexity and limitations of the underlying system software. Our VP system interaction is not based on the use of a traditional mouse and keyboard or pull down menus, but uses natural speech recognition. Speech interaction can cause confusion in the user if the character responds in a way that the user doesn’t expect. The user interacting with a person builds a mental model through dialog and non-verbal behavior of that person, if that representation is violated, then the user may lose engagement or be confused. The same confusion can happen with the virtual patient system and performance could suffer. For the medical domain we have some leeway as patients can, and usually do, act in non-traditional manners, thus if the patient responds in an off topic manner then this may not be thought of as incorrect. For our testing of the system we want to allow freedom of interaction and not constrain the user to specific bounds of what they can or can’t say or how they should say it. We usually do not know what kind of questions the clinicians may ask the patient, which makes it hard to design the domain. Additionally, clinicians have varied training and there are multiple approaches to Interviewing and Diagnosis [6] that the system will ultimately have to take into account. However, this freedom of exchange is a good thing to capture for it will allow us to evaluate the technology and character interaction to provide methods for improving or automating the system for a more natural human computer interaction in the future.
2 Virtual Patient System This virtual patient system consists of a computer generated 3D character that interacts with a human through natural speech. The character responds in kind through speech and gestures. Creating virtual patients that interact falls into two main areas; the technology and the domain. The technology needs to support what the character should do and how the user interacts with it, and in this case supporting the medical domain. In our virtual patients we have been concentrating on building characters with psychological problems in contrast to physical problems, they are more dialog based then motion or action based. One of the challenges of building characters for this domain is designing what needs to go into the patients in terms of the dialog, behaviors and actions to fit the patient profile. 2.1 Psychological Medical Domain The role of the clinician during an initial meeting and engagement with a patient is to capture a history of the person, find out what is going on, and try to narrow down the problem in what is called a differential diagnosis by ruling out issues and problems
Human Computer Interaction in Virtual Standardized Patient Systems
517
not relevant to the case. The virtual patient interaction should mimic the real patient interview as much as possible with the goal of allowing the clinician to ask appropriate and relevant questions to elicit correct responses from the virtual patient in a structured, yet free flowing, interview for history taking and diagnosis of the character.
Fig. 1. Justina Virtual Patient
The virtual patient for this research is an adolescent female character with PostTraumatic Stress Disorder (PTSD) called Justina, Figure 1. PTSD usually happens to people after some kind of traumatic event, such as a military engagement or assault and causes changes in behavior of the person. The effects of trauma exposure manifest themselves in a wide range of symptoms: anxiety, post-trauma stress, fear, and various behavior problems. New clinicians need to come up to speed on how to interact, diagnose and treat this trauma. According to the most recent revision to the American Psychiatric Association’s DSM Disorders, PTSD is divided into six major categories; refer to the DSM-IV category 309.81 [7] for a full description and subcategories; A. Past experience of a traumatic event and the response to the event. B. Re-experiencing of the event with dreams, flashbacks and exposure to cues. C. Persistent avoidance of trauma-related stimuli: thoughts, feelings, activities or places, and general numbing such as low affect and no sense of a future. D. Persistent symptoms of anxiety or increased arousal such as hyper vigilance or jumpy, irritability, sleep difficulties or can’t concentrate. E. Duration of the disturbance, how long have they been experiencing this. F. Effects on their life such as clinically significant distress or impairment in social or educational functioning or changes in mental states. Diagnostic criteria for PTSD includes a history of exposure to a traumatic event in category A and meeting two criteria and symptoms from each B, C, and D. The duration of E is usually greater than one month and the effects on F can vary based on severity of the trauma. Effective interviewing skills are a core competency for the clinicians, residents and developing psychotherapists who will be working with children and adolescents exposed to trauma. A clinician needs to ask questions in each of these categories to properly assess the patient’s condition.
518
P.G. Kenny, T.D. Parsons, and A.A. Rizzo
One of the challenges of designing and building these interactive virtual standardized patients has been in enabling the characters to act and carry on a dialog like a real patient that has the specific mental condition for the domain of interest. Additional issues involve the breadth and depth of expertise required in the psychological domain to generate the relevant material for the virtual character. The PTSD domain for the virtual patient allows exploration of the interaction with real end users while still being able to constrain the dialog and character behavior to a small and manageable corpus. The system needs to respond appropriately based on the six categories mentioned above and requires developing dialog for the virtual character to say around each of the categories. Table 1 is a set of example questions the clinician may ask and responses the virtual patient may say. Two additional categories were added for questions that involved building rapport or general questions, i.e. ‘what’s your name?’ and for technical issues, like accidental button presses while thinking about a question to ask. Table 1. Question / Response Categorization for PTSD Category (A) Trauma (B) Re-experience (C) Avoidance (D) Arousal
User Question So, what happened to you that night? Do you still think about what happened? Do you go out with your friends? Do you feel jumpy?
(E) Duration (F) Life Effect
How long has this been going on? Are you upset?
(G) Communication (H) Other
Hi Justina, I’m Doctor… ‘Button Press’
Virtual Patient Response Something really bad happened. Sometimes I feel like the attack is happening all over again I just stay away from everyone now. I feel like I have to watch my back all the time. A few months Sometimes I don’t do anything but stay in my room and cry. Hello I don’t get what you mean.
2.2 Technology The virtual patient system consists of a set of distributed modules of which only a few are interfaces to the user. The user interacts through speech recognition to talk to the virtual character and through the 3D graphics that shows the character’s animation in a virtual environment on a large monitor. The distributed set of components that make up the system form a pipeline that is the information flow from the input of the user to the output of the character. The main components can be divided into three areas: User Input • Speech Input – This component takes the user input from a microphone and translates that into a string of text to be used by the natural language system. The speech recognizer requires a speech and language model. Since everyone has a different voice, i.e. male, female, child, elderly, a different speech model is required and is changed for each user. In our case the users are male or female adults. A language model is required that defines the possible set of words in the domain that can be recognized. The corpus for the virtual patient consistes of 20K words.
Human Computer Interaction in Virtual Standardized Patient Systems
519
Processing • Natural Language – The text from the speech recognizer is sent to a statistical question/response module [8] that picks a response based on the input question. The question/response pairs are matched by hand prior to deployment. For the virtual patient PTSD domain there were 500 questions with 100 responses. If a question is asked that is not in the domain, an off topic response would be given such as; “I don’t know” or “I would rather not talk about that”. The questions and responses were acquired through expert knowledge, roleplaying and best guesses. • Behavior Generation – After a response is selected then gesture animations are applied based on a set of rules that govern the non-verbal behavior [9]. Gestures can be hand movement, body posture, gaze or the like and is only limited by available animations. Character Output • Speech Generation – The speech output can be generated with an automated speech generation system or with pre-recorded voice overs that match the response text. For the virtual patient pre-recorded voice was used. • Animation Output – The animations that were selected for the non-verbal behavior are combined with the generated speech to synchronize and play out together through a procedural animation system called Smartbody [10]. This drives the 3D character in a realistic fashion.
3 Subject Testing Subject testing of the virtual patient system was conducted with medical students to evaluate the interaction, systems usefulness, effectiveness and usability as a medium to communicate with the students in performing the interview task. The evaluation consisted of an assessment of the system as a whole through questionnaires and data collection during the interview. The human computer interaction factor evaluation examined the technology underlying the speech recognition, dialog interaction and behavior of the character for this task with the user. An important issue in the study of intelligent virtual agent interaction is to take into account the users openness to the interaction, new experiences and novel technologies. It has been suggested in a recent study [11] that physiological arousal appeared to be moderated by participant openness. High-absorption individuals may be more capable of imagining that the VP has PTSD when it is suggested. The users’ openness will be compared to their performance to assess if they did better on the interview task. 3.1 Participants Participants were asked to take part in a study where they would interact with an advanced prototype technical virtual patient system, in a similar matter to how they currently perform an OSCE. They were not told what kind of condition the VP had if any. Recruitment methods were by poster advertisements on the university medical campus, and classroom recruitment. A total of 15 people (6 females, 9 males; mean age = 29.80, SD 3.67) took part in the study. Ethnicity distribution was as follows:
520
P.G. Kenny, T.D. Parsons, and A.A. Rizzo
Caucasian = 67%; Indian = 13%; and Asian = 20%. The subject pool was made up of three groups: 1) Medical students (N=7); 2) Psychiatry Residents (N=4); 3) Psychiatry Fellows (N=4). For participation in the study, students were able to forgo certain medical round time with the time spent in the interview and questionnaires, which took approximately 45 minutes. 3.2 Method The subject testing was divided into three phases, a pre-test and pre-questionnaire, the interview and a post-questionnaire. The pre-questionnaire was performed in a separate room from the interview and took about 10 minutes. For the interview the participants were asked to perform a 15 minute interaction with the VP and assess any history or initial diagnosis of a condition of the character. The participants were seated in front of a large monitor that had the virtual patient sitting on a couch in the therapists’ room. The subjects used a head mounted microphone and were required to press the mouse button, talk, and then release the mouse button. The participants were asked to talk normally as they would to a standardized patient, but were informed that the system uses speech recognition and was a research prototype. They were free to ask any kind of question and the system would try to respond appropriately. At the end of the 15 minute exchange they would be sent to another room to take the post-questionnaire. Video, system logs and data from the various modules was logged as follows: • First, the user speech was recorded from the automated speech recognition (ASR) engine, the speech before and after the engine processed it was captured. The before speech was later transcribed to compare with the processed speech. • Second, the text from the natural language (NL) statistical question/response system was saved. The NL system records a transcript of the entire dialog session, this is used later to help analyze the question/response interaction. • Third, system log files of the messages between the modules were captured and could be used to reconstruct what happened during the interaction. • Fourth, cameras recorded participant’s facial expressions and body language during interaction with the virtual patient system to be used for future studies. 3.3 Measures The following measures were used to assess the impact of openness (absorption and immersiveness) upon the “believability” of the system. Prior to the experiment itself, the subjects were required to fill in the following questionnaires: 1. Tellegen Absorption Scale (TAS). The TAS standardized questionnaire aims to measure the subject’s openness to absorbing and self-altering experiences. The TAS is a 34-item measure of absorption [12]. 2. Immersive tendencies questionnaire (ITQ). The standardized ITQ measures individual differences in the tendencies of persons to experience “presence” in an immersive virtual environment (VE). The majority of the items relate to a person’s involvement in common activities. While some items measure immersive tendencies directly, others assess respondents’ current fitness or alertness, and others emphasize the user’s ability to focus or redirect his or her attention. The ITQ is comprised of 18 items, and each is rated on a 7-point scale [13].
Human Computer Interaction in Virtual Standardized Patient Systems
521
3. Virtual Patient Pre-Questionnaire (VPQ1). This unstandardized scale was developed to establish basic clinical competence for interaction with a person that is intended to be presented with PTSD, although no mention of PTSD is on the test. 4. Justina Pre-questionnaire (JPQ1). We developed this scale to gather basic demographics and ask questions related to the user’s openness to the environment and virtual reality user’s perception of the technology and how well they think the performance will be. There were 5 questions regarding the technology and how well they thought they might perform with the agent. After the experiment the subjects filled in the following questionnaires: 1. Presence questionnaire (PQ). The Presence Questionnaire is a common measure of presence in immersive virtual reality. Presence has been described of as comprising three particular characteristics: sense of being within the VE; extent that the VE becomes the dominant reality for users; and extent to which users view the VE as a place they experienced rather than simply images they observed. The PQ is a widely used questionnaire [12]. 2. Justina Post-questionnaire (JPQ2). We developed this unstandardized scale to survey the user’s perceptions related to their experience of the virtual environment in general and experience interacting with the virtual character, in particular the patient in terms of its condition, verbal and non-verbal behavior and how well the system understood them and if they could express what they wanted to the patient. Additionally there were questions on the interaction and if they found it frustrating or satisfying. There were 25 questions for this form. 3. Virtual Patient Post-questionnaire (VPQ2). This scale was exactly the same as the Virtual Patient Pre-questionnaire and will be used in the future for norming of a pre-post assessment of learning across multiple interactions with the VP.
4 Results The present focus is on effect sizes that describe the strength of association between question and response pair composites for a given PTSD category. An effect size of 0.20 was regarded as a small effect, 0.50 a moderate effect, and 0.80 a large effect.
Fig. 2. Effect Size on Question / Response Composites
522
P.G. Kenny, T.D. Parsons, and A.A. Rizzo
Figure 2 shows a chart of the data. Moderate effects existed for PTSD Category A,B,C,G, but only small effects were found for Category D and F. After controlling for the effects of the Tellegen Absorption Scale, increased effects were found for Category A, C, D, and F. To assess the impact of psychological characteristics such as absorption and immersiveness upon the “believability” of the VSP and student interaction we created a composite variable that included scores from the TAS and the ITQ. Strong effects existed between the ITQ and the PQ (0.78), and moderate effects existed between the ITQ and the VSP Post-questionnaire (0.40). These results showed that the users were able to ask question and elicit responses in each of the PTSD categories. Additionally the findings suggest that the presence and openness appears to moderate user reaction and perform better on the task. Future studies should make use of physiological data correlated with measures of immersion to augment and quantify the effects of virtual human scenarios.
5 Conclusion and Future Work Here we focused on effective interview skills—a core competency for psychiatry residents and developing psychotherapists. The keys aspects of the interview that we looked at were: interpersonal interaction; attention to the VP's vocal communications, as well as verbal and non-verbal behavior. Specifically, we wanted to assess whether the user (clinician in training) asked questions related to the reason for referral and received appropriate responses and also made attempts to gather information about the VP’s problems. Finally, we wanted to see if the user would attempt detailed inquiry to gain specific and detailed information from the VP, separating relevant from irrelevant information. The primary goal in this study was evaluative and the Question/response composites were developed to reflect the shared variance existing between the responses of the VP and the users Questions that are necessary for differential diagnosis. In future work we plan to compare the virtual patient system with live standard patient actors to assess if the technology constrains the communications or rapport between the clinician and patient. We will also compare a VP with PTSD against a VP that does not have PTSD to assess if a clinician can make a proper diagnosis. Additional improvements in the language and speech for the domain will allow for deeper and richer dialog. Building virtual characters is an iterative approach that improves the technology with feedback from real users to assess if these systems can be used as effecting teaching and training tools, which we believe is the case.
References 1. Kenny, P., Hartholt, A., Gratch, J., Swartout, W., Traum, D., Marsella, S.D.: Piepol Building Interactive Virtual Humans for Training Environments. In: Proceedings of I/ITSEC (November 2007) 2. Gratch, J., Rickel, J., André, E., Badler, N., Cassell, J., Petajan, E.: Creating Interactive Virtual Humans: Some Assembly Required. IEEE Intelligent Systems, 54–63 (July/August 2002)
Human Computer Interaction in Virtual Standardized Patient Systems
523
3. Mori, M.: On the Uncanny Valley. In: Proceedings of the Humanoids-2005 workshop: Views of the Uncanny Valley, Tsukuba, Japan, December 5 (2005) 4. Walters, K., Osborn, D., Raven, P.: The development, validity and reliability of a multimodality objective structure clinical examination in psychiatry. Medical Education 39, 292–298 (2005) 5. Gratch, J.N.W., Gerten, J., Fast, E., Duffy, R.: Creating Rapport with Virtual Agents. In: 7th International Conference on Intelligent Virtual Agents, Paris, France (2007) 6. Evans, D.H.M., Uhlemann, M., Lvey, A.: Essential Interviewing: A Programmed Approach to Effective Communication, 3rd edn. Brooks/Cole Publishing Company (1989) 7. DSM, American Psychiatric Association (DSM-IV-TR) Diagnostic and statistical manual of mental disorders, 4th edn., text revision. American Psychiatric Press, Inc., Washington (2000) 8. Leuski, A., Traum, D.: A Statistical Approach for Text Processing in Virtual Humans. In: Army Science Conference (2008) 9. Lee, J., Marsella, S.: Nonverbal Behavior Generator for Embodied Conversational Agents. In: 6th International Conference on Intelligent Virtual Agents, Marina del Rey, CA (2006) 10. Thiebaux, M., Marshall, A., et al.: SmartBody: Behavior Realization for Embodied Conversational Agents. In: International Conference on Autonomous Agents and Multi-Agent Systems, Portugul (2008) 11. Macedonio, M., Parsons, T.D., Rizzo, A.A.: Immersiveness and Physiological Arousal within Panoramic Video-based Virtual Reality. Cyberpsychology and Behavior 10, 508–516 (2007) 12. Tellegen, A., Atkinson, G.: Openness to absorbing and self-altering experiences (“absorption”), a trait related to hypnotic susceptibility. Journal of Abnormal Psychology 83, 268–277 (1974) 13. Witmer, B., Singer, M.: Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoperators and Virtual Environments 7(3), 225–240 (1998)
Towards Standardized Pen-Based Annotation of Breast Cancer Findings Suzanne Kieffer1, Annabelle Gouze1, Ronald Moncarey1, Christian Van Brussel1, Jean-François De Wispelaere2, Françoise Kayser2, and Benoît Macq1 1
Communications and Remote Sensing Laboratory, Université catholique de Louvain Place du Levant 2, B-1348 Louvain-La-Neuve, Belgium 2 Cliniques universitaires UCL de Mont-Godinne Avenue G.Thérasse 1, B-5530 Yvoir, Belgium {suzanne.kieffer,annabelle.gouze,ronald.moncarey, christian.vanbrussel,jean-francois.dewispelaere, francoise.kayser,benoit.macq}@uclouvain.be
Abstract. The development of computer technologies provides a means to support and facilitate the daily activities of potentially all users. This may be of particular importance for experts in breast cancer imaging and diagnosis. While many research efforts have been carried out separately on the implementation of task-oriented systems, much less effort has been undertaken to design and develop technologies compliant with domain standards or in accordance with end-user needs and expectations. This further suggests the need to improve both the usefulness and the usability of breast cancer-dedicated systems. This paper reports the results of a development method combining the application of usercentered design together with usability development methods. At different time frames in the life-cycle, the development method employed knowledge elicitation interviews, scenario-focused questionnaires, paper mock-ups and usability tests. Owing to its naturalness and its convenience, pen-based interaction with a graphics tablet was chosen as the modality to interact with the system. Additional innovative solutions were designed and implemented in order to facilitate and improve the visualization and the manipulation of data during the lesion characterization: namely an icon framework, a star-menu and a semi-automatic lesion detection system. The resulting user interface is a penbased interactive tool supporting visualization, navigation, standardized lesion characterization and reporting. The usability tests suggest that it provides endusers with an efficient, reliable and usable system.
Towards Standardized Pen-Based Annotation of Breast Cancer Findings
525
tend to split their work between a growing number of interactive tools, workstations and media (e.g., screen, mouse, keyboard, and handheld recorder). In addition, interactive tools for BC screening and reporting have to be compliant with the BI-RADS standard [1], an approved system of descriptive terms and reporting guidelines. Such tools not only facilitate reporting, providing radiologists with structured and standardized reports, but lead to data accessibility as well (e.g., data exchange and storage, interpretation monitoring [13], retrieval of useful and interesting cases for teaching and research purposes [14]). Finally, interests in user-centered design together with usability stem from the goal to design and implement interactive systems supporting the activities of domainexpert users, who are not necessarily experts in computer science. As highlighted in [3], great care must be devoted to the study of the needs and the expectations of such domain-expert users. In particular, attention must be paid to usability throughout the software life-cycle in order to design and implement user-friendly and easy-to-use interfaces [9,6].
2 Objectives and Significance of the Work The ultimate objectives of the research presented here are to design, implement and evaluate a BC-oriented interactive system which integrates the interactive annotation of significant findings (i.e., lesion characterization and reporting) and the semiautomatic lesion detection. This paper focuses on the design and the evaluation of the annotation tool; refer to [5] for more details about the semi-automatic lesion detection tool. A prototype for lesion annotation, based on semantic web technologies, was presented during SPIE Medical Imaging 2007 [4]. Our work goes further mainly by providing BC-experts with a new interaction style to characterize findings: the penbased annotation with a graphics tablet. The significance of the work can be highlighted depending on three axes: • End-user: provide experts in BC screening with useful and usable tools; • Accessibility: increase breast imaging data accessibility thanks to standardization (data exchange and storage, interpretation monitoring [13], useful and interesting cases retrieval for teaching and research purposes [14]); • Usability: develop and promote support for designers of usable systems.
3 Annotation of Breast Cancer Finding The pen-based user interface (PUI) is an effective method to provide end-users with a natural, intuitive and convenient interaction [12]. Owing to its high naturalness and mainly to its convenience to satisfy the lesion characterization requirement, pen-based interaction with a graphics tablet was chosen as the modality to interact with the system: navigating in a clinical case (i.e., among the mammograms), navigating in a specific mammogram (i.e., zoom-in, zoom-out), sketching a region of interest (ROI), annotating findings and reporting (i.e., direct manipulation of menus, icons, widgets). The BI-RADS [1] provides a standardized terminology for the description of BC findings. Any finding is described according to a lesion type (i.e., mass, calcification,
526
S. Kieffer et al.
architectural distortion, special case or associated finding), and type-related characteristics. Beyond the specific characteristics related to a lesion type, the breast imaging report contains the finding location and the comparison to previous studies, whatever the type. An icon framework was created according to this standard in order to enable any finding to be fully described. Every single term of the BI-RADS lexicon is represented by a unique icon (altogether about 150 different icons), so that the lesion characterization is straightforward and unambiguous. A color code was adopted in order to facilitate the discrimination between the findings: masses in blue, calcifications in yellow, architectural distortions in green, special cases in violet, and associated findings in orange. The schemes on the icons related to the finding location and the comparison to previous studies are common to all the lesion types; only colors are different. The Table 1 presents the icons related to the specific characteristics of masses. Masses are characterized by basic shape (round, oval, lobular or irregular), margin (circumscribed, microlobulated, obscured, indistinct or spiculated) and density (high-density, equal density, low-density or fat-containing radiolucent). Table 1. Icons related to the specific characteristics of masses: shape, margin and density Shape
Round
Oval
Lobular
Irregular
Obscured
Indistinct
Margin
Circumscribed
Microlobulated
High-density
Equal Density
Spiculated
Density
Low-density
Fat-Containing Radiolucent
4 Development Method The development method combined user-centered design together with usability development methods, and employed: knowledge elicitation interviews, scenariofocused questionnaires, paper mock-ups and usability tests. Domain- and task-relevant knowledge was collected early in the life-cycle thanks to knowledge elicitation interviews. Five domain-expert users were questioned thoroughly about the BC domain, the task series involved in their activity, their needs and their expectations with the goal to implement the collected information in the system. The equipment used was paper notes and video recording. Scenario-focused development method was used to define and select the interaction scenarios which would best support and fit end-user activities. Thanks to
Towards Standardized Pen-Based Annotation of Breast Cancer Findings
527
Table 2. Scenario-focused questionnaire. The end-user activity is the description of the lesion type (column 1). There were two proposed interactive scenarios to support this activity: using an array of buttons or using a pie-menu (respectively, columns 2 and 3). Screening analysis activity Description of the lesion type: • • • • •
Scenario#1 Array of buttons:
Scenario#2 Pie menu:
Masses Calcifications (CA++) Architectural distortion Special cases Associated findings
Fig. 1. Paper mock-ups. On the left: the experimental material such as paper, glue, and pen. On the right: the device in use.
end-user involvement, such a design method proved to reduce both development time and development cost, and to improve usability [12]. Referring to [12], a written scenario-focused questionnaire (see Table 2) and paper mockups (see Fig. 1) were implemented complementarily. Both were elaborated from the information collected during knowledge elicitation interviews and were presented to six domain-expert users in order to evaluate the icon framework and the overall spatial organization of the interface, and to select the potential interactive scenarios. The questionnaire was used by the interviewer as visual aid during face-to-face interviews. Paper mock-ups were preferred to computer prototypes since the available prototyping tool did not support the scenarios applying for implementation, and since the evaluation should lead to a lot of drawings, direct manipulation of paper components and discussions between designers and domainexpert users [10]. The analysis of the data collected from scenario-focused questionnaire and paper mock-ups led to the expert validation of: the terminology and the iconic framework, the color code, and the preliminary expert validation of the Pie and the Star menus. Providing experts in BC imaging with an interactive tool supporting their activity is a tough problem of human-computer interaction considering the user requirements of usefulness and usability. The usefulness is ensured by the compliance of the system with the domain standard (see section 3), and by the integration within a single
528
S. Kieffer et al.
interactive tool of the functionalities of image visualization, finding characterization (i.e., annotation), semi-automatic lesion detection and reporting. In order to guarantee the usability of the system, especially during the lesion characterization task, sustained attention has been paid to the graphical representation of the mammographic BI-RADS terminology (by the design of an exhaustive set of icons), to the spatial organization of multimedia data (not only the overall spatial organization of the user interface, but the specific location of widgets as well), and to the design of new interactive solutions suited to the finding annotation with a pen on a graphics tablet. Therefore, two complementary menus were implemented to support the pen-based annotation of BC findings: a pie menu [2] for the pen-based selection of the lesion type, and a star menu for the pen-based description of the type-related characteristics of the lesion. The pie menu (see Fig. 2, left) was implemented in order to facilitate the pen-based annotation of the lesion type. This format was chosen because it reduces the target seek time and improves the accuracy of target selection [2,8]. The star menu (see Fig. 2, right) was implemented in order to facilitate the pen-based annotation of complementary characteristics by grouping icons related to the same characteristic on a single line. This format was chosen because such a display layout was proved to be very efficient and accurate for visual inspection or visual detection by comparison with matrix, elliptic and random spatial structures [11].
Fig. 2. Menus for the pen-based annotation of findings. Pie menu (left) and star menu (right).
5 Usability Evaluation of the Pie and the Star Menus The purpose of this investigation was to evaluate the usability of the pie and the star menus (pie-star menus) during the pen-based annotation of BC findings. The usability evaluation criteria were the system effectiveness, the system efficiency and the satisfaction of the users [9,6]. The effectiveness metric was the task completion, whereas the efficiency metrics were task completion times and the number of clicks. Participants were asked to annotate BC findings in clinical cases by using either the pie-star menus, or an array of icons located at the top of the user interface. The array icons and the star menu icons are exactly the same in terms of scheme, color and size. The array allows the sequential selection of finding characteristics, according to the following order: lesion type, type-related characteristics (each characteristic has to
Towards Standardized Pen-Based Annotation of Breast Cancer Findings
529
Fig. 3. Array of icons. Lesion type (top). Shape of a mass (bottom).
be described one at a time), comparison to previous studies and location. Fig. 3 presents the array of icons displayed to describe the lesion type (top) and the shape of any mass (bottom). Nine volunteers ranged in age from 43 to 58 and including 3 females and 6 males participated in this study. All participants were experimented breast radiologists practicing in different hospitals in Belgium. They were recruited regarding their experience in breast cancer screening. Computer skills were assessed thanks to a background questionnaire. All participants were familiar with computers and especially with medical computer-based applications and all were experienced in visual search and navigation activities on computer displays. They were also standard mouse and keypad users with similar quick motor reactions. The usability test employed a 2x5 factorial design, with two experimental conditions (pie-star menu versus array) and five medical cases to characterize (i.e., five tasks). Each participant carried out ten tasks: five per experimental condition. The order of the experimental conditions was counterbalanced between participants according to a 2x2 Latin Square design. Likewise, the order of the five medical cases per condition was randomized. Counterbalancing and randomization were used in order to neutralize possible task learning effects and to control inter-individual diversity. The tests were carried out in an isolated room in each hospital. Participants were seated approximately 40 cm from the graphics tablet. Pen-based annotation was used as input modalities whereas visual display was used as output modality. The computer system used in this study was a computer with an Intel Core2 Duo E8400 (3GHz) processor, 4 GB of DDR SDRAM and a 9600GT Nvidia graphic card. The screen was a WACOM CINTIQ 21UX. DICOM (Digital Imaging and Communications in Medicine) images were loaded into the viewer. The test sessions involved one volunteer at a time. First, participants were given an oral presentation of the project, an explanation of their role in the usability tests, and a demonstration of the functionalities of the tool. Then, they started the training session: one clinical case to annotate per experimental condition. Once they felt comfortable enough with the tool and got used to the manipulation of the pen, they were provided with the paper printed instructions, and the demographic and background questionnaires to fill prior to the effective test. After each condition, they were asked to fill a satisfaction questionnaire. After the two conditions, they were asked to fill the CSUQ [7], a 19-item questionnaire which aims at evaluating the usability of a system in terms of System Usefulness (SysUse), Information Quality (InfoQual) and Interface Quality (IntQual) on a 7-point Linkert scale. A debriefing ended the session. The effective tests lasted approximately 30 minutes.
530
S. Kieffer et al.
6 Results 6.1 Statistical Analysis The sample includes 219 entries (i.e., 219 findings were annotated in all). Analyses of variance (ANOVA) were used to examine the presence of significant differences in task performance, as measured by both annotation times in seconds and number of clicks: per conditions (pie-star and array), per view (CC1 and MLO2), and per finding type (mass, calcification, architectural distortion, special case and associated finding). Table 3. ANOVA Procedure. Factors: experimental condition, view, and finding type. DF stands for degree of freedom, AT for annotation time, and NC for number of clicks. The statistically differences are bold. Factors Condition View Finding type
Annotation times. The results presented in the Table 3 show no statistically significant difference between the experimental conditions, but a tendency (F=3.5605; p=0.0605). On the other hand, they show a significant view effect (F=5.6496; p=0.0183) and a significant finding type effect (F=2.7884; p=0.0274). Number of clicks. The results presented in the Table 3 show a highly significant finding type effect (F=9.0073; p<0.0001). The results from Table 4 show that the annotation of BC findings with the pie-star menus is faster than with the array of icons (pie-star: 17.5 sec versus array: 20.5 sec). Since the results highlight no significant effect of the number of clicks between the two conditions, this difference of speed may be a matter of visualization and visual perception: the star menu displays simultaneously all the icons related to a finding type, whereas the array displays the icons related to one characteristic at a time. In 1 2
The Cranial-Caudal view (CC) is taken from above. The MedioLateral Oblique view (MLO) is taken from an oblique view.
Towards Standardized Pen-Based Annotation of Breast Cancer Findings
531
opposition to the array, the star menu enables the users to anticipate their next clicks and, consequently, to be faster. In addition, the results from Table 4 show that the annotation of BC findings is faster in the CC view than in the MLO view (CC: 20.5 sec versus MLO: 16.34 sec). This difference may be explained by the combination of the following two reasons. First, in practice, breast radiologists start the diagnostic by the analysis and interpretation of the CC view. It may be natural to adopt the same task order with the interactive tool. Second, findings such as masses and calcifications need to be characterized in both views. Thus, a “duplicate” button was implemented in order to reduce the number of clicks necessary for the complete annotation of findings. Table 5. Means and standard deviations of number of clicks Finding type Mass Calcification Arch. Dist. Special Case Associated finding
N 69 89 39 5 17
Mean 5.46377 4.58427 4.02564 4.00000 6.58824
Standard deviation 0.21331 0.18782 0.28373 0.79241 0.42975
Finally, the results from Table 3, Table 4 and Table 5 show that the speed and the number of clicks to perform the annotation task depend on the type of the finding under annotation. This difference may be explained by the combination of the following two reasons. First, the number of characteristics differs from a finding to another (i.e., six characteristics for calcifications, five for masses, and only three for architectural distortions, special cases and associated findings). Second, the important amount of icons to memorize (i.e., about 150) necessarily involves a considerable learning time, and it sounds acceptable that the annotation of unusual findings such as special cases and associated findings requires more time and more clicks in comparison with masses, calcification and architectural distortion which are more frequent. 6.2 User Satisfaction and Preferences Through the questionnaires and during the interviews, participants considered the interaction with the system as natural, intuitive and reliable. A majority of participants (8) hesitated less than five times, and all participants were satisfied with the compliance with the BI-RADS. Five participants expressed very positive judgments on the star menu in terms of information visualization, speed and comfort. They preferred the star menu because: “it enables the parallel visualization of the items thanks to its spatial organization”, “it is more comfortable thanks to its position close to the center of the screen”, and “it is faster (than the array)”. Four participants preferred the array because “it is usual” and “the characteristics follow a logical sequence”. These results are consistent with the CSUQ results reported in the Table 6. The interface quality, especially, is assessed very positively by the participants.
532
S. Kieffer et al.
Table 6. Summary of the overall sample CSUQ. Each 19 item was score on a 7-point Linkert scale (1=totally disagree, 7=totally agree). Statistical indices are mean and standard deviation. Mean SYSUSE INFOQUAL INTERQUAL OVERALL
5.46 5.56 5.81 5.56
Standard deviation 0.96 1.11 0.88 1.89
7 Conclusion A pen-based interactive tool for standardized annotation of BC lesions was designed and implemented combining user-centered design and usability development methods. Our approach employed knowledge elicitation interviews, scenario-focused questionnaires, paper mock-ups, lab tests, field tests and post-test questionnaires. Additional complementary solutions were designed and implemented in order to facilitate and improve the manipulation of data during the BC finding annotation: the pie and the star menus. The emphasis of this approach is the attention paid to users and usability. The benefit of this approach is improved user satisfaction. The pie and the star menus lead to better user performances than with the array of icons. This is remarkable with respect to the fact that this unusual interaction style is brand-new and the users thus had no previous experience with it. Furthermore, participants to the usability tests expressed very positive judgments on the star menu in terms of information visualization, speed and comfort and on the user interface. In particular, the interface is judged easy-to-use and adapted to the human activity. However, as it requires a substantial amount of collected data and numerous individual interviews in great details, the availability of representative users can be an obstacle to the implementation of this novel interaction concept introduced here for other applications. Acknowledgments. The research presented in this paper has been financed by the Walloon Region within the WIST2-616446 VIGILE project. The authors would like to thank the radiology departments of the Cliniques universitaires UCL (MontGodinne), the Institut Jules Bordet (Brussels), the Clinique Saint-Pierre (Ottignies), the Clinique Sainte-Anne (Anderlecht), and the Clinique Notre-Dame (Tournai).
References 1. American College of Radiology, Breast imaging reporting and data system (BI-RADS®) atlas, 4th edn. American College of Radiology, Reston (2007) 2. Callahan, J., Hopkins, D., Weiser, M., Shneiderman, B.: An empirical comparison of pie vs. linear menus. In: Proc. ACM CHI Conference on Human Factors in Computing Systems, pp. 95–100 (1988)
Towards Standardized Pen-Based Annotation of Breast Cancer Findings
533
3. Costabile, M.F., Fogli, D., Fresta, G., Mussio, P., Piccinno, A.: Computer environments for improving end-user accessibility. In: Carbonell, N., Stephanidis, C. (eds.) UI4ALL 2002. LNCS, vol. 2615, pp. 129–140. Springer, Heidelberg (2003) 4. Gemo, M., Gouze, A., Debande, B., Grivegnée, A., Macq, B.: A versatile knowledgebased clinical imaging annotation system for breast cancer screening. In: Proc. SPIE Medical Imaging, pp. 6514–6565 (2007) 5. Gouze, A., Kieffer, S., Van Brussel, C., Moncarey, R., Macq, B.: Interactive breast cancer segmentation based on relevance feedback: from user-centered design to evaluation. In: SPIE Medical Imaging, vol. 7260, p. 726021 (2009) 6. ISO 9241, Ergonomics requirements for office work with visual display terminals (VDTs) (1998) 7. Lewis, J.R.: IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. International Journal of Human-Computer Interaction 7(1), 57–78 (1995) 8. Moyle, M., Cockburn, A.: Analysing Mouse and Pen Flick Gestures. In: Proc. SIGCHI-NZ Symposium On Computer-Human Interaction, pp. 266–267 (2002) 9. Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers, San Francisco (1994) 10. Sefelin, R., Tscheligi, M., Giller, V.: Paper prototyping – what is it good for? A comparison of paper- and computer-based low-fidelity prototyping. In: Proc. CHI 2003 Extended abstracts on human factors in computing systems, pp. 778–779 (2003) 11. Simonin, J., Kieffer, S., Carbonell, N.: Effects of display layout on gaze activity during visual search. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 1054–1058. Springer, Heidelberg (2005) 12. Wang, D., Dai, G., Wang, H., Chiu, S.C.: Scenario-focused development method for a pen-based user interface: model and applications. The Journal of Supercomputing (Online First) 13. Wittenberg, T., Elter, M., Schulz-Wendtland, R.: Bildverarbeitung für die Medizin 2007. In: Complete digital iconic and textual annotation for mammography, pp. 91–95. Springer, Heidelberg (2007) 14. Zheng, Y., Wu, M., Cole, E., Pisano, E.D.: Online annotation tool for digital mammography. Academic Radiology 11(5), 566–572 (2004)
ImproV: A System for Improvisational Construction of Video Processing Flow Atsutomo Kobayashi, Buntarou Shizuki, and Jiro Tanaka Department of Computer Science, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan {atsutomo,shizuki,jiro}@iplab.cs.tsukuba.ac.jp
Abstract. ImproV is a video compositing system for live video. It uses a dataflow diagram to represent the video processing flow and allows performers to edit the diagrams even while the video is running. Traditional live video is limited to video editing, but ImproV allows users to construct video processing flows on the fly. We present the design of ImproV and report on some actual live video performances using ImproV as preliminary evaluation in this paper. Keywords: live performance, visual music, visual performance, visual jockey, VJ, improvisation, dataflow, visual programming, video authoring, video compositing.
1 Introduction In live video performances, such as those at musical events or fashion shows, the video performer selects videos and switches between them according to the event’s atmosphere, the occasion, and the type of music used at the event. Therefore, while showing a video to the audience, the performer must select video and optionally adjust the cue point, playing speed, and effects parameters among others. Although each performer has their own style and method, their performance includes some improvisational attributes. We have developed a video compositing system for live video performances called ImproV. In Sec. 2, a standard live video performance and its requirements are explained. In Sec. 3, systems similar to ImproV and live video performance related researches are discussed. In Sec. 4, we report on our preliminary evaluation and results of this system. Our conclusion is given in Sec. 5.
ImproV: A System for Improvisational Construction of Video Processing Flow
535
In the step 1, the video creators create 2D/3D animation or shoot with a video camera. The animation can be character animation or moving abstract graphics, which are called motion graphics. The footage can be real world scenes or live-action. Then, in step 2, the video creators process the raw video footage. They correct the colors, apply some video effects, layer multiple video footage and many other things. Common video compositing software applications, e.g. Adobe After Effects, uses a layer model to represent the video processing flow. In step 3, the video compositions are arranged along a time axis. Editing software applications, e.g. Adobe Premier, use a timeline representation in contrast to what a layer model uses. For live video performances, switching videos corresponds to step 3 in video creation. The live video performers finish steps 1 and 2 before the live video performance is held. This means it is impossible for the live video performers to change the effect parameters and video processing flow of the compositions. Wires are used to connect the hardware devices necessary to construct a video processing flow when using a video mixer, such as a Roland V-4. The live video performer can find it very hard to change the video processing flow. Even with the proper software applications, most being modeled after a video mixer, the video processing flow is fixed such that typically two video sources are mixed using a two-channel video mixer with optional effectors. We thought this fixing of video processing flow limits the performer's improvisational possibility and believe a performance is more expressive with the improvisational construction of the video processing flow. 2.1 Design Requirement: Background Compositing One of the most common methods used by a performer during a live video performance is to prepare another video in the background of the current video the audience is watching, and then switch them. Live video performers use hardware video mixers or software applications based on a video mixer. M. Lew analyzed the method of editing video on the fly using a video mixer [1]. In the analysis, the following three steps are used. STEP 1: Media retrieval STEP 2: Preview and adjustment STEP 3: Live manipulation A performer browses the video clips and selects the next one (STEP 1). Then, the performer adjusts the cue point or playing speed of the video clip, applies any necessary effects, and adjusts the effect parameters by previewing them using a monitor display (STEP 2). Finally, the performer starts to switch the current video clip to the next video clip by manipulating the video mixer and optionally animates the effect parameters by directly manipulating the effectors (STEP 3). With traditional live video performances, STEP 2 is merely controlling the parameters and optionally selecting an effect. We think this severely limits expressiveness in traditional live video performances. To solve this problem, we thought that STEP 2 should include the constructing and changing of the video processing flow. That is to say, a performer can repeatedly use trial and error to create videos till they are satisfied. To prevent unintended videos from being shown to the audience, previewing is important in this method. We call this method background compositing.
536
A. Kobayashi, B. Shizuki, and J. Tanaka
2.2 Representation of Video Processing Flow Video creators mix several video materials in video compositing. The video mixer and video mixer representation in the software are not powerful enough to treat three or more videos concurrently. The systems with layer representation such as Adobe After Effects provide more flexible video processing flow than systems with a mixer. The systems with layers can overlay an arbitrary number of images. However, while showing a video, the user can't make and mix/switch to another video composed of a different processing flow from the one being shown. As mentioned above, the background compositing and previewing features are necessary for video compositing during live video performances. Moreover, a layer representation works only when all the videos are simply overlaid. However, when the video processing flow is complex such as when using a mask, in which a mask video is correlated to another video and overlaid with yet another video, the user carefully checks each layer, and sometimes opens or pulls down the layer property to see which layers are correlated to each other. Moreover, the applied effects are hidden in the layer representation. Most of these software feature an extra window and show only the effects applied to a selected layer. These modal interfaces force the user to search inside the video processing flow. During a live performance, glancability and accessibility should be preceded since it’s a realtime situation. For all the above-mentioned reasons, we decided to use a dataflow diagram to represent the video processing flow.
3 Related Work There are many systems that use a dataflow diagram to represent the video processing flow. Jitter [2] is one of most standard dataflow systems for videos in the fields of art and entertainment, but it only supports a workflow that separates the construction and execution of a dataflow. On the other hand, systems like vvvv [3] and Quartz Composer [4] support the dynamic construction of a dataflow. However, these systems, including Jitter, are designed as programming environments and the abstraction level of dataflow diagrams are too low to be used by video creators and live video performers. A similar dataflow system that is not for programmers but for end-users to ImproV is FindFlow [5], which aims at interactive retrieval from mass data. Some studies have been conducted on live video performances. EffecTV [6] is a real-time video effects framework and is aimed at live video performances. Novel controllers, such as SPATIAL POEM [7], Rhythmism [8] and video-organ [9], are used for controlling the visual attributes of live video performances. However, these controllers assume that the system settings or video processing flow are already set and fixed before the live video performance is started. Soundium [10] by Mueller et al. started from a similar concept to that we used for ImproV, which was to use impromptu video creation to create a dynamic image to improve the expressiveness of a live video performance. Their challenges were the technical issues of real-time media processing. In contrast to these researches, we worked on the exploration of the user interface designs for the interactive construction of video processing flows.
ImproV: A System for Improvisational Construction of Video Processing Flow
537
4 ImproV We have developed ImproV, a real-time video compositing system. To increase a performer's improvisational availability, we designed ImproV to allow a performer to construct a video processing flow on the fly. To this end, we decided to use a dataflow diagram to represent the video processing flow. Several nodes are connected to each other with the edges in the dataflow diagram. In the dataflow diagram in ImproV, each node represents a video source (e.g., a video file or a camera), an effector (e.g., a blur effector or a mixer), or an output screen. The video processing flow is constructed by connecting these nodes. 4.1 Understandability for Live Video Performers A performer can easily understand the concept of connecting nodes in the dataflow, because they are used to connecting video processing devices such as video tape players, video effectors, and video mixers. We carefully designed the nodes of ImproV so that the performer can use their knowledge about video processing devices. In the other words, we simplified the design barriers part of the six learning barriers of programming systems [11]. 4.2 Dataflow Diagram of ImproV Figure 1 shows the dataflow diagram of ImproV. The most basic node is shown as part a “Camera” in Fig. 1, which has a label to show the type of node and has an output port at the right. Many of the nodes have some input ports listed under the name label, as shown as part c “Mixer” in Fig. 1. The user uses the menu to create these nodes and connects the nodes by dragging and dropping from the output port of a node to the input port of another node. Figure 1 part b is the same node “Camera” as in part a, but it is connected to the “Mixer”. Some nodes have original user interfaces. Figure 1 part d “Output Screen” this type of user interface to show a preview of the output.
Fig. 1. a: a node that has no input, b: same node as a that is connected to another node, c: a node which has four inputs, and d: a node that has an input and custom user interface
There are two data types, video image and floating point number, for controlling the effect parameters, in an ImproV dataflow diagram. In Fig. 2, the video image is distorted by “Distortion”. The degree of distortion is decided by the value input to “Value” input port. The “Slider” outputs a floating point value of about 3.0 in Fig. 2.
538
A. Kobayashi, B. Shizuki, and J. Tanaka
Fig. 2. Dataflow diagram with two data of types
Fig. 3. Dataflow diagram mixes two videos
In Fig. 3, two videos, one is an image of a flower and the other is particle animation, are overlaid with additional blending. The video with the image of a flower is applied and gradated using a “Blur” effect. The degree of blur is decided by the “Slider”. This video processing flow is similar to traditional live video performance systems. 4.3 Supporting Complex Video Processing Flow Traditional live video performance systems mainly focus on mixing/switching wellpreprocessed videos, and their video processing flows are simply made. For example, one may want to copy a video to make eight, rotate them 45 degrees, and mix them with additional blending (this will result in a kaleidoscope-like effect). These systems do not support this idea without preprogrammed effects. In ImproV, the performer can create several edges from one output. Branching edges from one output is treated as copying the value from the output. Therefore, the performer can construct the video processing flow of this example on the fly. Figure 4 shows the video processing flow of this example in ImproV. 4.4 Supporting Background Compositing and Previewing Since constructing a video processing flow can cause unintended results, we designed ImproV to support background compositing and previewing. Figure 5 illustrates how to mix two video processing flows.
ImproV: A System for Improvisational Construction of Video Processing Flow
Fig. 4. Video copied, rotated, and overlaid
Fig. 5. Mixing two video processing flows
539
540
A. Kobayashi, B. Shizuki, and J. Tanaka
• Fig. 5 (a): After the example above, the performer makes a new video of another video processing flow, which consists of a video file and an effector “Transparency”. • Fig. 5 (b): The performer inserts a mixer before the output screen. • Fig. 5 (c): The performer makes the new video transparent by using the effector “Transparency”, and then overlays the new video over the old one by connecting the effector and the mixer. • Fig. 5 (d): The performer remakes the transparency, resulting in the new video fading-in. Note that we treat two video processing flows, the old complex one and the new simple video file, equally. ImproV allows users to edit video processing flows arbitrarily. The result is that users can independently create a video processing flow from the current main output, which the audience is watching. As mentioned above, previewing images is important for the background compositing method. A user can connect any nodes on the dataflow to output screens because of the nature of a dataflow diagram. Therefore, the user can preview videos at arbitrary points in the video processing flow. Figure 6 (b) shows the previewing from three points in the video processing flow in Fig. 6 (a).
Fig. 6. Performer can preview any points in video processing flow
ImproV: A System for Improvisational Construction of Video Processing Flow
541
5 Preliminary Evaluation The first author recorded a live video performance of a musical event for use as a preliminary evaluation. Several bands played jazz music at the event. The live video performance covered two hours of the whole event. Figure 7 shows the band and the display of the computer running ImproV at the event. We used a USB camera and some pre-rendered video clips. The projection was directed toward the screen behind the bands through the bands themselves. The computer running ImproV and the first author were behind the audience. From a stability aspect, ImproV worked continuously without any problems for two hours. In addition, the first author could keep the video playing by using background compositing for the entire two hours. At the most complex situation, there were over 20 nodes on the display. We confirmed that novel expression is practicable using ImproV. Figure 4 shows images that are the same as the ones of the video processing flows made in this evaluation. The first author tried to make videos expressing a different feeling for each song. We found that a video processing flow with 10-20 nodes could be made in 3-5 minutes. This is reasonable span of time to create video expressing different feelings for each song, which normally take around five minutes. Moreover, we found that a 17-inch display enabled the performer to edit a video processing flow with around 30 nodes. However, we also realized that the edge connecting operation is perplexing.
Fig. 7. Actual live video performance using ImproV
6 Conclusion We described ImproV, which is a system focused on the improvisational construction of video processing using a dataflow diagram. ImproV allows a live video performer
542
A. Kobayashi, B. Shizuki, and J. Tanaka
to make novel improvisational video compositing. ImproV supports background compositing and previewing to protect from showing an unexpected video to the audience. We tested ImproV in an actual live video performance environment and confirmed its capability. We also found the edge connecting operation problem while conducting this evaluation. We are planning to work on this problem and conduct a more formal evaluation.
References 1. 2. 3. 4.
5. 6.
7.
8.
9.
10.
11.
vvvv group: vvvv Apple Inc.: Quartz Composer Cycling ’74: Jitter Choi, J., Hong, H.: SPATIAL POEM: A New Type of Experimental Visual Interaction in 3D Virtual Environment. In: The 8th Asia-Pacific conference on Computer-Human Interaction, pp. 167–174 (2008) Bongers, B., Harris, Y.: A structured instrument design approach: the video-organ. In: the 2002 conference on New interfaces for musical expression, pp. 1–6 (2002) Fukuchi, K., Sam, M., Ed, T.: EffecTV: a real-time software video effect processor for entertainment. In: The international conference on entertainment computing, pp. 602–605 (2004) Hansaki, T., Shizuki, B., Misue, K., Tanaka, J.: FindFlow: visual interface for information search based on intermediate results. In: The 2006 Asia-Pacific Symposium on Information Visualisation, pp. 147–152 (2006) Ko, A., Myers, B., Aung, H.: Six Learning Barriers in End-User Programming Systems. In: 2004 IEEE Symposium on Visual Languages - Human Centric Computing, pp. 199–206 (2004) Lew, M.: Live Cinema: designing an instrument for cinema editing as a live performance. In: International Conference on Computer Graphics and Interactive Techniques, pp. 144–149 (2004) Mueller, P., Arisona, S., Schubiger-Banz, S., Specht, M.: Interactive media and design editing for live visuals applications. In: International Conference on Computer Graphics Theory and Applications, pp. 232–242 (2006) Tokuhisa, S., Iwata, Y., Inakage, M.: Rhythmism: a VJ performance system with maracas based devices. In: International conference on Advances in computer entertainment technology, pp. 204–207 (2007)
E-Assessment: A Suitable Alternative for Measuring Competences? Martin Kröll Ruhr-Universität Bochum, Institut für Arbeitswissenschaft, Universitätsstraße 150, 44780 Bochum, Germany
Abstract. More and more companies accomplish tests for the assortment of trainees to measure the aspirants competences and competence potentials. The article in hand is dedicated to the academic evaluation of the adoption of new E-Assessment methods. Thereby it has to be resolved in how far the expectations linked with the E-Assessment’s assignment for example seen from the company’s view or from the proband’s one, can be achieved. To adopt the computer-aided diagnostic professionally, its advantages and disadvantages are to be considered. Keywords: employability, e-assessment, self-organization, key qualification.
2 Understanding the E-Assessment Procedure E-assessment is commonly understood to be: “the use of internet-based tests (instruments) …, which serve to appraise and predict relevant job-related biographical and psychological variables and to assess the suitability of applicants” [7] (translated by the author). This paper deals not only with online processes, but also with computer-based diagnostics of suitability as e-assessment approaches (Ridder et al., 2004: 34). Such tests, carried out in the context of formal selection procedures, often take place on site. E-assessment can and is used in a range of organizations (e.g. businesses, public authorities, schools, universities) and applied to different functions (e.g. pre-selection, command of problematic situations, education controls). Furthermore, e-assessment can be used for various target groups such as (potential) apprentices, students, specialist workers and middle management. Empirical studies which deal with the prevalence of e-assessment are mostly of an exploratory character. The opacity of the multitude of tests and uncertainty as to their quality are considered by businesses to be an obstacle to their take-up of the eassessment procedure. In order to evaluate the quality of the procedure for a jobrelated appraisal of suitability, quality standards were developed at different levels. At the international level we have from AERA “Standards for educational and psychological testing”, “Principles for the validation and use of personnel selection procedures” from SIOP, and the standards from the “Task Force on Test User Qualifications”. In the German-speaking world we have DIN 33430, whose focus is on the entire process of suitability appraisal. The efforts concerning all aspects of DIN 33460 are to be critically reflected upon against the background of various approaches to evaluation (e.g. responsive evaluation). Further points of critique are adequacy and timeliness.
3 Diagnosis of the Construct ‘Competence’ In the context of e-assessment, bespoke procedures for competence measurement should be deployed and related to one another, in order to capture the competences of the individuals holistically in terms of occupational competence – the competence to act autonomously in the chosen occupation/work situation. Occupational competence is achieved when a relationship is present, or is established between the competence dimensions – technical, method-based, social and co-operative. Competences are not constituted by the accumulation of knowledge for testing. Rather, competences are to be interpreted as the “holistic development of capabilities, units of knowledge and lines of reasoning, as relevant to the action in question” [1] (translated by the author). What then is a holistic comprehension of competences? Two concepts can be differentiated: the first seeks a process which captures the individual’s capability for action as exhaustively as possible; the alternative seeks to analyze discrete competence dimensions through various means. It is, however, questionable whether a cumulative measure of various competence dimensions can satisfy the claim to capture results holistically. The relationship between the results is crucial.
E-Assessment: A Suitable Alternative for Measuring Competences?
545
Returning to the question as to which competences are necessary for the employment of an organization’s staff, the various competence dimensions – technical, method-based, social and co-operative competences – are given a particular value. There is disagreement in the research on competence development as to the role of the different dimensions, to do justice to the changes in working life. Great hope was and is attached to the concept of key qualification. The belief that a onesided focus on technical competence should be overcome gained currency. Technical competence alone was not sufficient to cope with the demands of the workplace, let alone to shape it. Within the competence development research, there is controversy surrounding which qualifications belong to the ‘real’ key qualifications. Nevertheless, almost all these approaches assume that the key qualifications concern superficial knowledge and capabilities. The great expectations vested in the concept of key qualifications, in accordance with Mertens, have not yet been met. A central assumption of the approaches to the key qualifications is that these cannot age and therefore their acquisition is more important that that of the technical competences. However, simply because technical competences become outdated with time, this does not mean that the technical competences fundamentally lose their value in the context of job-related activities. This is the first weakness in the debate about key qualifications. When it comes to meeting changing work requirements, not just the lack of technical competences, but in particular outdated technical competences become a decisive obstacle. The dominance of the discussion on key qualifications hides a second weakness of that debate: from the perspective of the individuals concerned it can be entirely reasonable to acquire competences in different technical areas in the context of an interdisciplinary focus. Furthermore, it is noteworthy that there are to date hardly any empirical comparative studies which prove that the possession of key qualifications significantly increases the likelihood of employment. In her empirical study, Müller [8] explores the extent to which the possession of key qualifications influences the likelihood of apprentices remaining in the same trade after completing their training. The ability to communicate and co-operate, self-regulated learning in accordance with PISA 2000, performance motivation as well as aspects of self-competence (such as self-effectiveness) were all included as key qualifications. She could not identify any favorable patterns in their progression through their chosen vocations as a result of the acquisition of key qualifications. A differentiated and critical analysis of approaches which deal with key qualifications is required; this is however complicated by the unclear and shifting use of terminology. It is then clear that the exclusive focus on key qualifications, whereby technical aspects (e.g. expertise as know-how imbued with experience) are diminished in their importance, proves inadequate. I now proceed to discuss the principles and models to which various approaches to competence research refer, and how they differ between competence dimensions (see Table 1). These ideas are the basis for judging the use of the e-assessment procedure. Bunk bases his definition of occupational competence on the construct of the occupation [3]. As a consequence of structural change, the boundaries between clearly defined occupations are increasingly blurred, which causes this definition to appear of limited suitability. A new model which does justice to the transformation on the internal and external labor markets is the concept of ‘employability’. This is defined as „the ability of a person, founded upon their technical and occupational
546
M. Kröll
competences and their ability to create value and to perform, to supply their labor and thus to enter into working life, to keep their job, or if necessary, to seek new gainful employment” [2] (translated by the author). The ideas surrounding the concept of employability are at least in part too unspecific to serve as a point of reference for the competence development of individuals, and should be put in concrete terms. Wunderer & Bruch [9] examine the distinguishing features of entrepreneurial competences, whose degree they consider to be dependent on five classes of coentrepreneurial behavior: (1) subcontractor, (2) co-entrepreneur, (3) entrepreneurially inclined staff, (4) staff with low co-entrepreneurial competence and (5) individuals who act as if sacked, are overburdened or procrastinate. Current approaches in the competence development research treat the competence to self-organize as being at the heart of the scientific debate [4]. In order to satisfy the requirements of the working environment (increasing complexity and insecurity), it is necessary for staff to possess the competences of self-organization and self-regulation. The actions of individuals must be increasingly self-directed. Competences are interpreted as a predisposition to self-organized action. They would escape direct scrutiny. Table 1. Points of reference for occupational competences Author
- technical competence (continuity: knowledge, skills and capabilities) - method-based competence (flexibility: procedural methods) - social competence (sociality: behavioral patterns) - co-operative competence (participation: ways of structuring) - ability to create value for the businesses, - ability to organize oneself and one’s career, - to market the abilities oneself which are in demand on the internal and external labor market.
Wunderer & Bruch 2000
Erpenbeck & Rosenstiel 2003, p. XV f.
Competence as a predisposition of self-organized action
- strategy-orientated problem-solving competence (deployment of value-creating innovations), - implementation competence (creating acceptance of innovative ideas) - co-entrepreneurial social competence (combination v. independent-autonomous action and co-operative-integrated behavior) - personal competences (attitudes, values system, self-perception) - activity-oriented competences (integration of emotions, motivations and experiences in drives of the will, etc.) - technical-methodical competence (solving objectiverepresentational problems) - social-communicative competences (developing group and relationship-orientated behavior, tasks and goals)
Occupational competences, relevant for future employment, are dependent on (1) the concrete work tasks, (2) the requirements of the work in terms of quality, and (3) the authorization which a work task attributes to individuals. In light of the view of competences as occupational competences of individuals (Ortmann, 2007), the
E-Assessment: A Suitable Alternative for Measuring Competences?
547
capacity to act (4), which they can develop and activate, is also relevant for their (continuing) employment. Furthermore the relevant occupational competencies (5) are subject to the repetitive interaction between the assignment of competences (in the sense of tasks) and the (potential) capacity to act of individuals. Clearly, selforganization and the corresponding (6) competence to self-organize play an evergreater role in working life. But without (7) a corresponding other-organization, e.g. from the side of the business, there is a risk that the efforts of self-organization will not have the desired results. In contrast to personality features, competences relate to a high degree to concrete situations which call for their application. The possession of competences can only be measured in context. In what manner will the e-assessment procedure be undertaken in order to capture the competences and/or personality features of individuals? It will either be based upon the established paper-pencil form, or new tests will be developed against the background of a firm’s specific understanding of competence. In the latter scenario, the firm would firstly determine precisely what its requirements were in terms of relevant competences which every staff member should possess [5]. A new requirement analysis would not need to be carried out for every new appointment. Corresponding approaches are based on extensive algorithms, whereby various competences are placed in hierarchic relationships with one another and a different weighting of the expression of competences follows. Such an approach that leads to the forming of statistical opinions is to be distinguished from concepts which seek to capture the entirety of a person (the clinical process of forming opinions).
4 Case Study of the Application of E-Assessment Based on an actual case study of a large energy trading company, the difficulties of the use of new e-assessment forms as an attribute-oriented selection process are demonstrated. Over the course of a project, running over several years, an eassessment tool was developed. In contrast to the classic online methods, this new form of e-assessment uses test landscapes and the development of short stories. Cover stories involve a gradual construction of drama and should leave a holistic impression on test persons, with potentially a meaningful impact. Various interrogation methods are assigned to particular objects of the test landscape, and the test persons are guided by a virtual test leader. Characteristic of this new form is a stronger use of mediaspecific conditions, manifesting in experience-driven question elements such as reaction tests or movement-related tests, together with multimedia extras in classic questioning. Through a real-time, standardised and self-explanatory reporting system, the test persons receive prompt feedback about their results. There is simultaneously an orientation towards DIN 334340. By means of this new concept, the firm strives to enhance its image through an applicant-orientated test. From the perspective of the test persons, these new forms are of interest due to their aim of reducing anxiety of the tests, increasing test motivation, individualising the length of the test, reducing the experience of strain, as well as enabling faster feedback. Firms pursue these goals in order to increase social acceptance of the test procedure.
548
M. Kröll
First the occupational orientation of the test persons is determined. When they move about in the test landscape and take a virtual tour of the firm or its departments, they can choose between the various test modules. For example: (1) Figural intelligence: the test persons proceed in a knowledge management course by inserting an object in a door lock. (2) Arithmetic intelligence: this form of intelligence is examined, for instance, when the test persons must find a particular poster in a poster display which reflects requested data. (3) Mechanical-technical understanding: the test persons must solve particular tasks in order to progress through an obstacle course. (4) Memory capacity: the test persons should note various paths within the building and between the buildings which are shown briefly. (5) Performance motivation: the LMI-test is used for this and a depiction along the lines of a paperpencil test follows (answers through clicks). (6) Concentration/ attention: the test person is confronted with dangerous situations and a role-playing adventure game in the test landscape (e.g. dust which shouldn’t be breathed in). (7) Learning potential: short stories are presented, for instance, about which test persons must later answer questions. (8) Fear of texts/exams: situations will be portrayed which typically trigger fear, ending with an evaluation via a fear thermometer “How scared are you at this moment?” (9) Stress management: short stories will be introduced as situational impulses. Questions are subsequently asked. It appears plausible that, through the implementation of this form of e-assessment, social acceptance rises compared with conventional forms of e-assessment, but this must be tested. Empirical studies generally indicate that the acceptance of internetdriven personality tests is comparatively high. It must furthermore be clarified whether the validity of the test results is influenced through test persons having a relatively large amount of freedom to choose their tests. There are no consistent research results as to whether paper-pencil tests or internet-supported tests lead to better results [6]. The test case has also highlighted the danger that primarily those competences will be rated as significant for competence measurement, in respect of which approved tests are already available. The existence of, for example, the competence for self-regulation, entrepreneurial competence or technical competence is not examined in this case study. On the other hand, it could be claimed that the points dealt with here mostly concern key qualifications. The results of Müller [8], which could not be confirmed, indicated that possessing key qualifications, such as performance motivation pursuant to LMI inevitably leads to improvement in the employment situation. Finally it should be noted that, in the context of the tests, personality features are predominantly measured. These features are comparatively stable over time and only change to a limited extent. The results of such tests offer little indication at present as to how the test persons may further develop their competences.
5 Conclusion and Outlook The advantages and disadvantages of computer-aided diagnostics should be taken into account for their professional application. The following disadvantages may arise: high purchase costs, additional expenditure through the creation of an infrastructure, software-ergonomic deficits, hardware and software non-compatibility, high costs
E-Assessment: A Suitable Alternative for Measuring Competences?
549
through long dial-in times, low quality of e-assessment concepts (e.g. often simply 1:1 replication of paper-pencil tests), out-of-date content and unsuitability of the norms (see DIN 33430), inflation of applicant numbers, unsecured access of the test persons with the computer, direct contact with the applicants taking place relatively late, and abuse or dissemination of model answers. The advantages include: savings of time, space, material and personnel, increased independency of time and place constraints in carrying out the tests, more precise recording of time, greater flexibility in the possibilities for constructing the test, valid results and better security against error, greater neutrality and objectivity due to removal of the test leader effect, image benefits as an attractive employer, real-time analysis of results and quicker feedback, as well as higher acceptance through greater consideration of the needs of the test persons. The degree to which the advantages and disadvantages of e-assessment are manifest is dependent upon how e-assessment is deployed and embedded organizationally. The latter point is central task for the future utilization of e-assessment. In light of the current level of experience it becomes clear that use of the eassessment procedure at present is only partially able to contribute to overcoming central challenges which arise in relation to the professionalization of competence development (e.g. competence transfer, evaluation of competence development). Against the background of the current usage of the e-assessment procedure, the preselection of applicants is in the foreground. The demands that will be made of staff in the future are for many reasons increasingly more difficult to predict with precision. At the same time, the scientific debate presumes that a precise determination of competences is only possible if the demands on the (potential) staff members are available in a specific form. This insight confronts the application of the e-assessment procedure with huge challenges. Moreover the e-assessment procedure at the level of the test persons can lead to an intrapersonal contradiction: people ‘slip’ into the role of an object under examination and/or they experience it as such. However later on in working life they will be expected to act entrepreneurially and on their own authority. The following research questions are for consideration in the further scientific debate: how can internet-based procedures in the strategy of personnel recruitment and human resource development be better integrated? How high is the unintentional drop-out rate of competent applicants during e-assessment processes, who would be of interest to the firms? Could the results of e-assessments provide a foundation for the construction of skill or competence databases in firms? To what extent can the results of e-assessments be relevant to the employees of the HR department, managers, directors and the individuals themselves? How can/is the connection between the (relevant) competences of individuals and the firm capabilities considered? Under what conditions could the results of e-assessment provide the foundation for organization and control of competence development at the level of the firm and at the level of the individual? Depending on the answers to these questions, the next step is to elaborate the requirements in respect of the use of the e-assessment procedure.
550
M. Kröll
References 1. Bernien, M.: Kompetenzen messen und bewerten. In: QUEM (Hrsg.): Kompetenzentwicklung 1997. Waxmann-Verlag. Münster (1997) 2. Blancke, S., Roth, C., Schmid, J.: Employability “Beschäftigungsfähigkeit” als Herausforderung für den Arbeitsmarkt dem Weg zur flexiblen Erwerbsgesellschaft. In: Akademie für Technikfolgenabschätzung in Baden-Württemberg (Hrsg.): Arbeitsbericht Nr. 157 Stuttgart (2000) 3. Bunk, G.P.: Kompetenzvermittlung in der beruflichen Aus- und Weiterbildung in Deutschland. In: Kompetenz: Begriff und Fakten. Europäische Zeitschrift Berufsbildung H. 1, S. 9–15 (1994) 4. Erpenbeck, J., Rosenstiel, L.v. (Hrsg.): Handbuch Kompetenzmessung. Schäffer-Poeschel Verlag, Stuttgart (2003) 5. Kirbach, C. et. al,: Recruting und Assessment im Internet. Vandenhoeck & RuprechtVerlag Göttingen (2004) 6. Klinck, D.: Computergestützte Diagnostik. Göttingen u.a. Hogrefe (2002) 7. Konradt, U., Sarges, W.: E-Recruictment und E-Assessment. Göttingen Hogrefe (2003) 8. Müller, K.: Schlüsselkompetenzen und Verbleib nach der Ausbildung. Dresden (unveröffentlichte Dissertation (2007) 9. Wunderer, R., Bruch, H.: Unternehmerische Umsetzungskompetenz. Diagnose und Förderung in Theorie und Praxis, Vahlen, München (2000)
Green Advocate in E-Commerce Ying-Lien Lee1, Fei-Hui Huang2, and Sheue-Ling Hwang3 1
Dept. of Industrial Engineering and Management, Chaoyang University of Technology, No. 168 Jifong E. Rd., Wufong Township Taichung County 41349, Taiwan [email protected] 2 Dept. of Marketing and Distribution Management, Oriental Institute of Technology, No. 58,Sec.2,Sihchuan Rd.,Pan-Chiao City,Taipei County 22061,Taiwan [email protected] 3 Dept. of Industrial Engineering and Engineering Management, National Tsing Hua University, No. 101, Section 2, Kuang-Fu Road, Hsinchu 30013, Taiwan [email protected]
Abstract. The continuous growth of e-commerce sees waves of information explosion. Online shoppers have to confront with more information than ever when they are making purchasing decisions. Among the tools that try to reduce this burden of information overload, recommender system is one of the widely employed techniques which can be seen in stores such as Amazon.com and iTunes Store. This paper presents an approach of the interaction design of recommender system in the context of green digital products. By cultivating the field of game design, elements that make game fun and engaging are borrowed and applied to the design of the recommender system to motivate shoppers to opt for greener choices. In addition, the idea of Kansei Engineering will be employed in the system to recommend according to the perceived characteristics of products. A framework of such system will be described, along with future extensions of the framework in the realm of e-commerce. Keywords: e-commerce, recommender system, green digital product, game design, interaction design, Kansei Engineering.
information overload. This research focuses on recommender systems among these shopping tools. Conventional recommender systems usually base their recommendation mechanism upon profit revenue or customer preference. Such systems recommend items so that the potential profit for the store can be higher, or that the customer is more likely to prefer if the customer actually buy items in the recommendation list. However, such profit or preference seeking systems may not be adequate when there is an agenda that the store wants to advocate, or when the preference of a customer is not easy to determine. Consider green consumer electronics. More and more companies are promoting their brand images or products by adopting green practices or green design. They utilize the trend of environmental awareness and set their green marketing strategies, but such trend is not reflected in the design of recommender systems. Moreover, greenness of a product may drive the price higher or lower. Customers need to be informed and encouraged to buy green products. Recommender system can take on the role of an advocator in this respect. As to the determination of the preference of a customer toward a product, some product fits the bill better than the other. Take movies for example. Each movie has a set of characteristics that can be directly related to customers’ preferences, such as casts, directors, and movie genre. It is the same with music or books, but not quite so with consumer electronics. Recommender systems for consumer electronics are usually specification-wise, which means they recommend products that are similar to each other in terms of specifications. Such design is useful for consumers who have a clear utility goal in mind, but it is only a specification-wise match between a product and a consumer’s functional needs. Although some products are perceived as fashionable and some others as professional, items are still categorized according to their specifications, not their perceived characteristics. In addition, for specification-based recommender systems, it is difficult in the regard of recommending across product categories. For example, a fashionable customer has bought a fashionable mobile phone and now he or she wants to buy a headset that is as stylish as the phone. Since the specification ontology of these two types of product is different, it is not easy to recommend products by specification only. Additional characteristics need to be considered for this purpose. To extend the field of recommender systems toward the directions mentioned above, this research proposes a framework that integrates an advocate module and a Kansei (“feeling” in Japanese) module in addition to a conventional recommender module, and the framework will be illustrated in the context of green consumer electronics. The task of the advocate module is to recommend greener products by a mechanism based upon a theory borrowed from game design called Flow Theory. This module tries to advocate green consumerism by advancing consumers’ green appetite in a progressive way. The task of the Kansei module is to recommend products according to the perceived characteristics of the products whose perceived characteristics are defined as a set of Kansei words. A Kansei scoring system will also be developed to encourage consumers to contribute Kansei words, much like the concept of Web 2.0. A fuzzy inference system is used to synthesize the inputs from these three modules.
Green Advocate in E-Commerce
553
2 Literature 2.1 Game Design Games have a lot to offer when it comes to making systems fun and captivating. Applying game-like interface or game-like mechanism is quite promising in the design of ordinary systems, such as process management tools and educational programs. A theory which game design bases heavily upon is the Flow Theory by Csíkszentmihályi [1]. Csíkszentmihályi studied extensively to find out what makes enjoyable experience, or flow experience, which is an experience “so gratifying that people are willing to do it for its own sake, with little concern for what they will get out of it, even when it is difficult or dangerous”. An important condition for the flow experience is the match between the challenge of a task and the skill of a user. 2.2 Kansei Engineering Kansei Engineering (KE) is a methodology established by Nagamachi in 1970 to identify the relationship between consumers’ feeling and design elements [2]. It has been successfully applied to areas such as automotive industry, construction machinery industry, office and home appliance, house construction, fashion industry, and so on. There are three types of procedure to apply KE [3]: Type I is category classification, Type II is computer-assisted system, and Type III is mathematical modeling. The Kansei module used in this research is closely related to that of Type II KE.
3 The Proposed Framework 3.1 General Structure and the Framework There are four major modules in the proposed framework: Recommender module, Advocate module, Kansei module, and Fuzzy Inference System. Recommender module is responsible for recommending products based upon product information and derived consumers’ needs, which works like most conventional recommender systems. Advocate module is responsible for deciding a suitable level of product greenness for a consumer. Kansei module is responsible for the determination of the match of users’ Kansei requirements and the perceived characteristics of products. Finally, Fuzzy Inference System is responsible for fusing together the outputs of the three modules and other user-defined criteria. The general structure of these modules is shown in Fig. 1, where the inputs (user model and product model) and outputs (recommended items) are also included. 3.2 Determination of Product Greenness Different eco-labels have different meaning. Some refers to the raw material while some refers to the energy efficiency. To determine the overall greenness of a product, the framework uses Analytical Hierarchical Process [4], or AHP, to integrate experts’
554
Y.-L. Lee, F.-H. Huang, and S.-L. Hwang
Fig. 1. General structure of the proposed framework
opinions on how important an eco-label is to the greenness of a product. The hierarchy can be one or two levels. Once the weight of each eco-label is determined, the approximation of product greenness can be calculated. Let PEi be the row matrix of eco-label status of product i. The j-th item in the row matrix is either 1 or 0 indicating whether product i has ecolabel j or not. Let EW be the column matrix of eco-label weights obtained from the AHP mentioned above. Then, the overall greenness of product i, or PGi, is PEi × EW . 3.3 Determination of Consumer Green Consciousness In the proposed framework, the green consciousness of a consumer is defined as the relative weight of product greenness among other subjective criteria. For example, in some products, greenness may drive the price higher or lower. It may also impede or facilitate some product features, such as slow computing speed for energy efficiency. AHP is also used in the elicitation of the weights, and to simplify the process, only one level is used. The weights of these factors are then stored in User Profiles and will be used in the Fuzzy Inference System. 3.4 Product Kansei Tagging To encourage users’ involvement in the product Kansei tagging effort, a user reputation system is developed which works like a product rating system. In a product rating system, a user can rate and comment on the product, while other users can rate his or her ratings and comments. If the ratings or comments are well-received by other users, the user who gives ratings or comments can build up his or her reputation in the website. The process of Kansei-tagging is very similar to that of rating a product. Once a user becomes involved in a Kansei tagging session, he or she can tag a product with a set of Kansei words that describe the perceived characteristics of the product in question. Then, another set of Kansei words of the product is randomly retrieved from the Product Kansei Tags database. If there is no previous data for this product, an initial set of Kansei words is used instead. This initial set can be supplied by the retailer or by the brand owner of the product. The matching score between these retrieved and current sets are calculated and stored in User Profiles along with the set of Kansei word. This step updates the part of the User Model that indicating what
Green Advocate in E-Commerce
555
Kansei words are more likely to be associated with a user. To update the set of Kansei words of the Product Model, words which are frequently used as tags and highly matched are appended to the set of Kansei words of the product in question. Less frequent and lowly matched tags can be removed. The process of calculating the matching score of the retrieved and current sessions can be described as a “gaming with a purpose” (GWAP) scenario. GWAP is a class of games “in which people, as a side effect of playing, perform tasks computers are unable to perform“ [5] . In our case, tagging a product with Kansei words that describe the perceived characteristics of a product is not easy for a computer to perform. To engage users in the tagging, the framework pairs the current user with a previous to create the sense of competition or social interaction, which is an output-agreement game according to Ahn & Dabbish [5]. 3.5 Advocatory Recommender
ha nn el C ow Fl
Product Greenness
The core part of the framework is the advocatory recommender, which consists of a green advocate module, a Kansei ranking module, a recommender module, and a fuzzy inference system. The advocatory recommender is activated explicitly or implicitly. By explicit activation of the advocatory recommender, a user actively seeks for advices on what he or she can buy. The user can choose to enter some Kansei words as the criteria of the perceived characteristics of product. Adjustments of the weights of the criteria can made at this moment, as well as the addition of extra criteria, such as brand or year of make. Those are user inputs. Then, the green advocate module is invoked to determine an appropriate range of greenness of products according to the user’s green consciousness. Finally, the fuzzy inference system is used to synthesize the outputs of the modules mentioned above according to the weights of user criteria.
Fig. 2. The flow channel of the green advocate module
The green advocate module adopts the concept of Flow Theory and provides a progressive challenge that encourages users to opt for greener choices over time. Consider the three dotted circles in Fig. 2. In the normal mode, this module recommends products whose greenness matches the green consciousness of a user, such as circle A in the figure. After some sessions of purchase, the module tries to recommend products whose greenness is a bit over the users’ green consciousness, as the
556
Y.-L. Lee, F.-H. Huang, and S.-L. Hwang
circle B in the figure indicates. If the user finds it acceptable and chooses it, then the module will give the user a “level up”, or moving up a notch, in the axis of green consciousness, as the circle C indicates. If the user doesn’t choose it, then the module should fall back to circle A. The advocatory recommender uses a fuzzy inference system to integrate multiple decision variables. The proposed rules for this inference system are summarized in Table 1. In our proposed form, only two decision variables are considered in each rule, and only two levels are considered in each variable. However, it is possible to extend to consider more variables and more levels. Table 1. The proposed rules of the fuzzy inference system Decision Variables Greenness
Customers with higher weight Feature-demanding
Price
Feature
High
Rich
N/A
High
Poor
N/A
Low
Rich
N/A
Low
Poor
N/A
Price-sensitive demanding Price-sensitive
High
N/A
High
Green-conscious
High
N/A
Low
Low
N/A
High
Low
N/A
Low
rule
Output Neutral Undesirable
and
feature-
Favorable Neutral Neutral Undesirable
Price-sensitive conscious Price-sensitive
and
green-
Favorable Neutral
4 Conclusions and Future Research The process of decision making of online shopping is inherently complex in this information age. On the one hand, information can be gathered easier and quicker than ever, promising to solve the problem of information imbalance. On the other, the volume of information is to vast for us to take advantage of, creating another problem of information overload. Several solutions to this dilemma are put forth, and recommender system is among the widely used. A recommender system mimics the interaction of people asking for advices from friends, gurus, or sales person. Some notable examples of recommender systems can be found on Amazon.com, Netflix, or iTunes Store. However, most recommender systems base their recommendation on profit or user preference. What if there is an agenda, such as green marketing, which a store wants to advocate? Given the rising concern of the environmental issues, this paper proposes a framework of recommender system that has a green advocate module that encourage users to opt for greener products in a progressive way. The green advocate
Green Advocate in E-Commerce
557
module is based upon a theory called Flow Theory borrowed from the field of game design. Products such as movies and music albums have a lot of characteristics that can be related to customers’ preference. Yet for products like consumer electronics, specifications and functions are usually used in recommender systems. To enrich the recommendation, the concept of Kansei Engineering from the field of Industrial Design is used in the proposed framework. Users are encouraged to tag products with Kansei words that describe the perceived characteristics of products. The tagging process is based upon the concept of gaming with a purpose, and the score of the tagging game can help the user build his or her reputation on the e-commerce website. Recommender system can take advantage of the Kansei tags of products and Kansei requirements of users to generate recommendations by determining the similarity of a product and user’s Kansei requirements. The integration of the conventional recommender module, the Kansei module, and the green advocate module is achieved via a fuzzy inference system. Acknowledgments. The authors would like to express their gratitude to National Science Council of Taiwan for the funding under the grant number NSC-97-2221-E324-018-MY3.
References 1. Csíkszentmihályi, M.: Flow: The Psychology of Optimal Experience. HarperCollins Publishers, New York (1991) 2. Nagamachi, M.: Kansei Engineering: A new ergonomic consumer-oriented technology for product development. International Journal of Industrial Ergonomics 15(1), 3–11 (1995) 3. Nagamachi, M.: Kansei Engineering: An ergonomic technology for product development. International Journal of Industrial Ergonomics 15(1), 1 (1995) 4. Saaty, T.: Risk-Its Priority and Probability: The Analytic Hierarchy Process. Risk Analysis 7(2), 159–172 (1987) 5. Ahn, L.v., Dabbish, L.: Designing games with a purpose. Commun. ACM 51(8), 58–67 (2008)
Gesture-Based Sharing of Documents in Face-to-Face Meetings Alexander Loob and Christian Rathke Hochschule der Medien (Stuttgart Media University), Wolframstraße 32, 70191 Stuttgart, Germany {loob,rathke}@hdm-stuttgart.de
Abstract. Many of the Electronic Meeting Systems on the market today only support sharing of electronic documents via email or a common user space. This kind of sharing enforces short work interruption and requires substantial mental effort than necessary. This contribution describes the development of a simple EMS, its shortcomings and the further development to a gesture-based EMS for sharing documents in face-to-face meetings. The system implements the concept of a “throwing” gesture for transferring documents to one or more participants. This gesture is explained and evaluated in further detail. Keywords: Gesture-based Interaction, Document Sharing, Electronic Meeting System, face-to-face meeting, reduction of mental effort and work interruption.
Gesture-Based Sharing of Documents in Face-to-Face Meetings
559
documents by dragging them to graphical representations of the participants. In a second step, the system is augmented by providing means for a gesture-based interaction which alleviates the necessity of the graphical representations. Our effort is embedded in our more general strive to design and develop theories, methods and tools to support cooperative work. We concentrate on face-to-face situations where people meet to work together as part of an overall purposeful collaborative process.
2 Meeting Support Infrastructure As a starting point for our research, we devised a simple EMS – called the Workshop Trader (Fig. 2 and Fig. 3) – as one of the software components of our Corporate Communication Lab [3]. In this Lab we pursue research in collaboration technology and explore improvements of common work situations such as collocated or distributed meetings. The Lab’s conference table is composed of several modular working places with touch-sensitive displays. It features one large wall-mounted display for moderation and presentation, and one large height-adjustable display, both of which are also touch-sensitive. The contents of any of the individual table displays may be mirrored to the two large displays (Fig.1).
Fig. 1. The Corporate Communication Lab at the Stuttgart Media University
The Workshop Trader software is capable of representing meetings and their respective participants. The clients of its client server architecture run on all of the Lab’s displays. They can be used for sharing electronic documents between the meeting’s participants. To support a wide spectrum of electronic devices and operating systems in addition to the Lab’s hardware, the entire Workshop Trader application is implemented in Java. The client application in particular does not require any additional software
560
A. Loob and C. Rathke
Fig. 2. The Workshop Trader
libraries. As long as a Java virtual machine (JVM) is available for the desired target system, the Workshop Trader client may run on the participant´s device. To start the collaboration process, each participant can choose a user and a meeting session for login to join the meeting. After joining a session each available recipient and additionally the group as a whole are represented by rectangular areas, respectively (Fig. 3). Using the client, any visible document on the screen may be dragged from its location and dropped on one of these areas.
Fig. 3. Representation of the participants of a meeting session
As a result, the electronic document is transmitted to the Workshop Trader server and it appears as a small icon within the sender’s area at the recipient’s client. From there, the document may be opened directly or copied to the recipient’s private environment. Due to security and (network) performance reasons the document will not be copied to the target system until the recipient accesses the document.
Gesture-Based Sharing of Documents in Face-to-Face Meetings
561
3 Shortcomings of the Trader System One of the shortcomings of interacting with the Trader or similar systems is the additional effort necessary to identify the area on the screen which is associated with the intended recipient. This can be interpreted as a more general issue when dealing with electronic representations of real world situations: The user is required to interrupt the intuitive flow of operation by mapping the real world situation to the electronic counterpart. According to Schön [4] professionals engage in situated work until their expectations are not met. Then a breakdown is experienced and they have to stop and reflect on how to overcome the current breakdown. In design activities, these breakdowns provide an opportunity to learn and construct new knowledge [5]. In face-to-face meetings, this breakdown diverts from the situated flow of activities because it forces the participant to realize and deal with procedures which are not an intrinsic part of the task at hand.
4 Our Approach Currently, the electronic document to be shared must be dragged from the desktop and dropped onto the application’s virtual representation of a meeting participant. This informs the application to transmit information about the shared document to the target participant. Compared to traditional approaches, the Workshop Trader effectively facilitates sharing of documents, but sharing activities requires explicit interaction with the application. Users need to be aware of its existence. They must know how to tell the system what they expect. Interacting with the application may be compared to asking a person to pass a document to someone else. In this case, communication would be done verbally, i.e. by using natural language speech. But communication with a third person is not necessary if the contributing participant has a way of directly addressing the intended recipient. For instance this is the case if the recipient is an immediate neighbor or the group is small enough for each member to be within direct proximity. In situations like this, the document is passed directly. No intermediate is involved. To transfer this direct mode to the electronic environment, we have developed a mechanism, which allows a person to use a “throwing” gesture for the purpose of passing an electronic document to another participant. Following this approach, we make use of an intuitively used gesture for transferring a physical object to another person “on the other side of the table”. No mental effort is required for becoming aware of a third party or some secondary representation of the recipient. A basic requirement to make use of the natural gesture for sharing a document is the spatial representation of all participants. The system must know each participant’s position and heading (Fig. 4). Additional information regarding position and heading in the physical environment is specified by the participant during the login process. The throwing gesture may be executed by using any of several pointing devices including fingers on touch sensitive surfaces, styluses on tablet PCs, or simple mouses. Also, the movement of the pointing device must be recognized as a throwing gesture and correctly interpreted as pointing to a unique target person.
562
A. Loob and C. Rathke
Fig. 4. Spatial representation of the participants of a meeting
Recognizing a movement as a gesture (not necessarily a throwing gesture) is simply done by measuring the velocity of the pointer. Any movement above a certain threshold is determined to be a gesture. The current implementation of this gesture recognition mechanism does not support different settings for the velocity threshold depending on the diagonal and the resolution of the input media. Prerequisite for the “throwing” gesture is the “grabbing” of the electronic document, which is synonymous with pressing of the left mouse button (resp. touching a touch-sensitive surface). Once this prerequisite is fulfilled, all movements of the pointing-device including the coordinates and the timestamp are recorded from this point of time. Following the completion of the “throwing” gesture by releasing the left button of the pointing-device, the recorded movement- and time-information is cleared of irrelevant information. For this purpose, the velocity in pixels per second is calculated continuously between two consecutive measuring points. Once the velocity is lower than the velocity threshold, the measuring point recorded earlier in the timeline will be removed. In the next step, the measuring points at the end of the potential gesture are inspected and the velocity continuously between two consecutive measuring points is calculated. In case the velocity value is lower than the given threshold the measuring point recorded later in the timeline is removed (Fig. 5). The red line in figure 5 shows the velocity curve between the measuring points remaining after the reduction mechanism. If there are a minimum of three measuring points or more remaining after the reduction of the measuring points the movement can be interpreted as a gesture (but not implicitly as a “throwing” gesture). In addition, a straight throw to one participant is separated from a round movement, which is interpreted as a throw to all participants.
Gesture-Based Sharing of Documents in Face-to-Face Meetings
563
Fig. 5. Example of a velocity curve of the pointer movements
In order to be able to distinguish between these different “throwing” gestures the absolute values of the angle-differences between two consecutive measuring points are calculated at each occurrence and added up. Afterwards this sum is divided by the amount of the measuring points. The result of this calculation is then compared with another given threshold which is used to differentiate between a straight “throwing” gesture and other gestures. In case the value is lower than this given threshold, the recently captured movement can be determined to describe a straight “throw” and so the gesture can then be interpreted as a document sharing offer to one single participant of the meeting. If the result of this calculation represents a higher value than the threshold the captured movement is inspected more closely. The goal is to be able to determine if the movement described a circular path and can be interpreted as a document sharing offer to the whole group of participants. Based on each participant’s position and on the throwing angle of the gesture refined using the heading of the source participant, the target person can be identified by the source person (Fig. 6). Special care is taken to distinguish between two or more potential targets and hidden participants. Comparable to the physical environment, participants could be covered up by other participants in the virtual representation, too. For example, a participant could be positioned behind another participant from the view of the “throwing” person. This possible cover-up-effect needs to be calculated by each client separately. In case participants are out of view, they are marked as non-reachable persons in the set of the potential targets. Given the calculated angle of the “throwing” gesture and the heading of the source participant it now becomes possible to determine a maximum of two participants extracted from all potential target participants. Here, one participant above and one participant below the angle of the “throwing” gesture are selected. Each participant shows the lowest difference between the angle from the view of source participant and the angle of the throw.
564
A. Loob and C. Rathke
Fig. 6. Identification of target
In Figure 6 the dotted line represents the angle of the real “throwing” gesture. The solid lines represent the angles to the two closest target participants from the view of the source participant. The participant with the smallest distance to the “throwing” angle receives the document from the sending participant. The real implementation of this gesture recognition mechanism is more complex than described here and several more specific cases are considered (e.g. zero degree pass through). However, discussing these details is not in the scope of this paper. Furthermore, it is possible to realize the recognition of other gestures with this gesture analysis method in the future. Due to several reasons, an individual analysis method for gesture recognition and interpretation was implemented. We decided not to use existing gesture recognition libraries (e.g. iGesture [6] or Mouse Gestures [7]). One of the reasons is that we plan to port our approach to the Java Mobile Edition at the next major release to integrate participants with touch-sensitive smart-phones in such meetings.
5 Evaluation / Testing For testing and evaluation the gesture controlled EMS described in the previous chapter we designed a multi staged testing and evaluation concept. All tests of these stages will be done with a special testing environment that allows us to record all important information for analysis afterwards in a very detailed way. This testing environment consists of a simple desktop with a dummy document placed in the center of it. In different sessions virtual participants can be placed in the surroundings to simulate several meeting situations. These sessions can be stored for later use. At the first stage of the testing and evaluation concept the testing environment is used to enhance the analysis method and adjust the used thresholds. Different input media requires different thresholds. Given touch-sensitive displays, the velocity
Gesture-Based Sharing of Documents in Face-to-Face Meetings
565
threshold depends on the size of the display and the used resolution. In this case it is possible to calculate the dots per inch (dpi) of the input media and vary the velocity threshold depending on the dpi. Given other input media like mouses or touchpads of notebooks, it is not as simple to calibrate this threshold because the required values cannot be generated automatically. This stage is also used to implement new gestures to control our EMS application. At the second stage of the testing and evaluation concept several test persons share the dummy document placed in the middle of the screen with a randomly selected virtual participant. This stage is divided in two phases. In the first phase the test person receives feedback if the submitted movement was recognized as a “throwing” gesture in addition to the time when it was recognized, if the gesture hits the right participant. The test person will have ten tries to input a gesture and hit the right participant. With this test we hope to measure the learning effect given by the visual feedback. In the second phase of this test the test person does the same test but without any feedback. In this phase the test person does also have ten tries. During both tests all movements are recorded and the differences between the actual gesture angle and the correct angle to the randomly selected participant are recorded, too. These tests will also be done with different input media. As reference media we use the mouse as a standard input device. At the third stage of the testing and evaluation concept we try to measure the differences of the mental effort between the use of the Workshop Trader and the use of the gesture based EMS. In this test several test persons share dummy documents. At first the documents are shared using the Workshop Trader. Then, same documents are shared using the gesture based EMS. This complete test will be recorded with an eye tracking application in one of our usability labs and evaluated afterwards. At the moment (February 2009) we are in the first stage of this testing and evaluation concept, but we strongly hope that we can present the final testing results of stage two and stage three in the second quarter of 2009.
6 Other Approaches to Sharing Documents in Face-to-Face Meetings An absolutely classic approach to sharing documents is via network file systems e.g. provided by Active Directory Services [8] or a samba server [9]. But these solutions have a high administrative effort to grant user access, so that this is inconvenient especially for including external persons. Another approach, pursued by some Electronic Meeting Systems, is the distribution of documents per (invitation) email before a meeting (like Meeting Sense Software Corporation MeetingSense [10]). Other systems [1, 2] have a common space, where documents may be placed before or during the meeting. The access to these systems is often granted by an invitation email to the participants of the meeting. Thereby roles and rights in the EMS can be assigned by the inviting person or an administrator of the EMS.
566
A. Loob and C. Rathke
7 Other Approaches of Gesture-Based Interaction Gesture-based interaction with electronic devices is actively pursuit in research and development. First outcomes of these activities are already commercially available such as the ART+COM Touchmaster [11], the Apple iPhone [12, 13], the Google Android Operating System [14], most of the HTC Phones [15], the HP TouchSmart PC [16], the Microsoft Surface [17], and the Nintendo Wii Remote controller [18].
References 1. GroupSystems, ThinkTank: The Leading Virtual Interactive Meeting, Brainstorming, Facilitation & Collaboration Technology (2009), http://www.groupsystems.com/technology/thinktank 2. Grouputer Solutions. Overview (2009), http://www.grouputer.com/grouputeroverview.html 3. Hochschule der Medien (2009), Corporate Communication Lab, http://www.hdmstuttgart.de/wi/is/laborwiese/ccl/index_html 4. Schön, D.A.: The Reflective Practitioner: How Professionals Think in Action. Basic Books, New-York (1983) 5. Fischer, G.: Turning Breakdowns into Opportunities for Creativity. Knowledge-Based Systems, Special Issue on Creativity and Cognition 7(4), 221–223 (1994) 6. Kurmann, U.: iGesture: A General Gesture Recognition Framework (2007), http://www.igesture.org/ 7. Smardec, Mouse Gestures (2008), http://www.smardec.com/products/mouse-gestures.html 8. Microsoft Corporation, Windows Server 2003 Active Directory (2009), http://www.microsoft.com/windowsserver2003/technologies/ directory/activedirectory/default.mspx 9. Hertel, C.: Samba: An Introduction (2001), http://us1.samba.org/samba/docs/SambaIntro.html 10. Meeting Sense Software Corporation (2008), How MeetingSense Works, http://www.meetingsense.com/meetingsense/ 11. ART+COM, A.G.: CeBIT 2009: ART+COM Technologies presents Touchmaster (2009), http://www.artcom.de/index.php?option=com_acnews&task=view& id=414&Itemid=136&page=0&lang=en 12. Apple, Inc. Apple-iPhone, http://www.apple.com/iphone/ 13. Hotelling, S., et al.: Gestures for touch sensitive input devices. United States Patent Application Ser. No. 12/118,659 (2008) 14. Google, Inc., What is Android? (2009), http://www.android.com/about/ 15. HTC Corporation. HTC Products (2009), http://www.htc.com/us/product.aspx 16. Hewlett-Packard Development Company, L.P, HP TouchSmart PC (2008), http://www.hp.com/united-states/campaigns/touchsmart 17. Microsoft Corporation, Microsoft Surface: The Power, The Magic, The Possibilities (2008), http://www.microsoft.com/surface/index.html 18. Nintendo, What is Wii? (2008), http://www.nintendo.com/wii/what/controllers
Developing, Deploying and Assessing Usage of a Movie Archive System among Students of Film Studies Nazlena Mohamad Ali1, Alan F. Smeaton2,1, Hyowon Lee2,1, and Pat Brereton3 1
Centre for Digital Video Processing and 2 CLARITY: Centre for Sensor Web Technologies Dublin City University, Ireland 3 School of Communications, Dublin City University, Ireland {nmohamadali,asmeaton,hlee}@computing.dcu.ie, [email protected]
Abstract. This paper describes our work in developing a movie browser application for students of Film Studies at our University. The aim of our work is to address the issues that arise when applying conventional user-centered design techniques from the usability engineering field to build a usable application when the system incorporates novel multimedia tools that could be potentially useful to the end-users but have not yet been practiced or deployed. We developed a webbased system that incorporates features as identified from the students and those features from our novel video analysis tools, including scene detection and classification. We deployed the system, monitored usage and gathered quantitative and qualitative data. Our findings show those expected patterns and highlighted issues that need to be further investigated in a novel application development. A mismatch between the users’ wishes at the interviews and their actual usage was noted. In general, students found most of the provided features were beneficial for their studies. Keywords: Video browsing, deployment effort, usage analysis.
but have not yet been practiced or deployed. The MovieBrowser2 system we developed incorporates automatic video analysis techniques, namely shot boundary detection, keyframe extraction and classification of scenes into action, dialogue or montage [3]. Our application domain is film studies where students need to study movie contents and to analyse sequences. We began with the identification of user needs through interviews and brainstorming, followed by sketching and prototyping an online system that incorporates the video analysis tools we offer as functional features. We then deployed this system, monitored its usage and gathered quantitative as well as qualitative data. In this paper we overview our developed system, discussing the findings from the user and usage analysis of our trial deployment.
2 Related Work Related deployment efforts of experimental video systems include Newsblaster at Columbia University [5], which automatically crawled news websites and summarised them for users. This was first deployed in 2001 and a number of user studies have been conducted; an Austrian interactive TV trial [1] deployed a novel TV application to a local cable TV provider in Salzburg, Austria, and ran for 4 months in 2004-5; Físchlár-News [4] incorporated a number of multimedia and recommendation techniques and was deployed within a University campus for 3 years, during which interaction logging and diary methods were used to capture its usage; SportsAnno [2], a video browsing system that allows users to make comments, share opinions and ideas on sports events, was deployed during the 6 weeks of World Cup 2006. Deploying experimental systems like these is always difficult because the underlying novel multimedia technologies, which they incorporate, are often beyond user expectations and the challenge is finding the right way to productively blend these novel functions into tools to support users’ tasks. Such efforts would show a growing awareness of the importance of user evaluation in realistic environments but studies which incorporate the end-user perspective from the conception stage of the project are rare; most technology trials start purely from a technical point of view and only after deployment do they try to get any form of feedback from real usage and users. Our work is similar to the other trial studies listed above, but instead of the technology being demonstrated dictating the design and deployment process, we based the development process from usability engineering, focusing on user-centred design and that is the significant differentiator of our work.
3 System Descriptions MovieBrowser2 was designed to support two undergraduate film studies course modules at Dublin City University (CM272 National and Ireland Cinema and CM135 Analyzing Media Content). In both modules, students were required to ‘read’ a variety of movie sequences in detail referring to the process of analysing a movie sequence closely in order to understand different levels of meanings intended by its creators and manifested from the elements like framing, music, plot, camera angle,
Developing, Deploying and Assessing Usage of a Movie Archive System
569
lighting and so on. Libraries of 30 movies were made available on MovieBrowser2. They include various genres (comedy, drama, romance, action, etc.), and ranged from contemporary Hollywood movies to old Irish movies. Fig. 1 shows examples of MovieBrowser2 screenshots. Fig. 1(a) shows a screenshot of the front page that displays movie posters and some movie information. Filtering by movie genre and director are provided by a drop down list at the top area. Fig. 1(b) shows a screenshot of the main viewing area with a visual timeline corresponding to events like dialogue, montage and exciting on the top area, shot keyframe view, the playback area and note-taking section under the playback area.
Fig. 1. Interface screenshots: (a) front page, (b) main view area
The note feature is public, similar to the common commenting features of blog sites. Each shot keyframe represents an automatically detected scene, clicking on a keyframe will play video from the scene. The similar colour schemes for event classifications were used in the timeline and the shot keyframe’s borderline and text (i.e. yellow colour for exciting events). The scenes also can be filtered using radio buttons. The playback from the keyframe list will play the sequences, while playing the whole movie can be carried out using the ‘PLAY’ button on top of radio buttons.
4 Usage and User Analysis 4.1 User Access MovieBrowser2 was deployed throughout the Spring semester (11 weeks) of 2007-8. Out of the total of 268 students in both classes, 107 students (40%) accessed MovieBrowser2. From the actual log data collection, we see that on average, almost all 107 students accessed MovieBrowser2 at least 2 times during the trial period. A vast majority of the students (84%) accessed the system for less than 2-hours. Only 17 users (16%) from the total group accessed between 2-8 hours in total. In thinking about the number of hours our students used the system over the semester, it is worth noting that:
570
N. Mohamad Ali et al.
• MovieBrowser2 was used as a complementary tool in the movie analysis classes where the lecturers encouraged the students to use the tool and there was a short introductory sessions conducted during the classes; • MovieBrowser2 featured some movies that are not available from the University library's collection especially for Irish movies; • MovieBrowser2 can be accessed only within campus. This means those students who want to work on their essay at home during the weekend or evening are unable to access the system; • MovieBrowser2 was developed for a specific technical environment in which it was deployed (computer labs in School of Communications, DCU), consisting of Microsoft Windows XP and Microsoft IE v6+. Thus compatibility with other machines and browsers when some of the students tried to use their own laptops was an issue (as found in post-trial questionnaires). The total access duration time was around 86 hours during the trial (CM272: 57 hours; CM135: 29 hours). Access duration time for CM272 was almost double than the other module. This may be because the assignment for the former class required students to use Irish movies as examples which were mostly not available in the university library, whereas the assignment for the latter class was not restricted to Irish movies thus much more accessible from conventional sources (library, DVD rental, cable TV, etc.). From the total movie collection, 23 were Irish movies with 7 contemporary Hollywood added to the collections. Our justification in having students from the CM272 and CM135 module was because these two modules were running during our trial semester and had a similar nature of textual analysis assignment. In this work, we are not focusing on comparing each module specifically but mainly to examine students’ general access patterns. All 7 Hollywood movies that were stored in the system library were accessed a total of 73 times (39%) with the movie Shrek (2001) mostly accessed by students, 24 times. Irish movies were accessed in total 116 times (61%) with the movie About Adam (2000) the most frequently accessed 21 times, followed by The Butcher Boy (1997) at 20 times accessed. A few short movies such as The Visit (1992) and Bent Out of Shape (1995) had no access at all by students. 4.2 Features Access We divided our movie collection into ‘Advanced’ and ‘Basic’ types for navigating movie content and the reason behind this idea was to see the pattern of user interactions when some added technology features are incorporated. The advanced type of browsing consists of some features that could enhance user browsing and navigating of movie content on top of standard playback features. These features are mainly designed to enhance film-reading based on the three events categorizations (i.e. montage, dialogue and exciting). The basic type incorporates only standard playback features (i.e. pause/stop/slider bar and etc). User interactions are captured and represented as in Fig. 2. The result shows that the percentage of interactions on the features: ‘Basic’ and ‘Advanced’ have a similar patterns in playback movie activities. ‘Click the whole movie’ has the highest interaction that shows activity in watching the whole movie, while ‘click play button’
Developing, Deploying and Assessing Usage of a Movie Archive System
571
Fig. 2. Types of interactions
denoted the activity of playing a movie after it being paused. User activities such as watching or playing a sequences or the whole movie were identified from user-action entries such as ‘click pause button’, ‘click stop button’, ‘click play button’, ‘click slider bar’ and so on, labelled in the chart as basic features. These are standard interactions that are mainly related to conventional movie playing activities as normally found in a video player (i.e. play, pause, stop, slider bar and volume adjustment). As for the advanced type of browsing, there are some ‘extra’ interactions on top of the standard playback activities. These extra features are provided in the advanced screen as well as standard movie playback (Fig. 1(b)). The result reveals that the amount of interaction of playback-related features was spread out into that of extra features in the advanced page as Fig. 2 shows. The spread of the interactions shows the tendency of the user to doing more exploration on the page. Users seem to appreciate the addon features provided in the advanced page with the increased number of interaction percentages and this result is also reflected by the increased hours spent on the advanced type with 45 hours as compared to 33 hours in the standard or basic interaction. This result indicates that the advanced features made the students stay longer on the system on the event segmentation that underlies the features. Users managed to jump from one point to another point easily on the movie using visual representations of a timeline or the shot keyframe view. Instead of browsing sequences from the normal playback interaction for example either using the pause button or slider bar, the playback of sequences shifted to playing from the shot keyframe view as depicted in the chart with the highest percentage (18%). The findings in the qualitative comments given by students also reveal a preference for these extra features (see Table 1) as well as some complaints from students about those movies with no advanced features.
572
N. Mohamad Ali et al.
It is worth noting that the interactions that have been represented in Fig. 2 are based on different movie watching. This analysis was not meant to compare between basic screen [A] vs. advanced screen [A+α] but mainly in monitoring the usage pattern of some value add-ons features in navigating and browsing of movie content and whether it would influence usage in a particular way. We also provided other features on how users make movie selections. Features included filtering movie collections based on film-director, film-genre or no filtering at all. In the percentage of log interactions, we noted only 11% of interactions filtered the movie collections based on film-director, 17% filtered based on film-genre with the rest (72%) no filtering at all. We listed all movie posters in the front page of the system by default (see Fig. 1(a)). From our observations, film genre seems an important factor in selecting a movie in the student’s actual textual analysis as the topics given by the lecturer are normally based on movie themes. Dealing with mismatch – A note-taking (or commenting) feature was incorporated as a result of initial requirements analysis but was underused and unappreciated (used by 3 students only), showing an interesting mismatch between what our users said would be beneficial and what they actually used (see Fig. 2). As this is a result that perhaps indicates how conventional usability engineering based on capturing user requirements/wishes is not sufficient in developing a novel interface, we want to analyse this point further. A follow-up email was sent to students asking questions regarding their use of notes. We asked them why they did not use the features during deployment. We got an email reply from 15 users. The reasons for not using notes features was calculated and grouped into several categories as concluded below: • • • •
“I don't want my colleagues to steal my ideas” --- Privacy issue “I like to write with pen and paper” --- Preference for conventional practice “I wanted to do it at home” --- Access limitation Interface design issue (e.g. notes button at the bottom of the screen, thus not emphasised enough)
A mismatch between students' initial wishes for a note or commenting feature and its actual usage during the trial triggered more question for us in how users' wishes collected at the requirement engineering stage should be interpreted in the context of usage rather than treated as an isolated feature in itself. 4.3 Usage over Time Students started using from the 10th Mar (week-6) of the semester. The system seems to have had quite heavy usage approaching the deadline of the assignment submission, which was on the 9th May 2008 (week-14) for both modules. It is generally believed that this pattern of usage corresponds with our previous observations that even though the topic was given early by the lecturer (i.e. week-6 for CM272), students tend to complete their assignment just before the deadline. The lecturer advises students not to do last minute assignments since reading and analysing movies cannot be done within a short duration, it needs longer time so that the skill of reading will evolve. Huge usage was found around the month of April until early May, 1-week before the deadline of the assignment. This pattern corresponds with our findings in
Developing, Deploying and Assessing Usage of a Movie Archive System
573
our user study that the process of reading and understanding movies starts by watching the movie many times before the essay can be written on paper. Another possibility from these usage patterns is that students were engaged with other assignments from other modules. Students have to follow priority deadlines. Some informal conversations with students indicated that this was strong possibility. A few email reminders were also sent to students regarding the deployment trial and we found that once an email was sent, there were some usages recorded.
Fig. 3. Student’s usage in the semester
4.4 User Opinion Out of the 107 students who accessed the system, only 60 students (56%) responded to the questionnaires that we administered within week-13 to -14. In identifying what are the features or functions students like or dislike most particularly in the system we developed, we gave students a qualitative question. Table 1 summarises several items mentioned most frequently by users on the system - both likes and dislikes. From the Table, it can be seen that note-taking was among the most frequently mentioned features that they liked, with 21 (30%) out of the total (71 mentioned) and this is followed by event categorization (24%), timeline (18%), shot keyframe view (11%) and playback of the movie (8%). These responses mainly correspond to the advanced features that had been adopted in the system. The idea of having the facility to take notes while playing a movie scene seems advantageous. We noticed that most important value of the system was simply the fact that it allowed easy access to movies in a non-linear fashion. The timeline and keyframe view which highlight where the action, dialogue and montage scenes overlap in a movie were praised as very useful, indicating that a strong temporal orientation with additional cues on the movie contents is useful as some comments show. “The timeline as it breaks down the film into the various sections - montage, action, dialogue, etc. --- this makes it easier to carry out a more in depth analysis of the movie” [P11].
574
N. Mohamad Ali et al.
“I found the combination of timeline and event categorization very useful since I can select those parts of the movie that contains the events of interest” [P2]. “I liked the way I could go directly to the exciting or montage parts” [P59]. Not only that, we also noticed some other features that our users liked from reading their comments, which we categorise under system design and access. For example, in the system design layout, comments were mainly on the well-designed layout and presentations, which makes it easy to navigate. The rest of the frequently mentioned items were on the convenience of access as an online-based application without having to borrow a DVD from somewhere else. Table 1. Frequency mentioned of system features. *convenience, **less coverage.
Features Note-taking Event categorization Visual timeline Shot keyframe view Movie playback Other: System design Access Limited movies Streaming problem Compatibility Total mentioned
System-likes 21 17 13 8 6 4 2* 71
System-dislikes 1 1 1 3 3** 10 9 3 31
Regarding those aspects that our users didn’t like (right column of Table 1), there are very few comments related to system features as there was only one mention found on each for the timeline, notes and shot keyframe view. We noticed system scalability among the issues in feedback on system dislikes. The highest frequently mentioned issue was about the limited number of movies that were stored in the library, which might restrict usage. Other comments we read were such as system compatibility (i.e. MAC user/Internet browsers). The system design (dislikes) was related to the lack of function, like not able to change the password and the advanced type of browsing did not apply for all movies. Ease of access in the ‘system-likes’ column in Table 1, was meant as a convenience factor by users, but in the ‘systemdislikes’ column, it was meant as less coverage of access. When comparing the frequency of mentioned, which were, either likes or dislikes, we noticed that no issues arose much on the “system design” and “features provided” aspects. Most of the disliked features were related to the system scalability issue. We also asked a question to the students about their overall experiences in using MovieBrowser2 after the semester had completed. Among these 60 students, 43 of them (72%) said they would use it in the future. We calculated the positive and negative expressions of their overall experiences and we estimate that 19 of them (32%) gave positive expressions and only 4 (7%) gave a negative tone of expression while
Developing, Deploying and Assessing Usage of a Movie Archive System
575
the rest of 37 (62%) did not express either positive or negative expressions. Examples of positive expressions include emotion (i.e. “I'm very happy/discovered …”) and feature usefulness (i.e. “I found it is useful/able to …”) and negative such as system limitation (i.e. “Not enough/database is too small…”). Finally, we compiled a ‘wish-lists’ from all user feedback. Some of these list elements appear due to the difficulties in the implementation and would not be expected during the development design stage. The list entries were categorized below: 1. 2. 3. 4.
Larger and varied type of movie database (i.e. Irish, Hollywood and Europe) System compatibility (i.e. Internet browsers and MAC users) Improved access (i.e. off-campus) Technicality constraints (i.e. high-speed access)
We also believe that these ‘wish-list’ elements contributed as the main reasons for low usage during the trial. Our users mainly want to access movie resources to be used in their textual essays. Having difficulties in the conventional way of assessing DVDs, means the tool is appreciated much by the students. We did not receive many complaints on the design aspects of the features we provided (i.e. navigation, colorcoding, page layout, buttons, graphics, ‘look and feel’ etc.) and these can be considered as a minor aspect. We believed that for the future, whatever the design for a similar system to this, it could be of potential benefit if these four ‘wish-list’ elements could be improved.
5 Conclusion The results presented in the trial deployment experiment highlighted some interesting patterns for students of film studies in browsing and playing movie content. User access and usage were found to be varied and influenced by many factors. In general, students found the features we provided were beneficial for their studies. Some issues or mismatches arose during the trial. A ‘wish-lists’ was drawn up that might be useful for the future system developer. Our big strength in this study was on the interactions among real users, namely students of the School of Communications, DCU throughout our 3-years of experiments. Acknowledgements. We would like to thank the School of Communications, DCU for the support given in this work. Also special thanks to Bart Lehane of CDVP for the automatic movie content analysis engine. The work was supported by the Ministry of Higher Education and University Kebangsaan Malaysia and by Science Foundation Ireland as part of the CLARITY CSET (07/CE/I1147).
References 1. Bernhaupt, R., Obrist, M., Tscheligi, M.: Usability and usage of iTV services: lessons learned in an Austrian field trial. ACM Computers in Entertainment 5(2) (2007) 2. Lanagan, J., Smeaton, A.F.: SportsAnno: What Do You Think? In: Proc. of Large-Scale Semantic Access to Content (Text, Image, Video and Sound), RIAO. Pittsburgh, PA, USA (2007)
576
N. Mohamad Ali et al.
3. Lehane, B., O’Connor, N., Smeaton, A.F., Lee, H.: A System for Event-Based Film Browsing. In: Göbel, S., Malkewitz, R., Iurgel, I. (eds.) TIDSE 2006. LNCS, vol. 4326, pp. 334– 345. Springer, Heidelberg (2006) 4. Lee, H., Smeaton, A.F., O’Connor, N., Smyth, B.: User Evaluation of Físchlár-News: An Automatic Broadcast News Delivery System. TOIS - ACM Transactions on Information Systems 24(2), 145–189 (2006) 5. McKeown, K., Barzilay, R., Evans, D., Hatzivassiloglou, V., Klavans, J., Nenkovo, A., Sable, C., Schiffman, B., Sigelman, S.: Tracking and summarazing news on a daily basis with Columbia’s Newsblaster. In: Proc. of the Human Language Technology Conference (2002) 6. Smeaton, A.F.: Indexing, Browsing and Searching of Digital Video. In: ARIST - Annual Review of Information Science and Technology, ch. 8, vol. 38, pp. 371–407 (2004)
Using Activity Descriptions to Generate User Interfaces for ERP Software Timothy O’Hear and Yassin Boudjenane Revelate SA, 38 ch. de Pré-Gentil, 1242 Satigny, Switzerland [email protected], [email protected]
Abstract. Delivering tailor-made ERP software requires automation of screen and printed report creation to be cost effective. Screens generated directly from data structures tend to have poor usability. An approach is considered using a domain specific language to describe use cases. Paper-prototyping and usability testing results define the usability characteristics the DSL portrays. The DSL is capable of defining a variety of screen types and user interface elements including forms, lists, pivot tables, Gantt charts, calendars and graphs. This approach is currently used in production to generate an interactive “AJAX” web user interface as well as HTML, PDF and Excel reports from descriptions stored in XML files. We believe that further research could extend our results to include non-ERP type software. Keywords: AJAX, domain specific language, DSL, ERP software, HTML, interaction design, paper-prototyping, usability, user interface.
1.2 Removing Dependence on Specific Rendering Technology A few years into the life of our company we had to deal with a rather traumatic event. The very first version of our application was developed with Visual Basic 6, a choice driven by a desire for maximum integration with the Windows platform. Well aware of VB6’s limitations we were betting on Microsoft’s promise of a much improved version in a couple of years. When .Net was finally released the upgrade process took a sledgehammer to every one of our screens and required months of work to fix. We realized then just how risky it is to store customer specifications inside a closed format, mixed up with platform specific code. 1.3 Improving Usability Though our windows client was generally liked by our clients, we felt that considerable improvements in both initial learning and everyday use were possible. We were particularly inspired by the work of Adam Cooper [1], Jakob Nielsen [2] and Don Norman [3]. Whatever new ideas we had would need to be prototyped and thoroughly user tested before making it into production. 1.4 Choosing a Starting Point When in 2006 we started to design the next version of Revelate it was clear that big changes were required for screen and report production. As a consequence of the above our brief included: • Screen and report layout must be fully automatic. • Screens and reports must be described independently from a specific rendering technology (e.g. Java’s Swing, .Net’s XAML, HTML or Flash). • Usability of the windows client must be improved upon. • Screen and report descriptions must be device independent; they can be rendered on anything from a mobile phone to a workstation. We simply couldn’t find much existing work even less an existent screen or report description language that matched these criteria. On the other hand Model Driven Architecture approaches appeared too abstract and generally focused on business logic rather then screens and reports. See [4], [5], [6], [7] for examples of these approaches. There are a number of user interfaces generators such [8] available that rely purely on a description of the underlying data structures. Typically, they use raw database tables or the business object domain. Our analysis of such generators is that, while the screens they generate are suitable for system administration tasks, they rarely meet reasonable usability requirements, and as such are uncompetitive with manually crafted user interfaces. This is due to the underlying data model rarely matching the user’s mental model of an application. Other generator approaches such [9],[10],[11] didn’t quite match our needs. Our existent approach required us to describe each of our clients’ use cases as part of the specification document. Hence we decided to consider the use case (or activity) as a container of screens and/or reports. We could then define the use cases with a
Using Activity Descriptions to Generate User Interfaces for ERP Software
579
domain specific language containing sufficient information to generate the actual screens and reports. This approach is examined below.
2 Hypothesis Design decisions resulting in a well crafted user interface can be systematically reproduced by an automatic generator. This is achieved by identifying individual tasks or activities and defining their key characteristics. These characteristics will not be related to a specific graphical user interface toolset, nor indeed a specific screen size or input method. Furthermore, similar characteristics can be defined for printed reports.
3 Methods We first manually “generated” a user interface according to a set of rules and user tested this with a paper prototype. This was not particularly successful so we switched to the design of a complete app
